As I finish up the post-production and assessment-design for the Test Design course, I’m writing these articles as a set of retrospectives on the instructional design of the series.
For me, this is a transition point. The planned BBST series is complete with lessons to harvest as we create courseware on software metrics, development of skills with specific test techniques, computer-related law/ethics (in progress), cybersecurity, research methods and instrumentation applied to quantitative finance, qualitative research methods, and analysis of requirements.
Instructional Design
I think course design is a multidimensional challenge, focused around seven core questions:
- Content: What information, or types of information do I want the students to learn?
- Skills: What skills do I want the students to develop?
- Level of Learning: What level of depth do I want to students to learn this material at?
- Learning activities: How will the course’s activities (including the assessments, such as assignments and exams) that I use support the students’ learning?
- Instructional Technologies: What technologies will I use to support the course?
- Assessment: How will I assess the course: How will I find out what the students have learned, and at what level?
- Improvement: How will I use the assessment results to guide my improvement of the course?
This collection of articles will probably skip around in these questions as I take inventory of my last 7 years of notes on online and hybrid course development.
Objectives of the BBST Courses
The changing nature of the objectives of the BBST courses.
Courses differ in more than content. They differ in the other things you hope students learn along with the content.
The BBST courses include a variety of types of assessment: quizzes, labs, assignments and exams. For instructional designers, the advantage of assessments is that we can find out what the students know (and don’t know) and what they can apply.
The BBST courses have gone through several iterations. Becky Fiedler and I used the performance data to guide the evolutions.
Based on what we learned, we place a higher emphasis in the early courses on course skills and learning skills and a greater emphasis in the later courses on testing skills.
A few definitions:
- Course Skills: How to be an effective student. Working effectively in online courses. Taking tests. Managing your time.
- Social Skills: Working together in groups. Peer reviews. Using collaboration tools (e.g. wikis).
- Learning Skills: How to gather, understand, organize and be able to apply new information. Using lectures, slides, and readings effectively. Searching for supplementary information. Using these materials to form and defend your own opinion.
- Testing Knowledge: Definitions. Facts and concepts of testing. Structures for organizing testing knowledge.
- Testing Skills: How to actually do things. Getting better (through practice and feedback) at actually doing them.
- Computing Fundamentals: Facts and concepts of computer science and computing-relevant discrete mathematics.
You can see the evolution of emphasis in the course’s specific learning objectives.
Learning Objectives of the 3-Course BBST Set
- Understand key testing challenges that demand thoughtful tradeoffs by test designers and managers.
- Develop skills with several test techniques.
- Choose effective techniques for a given objective under your constraints.
- Improve the critical thinking and rapid learning skills that underlie good testing.
- Communicate your findings effectively.
- Work effectively online with remote collaborators.
- Plan investments (in documentation, tools, and process improvement) to meet your actual needs.
- Create work products that you can use in job interviews to demonstrate testing skill.
Learning Objectives for the First Course (Foundations)
This is the first of the BBST series. We address:
- How to succeed in online classes
- Fundamental concepts and definitions
- Fundamental challenges in software testing
Improve academic skills
- Work with online collaboration tools
- Forums
- Wikis
- Improve students’ precision in reading
- Create clear, well-structured communication
- Provide (and accept) effective peer reviews
- Cope calmly and effectively with formative assessments (such as tests designed to help students learn).
Learn about testing
- Key challenges of testing
- Information objectives drive the testing mission and strategy
- Oracles are heuristic
- Coverage is multidimensional
- Complete testing is impossible
- Measurement is important, but hard
- Introduce you to:Basic vocabulary of the field
- Basic facts of data storage and manipulation in computing
- Diversity of viewpoints
- Viewpoints drive vocabulary
Learning Objectives for the Second Course (Bug Advocacy)
Bug reports are not just neutral technical reports. They are persuasive documents. The key goal of the bug report author is to provide high-quality information, well written, to help stakeholders make wise decisions about which bugs to fix.
Key aspects of the content of this course include:
- Defining key concepts (such as software error, quality, and the bug processing workflow)
- The scope of bug reporting (what to report as bugs, and what information to include)
- Bug reporting as persuasive writing
- Bug investigation to discover harsher failures and simpler replication conditions
- Excuses and reasons for not fixing bugs
- Making bugs reproducible
- Lessons from the psychology of decision-making: bug-handling as a multiple-decision process dominated by heuristics and biases.
- Style and structure of well-written reports
Our learning objectives include this content, plus improving your abilities / skills to:
- Evaluate bug reports written by others
- Revise / strengthen reports written by others
- Write more persuasively (considering the interests and concerns of your audience)
- Participate effectively in distributed, multinational workgroup projects that are slightly more complex than the one in Foundations
Learning Objectives for the Third Course (Test Design)
This is an introductory survey of test design. The course introduces students to:
- Many (nearly 200) test techniques at a superficial level (what the technique is).
- A detailed-level of familiarity with a few techniques:
- function testing
- testing tours
- risk-based testing
- specification-based testing
- scenario testing
- domain testing
- combination testing.
- Ways to compare strengths of different techniques and select complementary techniques to form an effective testing strategy
- Using the Heuristic Test Strategy Model for specification analysis and risk analysis
- Using concept mapping tools for test planning.
This post is partially based on work supported by NSF research grant CCLI-0717613 ―Adaptation & Implementation of an Activity-Based Online or Hybrid Course in Software Testing. Any opinions, findings and conclusions or recommendations expressed in this post are those of the author and do not necessarily reflect the views of the National Science Foundation.