Updating to BBST 4.0: Financial Model & Concluding Thoughts

This is the 5th section of my post on BBST 4.0. The other parts are at:

Financial Model

BBST 3.0 was developed with support from the National Science Foundation. That funding supported many student assistants and the logistics of the Workshops on Teaching Software Testing. It also paid for a portion (substantial, but well under half) of the time Becky and I spent on BBST development and the equipment and software we used to create it. We also received financial support from the Association for Software Testing and from Florida Institute of Technology. We all understood and agreed that with this level of public support, BBST should be open source.

We also had a vision of BBST becoming self-supporting, like Linux. Some of our thoughts are published here and here and here. That was an ambitious idea. I pursue many ambitious ideas. Some become reality, others don’t. This one didn’t.

BBST 3.0 will continue to be available under a Creative Commons License. We expect some organizations will continue teaching it. We think it’s still a good course, and we’re proud of it. To increase the lifespan for BBST 3.0, we spent an enormous amount of time over the past three years creating the BBST Workbooks. These update the assessment materials and some of the content significantly. The time we spent on BBST 3.0 Workbooks was opportunity cost for BBST 4.0. With limited time, we couldn’t start BBST 4.0 until now (now that the Test Design Workbook has gone to press).

For the future, BBST has to become a traditional commercial course. We just don’t see a way to create it or teach it without charging for it. Becky and I created Kaner, Fiedler & Associates, LLC to take over the maintenance and evolution of BBST. This is a business. It needs income, which we can then spend on instructors and on BBST.

We would very much appreciate your guidance for BBST 4.0, but please understand that the help you give us is going to a commercial project, not to an open source one.

Concluding Thoughts

The most difficult two challenges in instructional design are:

  • Assessment: Figuring out what the students understand, using activities (including exams) that help students learn more, and learn more deeply, during the assessment process
  • Focus: Limiting the scope of the course so that you can (a) fit the material in without cramming it in and (b) teach the material deeply enough that the students can get practical value from it.

Moving the programming content into supplementary instruction makes room for more content. The new content that we think is most important is a sympathetic survey of test automation, but with a frank presentation of the limitations of the common oracles. (We’re going to rely heavily on a great survey by Knott.) We can’t fit everything in that we want to include. Intimately connected to automation, in our view, is discussion of career path. We can’t fit both into the course, there’s just not enough time. Our tentative decision is to make the career path segment supplementary — we’re pretty confident that people who want this will get to it and will get what value they can from it. The automation material will be harder, which means we should support it with assessment.

We’d love to hear your thoughts on this. BBST Foundations / Bug Advocacy / Test Design took almost 6000 hours of our development time (not counting the time spent on the workbooks). Foundations 4.0 will take a lot of effort. We’d like to produce something that creates value for the profession. Your guidance will make that more likely.

2 Responses to “Updating to BBST 4.0: Financial Model & Concluding Thoughts”

  1. Based on the complexity of the changes being made, perhaps it’s worthwhile to consider a restructuring of the courses themselves. I’ve just recently finished teaching a 14-week software testing course to a class of Aboriginal students in Canada, and found that even in 14 weeks it can be a serious challenge to cover enough to get a person started in software testing. That being said, I do think the changes you’ve proposed make sense. In particular, my own views on automated testing have experienced a similar shift; there may be value in considering an automation course in the future.

    Do you plan to license the 4.0 material to the AST for their classes? I completely understand and agree that this needs to be a commercial venture, I’m just curious.

    Matt: AST and I haven’t had this discussion.
    — Cem

  2. JCD says:

    Having taken one of your classes (via AST) and read nearly all the material included in all the open BBST series. For me personally, I did not get much out of the classroom interaction. Not saying it was bad, simply not my cup of tea. So for me, having it as just a commercial classroom opportunity seems a shame. I do appreciate that you need to make money and that the input time is non-trivial, but I wonder if there can be any sort of compromise.

    Perhaps, the ability to purchase access to the videos and study materials without any certification or classroom activities at a discount? I could even see you selling your work books in 4.0 for relatively cheap while the class is more pricey. While I liked the audio-visual elements from the videos and would miss them, that might be a reasonable compromise. As a final suggestion on the topic, you could even include some sort of video download code in your workbooks which could be unlocked by the student. That way those of us who don’t need the full classroom experience can still benefit from your work.

    On a different note, one of the bigger concerns I have is that some of the industry is talking about is trying to eliminate/replace some roles with devs being responsible for the testing. Be it Microsoft or Facebook, there are big organizations who see testing as an activity rather than a role. I’m not passing judgement, but it could make what you present very different than if you are presenting to someone who has the role of tester. Since you are talking about being commercially driven, that might impact both the depth and the audience for your material. From what you said in “What We Think Should Change”, your audience appears to still be in the role of tester (even if you note that it might be a dynamic career), but with some companies changing I’m wondering what you think of this and if it will have an impact in your course work?

    – JCD

    JCD, Thanks very much for your thoughts.
    Instructional Design: We designed BBST 1.0 for self-study from online materials, just as many other course designers were doing. We discovered, as so many other educators did, that most people need interaction, coaching and feedback to stay motivated, to discover what they need to work on and to get over their barriers. BBST 2.0 and 3.0 were evolutions toward a more interactive model, but with intentional support for people who want to study on their own or small groups who want to create their own (free) course.

    BBST 4.0 will support the self-studiers and small groups by publishing the Workbooks at a very inexpensive price. These will provide lots of annotated slides and some activities and feedback. However, the videos, the application videos / transcripts / slides, several of the activities and many of the activity-updates will remain proprietary. This is partially for instructional reasons and partially to differentiate the commercial course from the inexpensive, public materials. These courses take me years to create. I have given an enormous amount of my legal and technical work away for free (or below cost) but as I age, I have to worry seriously about savings that cover my costs in retirement. Work I do has to pay for itself.

    Testers’ roles. You raise an interesting problem. My testing-related courseware will be designed to appeal to people who are going to do the majority of their current work in testing and who want to take pride in what they do. Similarly, the metrics-related courseware that I’m designing will appeal to people who want to take software-related measurement seriously and take pride in what they do. Neither of those have to be full-time roles, but the courses will tightly focus on that work. The question is, “How do we encourage people to view their roles?”

    1. From a “schools of testing” point of view, we won’t encourage them to view themselves as part of any school. We’ll explicitly teach a context-driven approach, but that’s a style of analysis, not an identity.
    2. From a “role in development” point of view, I have to find a way to talk to people that is understandable across a wide variety of roles that exist. The traditional role is still very common and many people who live inside that role find it very hard to imagine the alternatives. I don’t yet have a solution.

    Thanks again for your thoughts,
    — Cem