If you are not aware of it, Udacity is a huge provider of a type of online courses called MOOCs (Massive Open Online Courses). Recently, a founder of Udacity announced that he was disappointed in Udacity’s educational results and was shifting gears from general education to corporate training.
I was brought into some discussions of this among academics and students. A friend suggested that I slightly revise one of my emails for general readership on my blog. So here goes.
My note is specifically a reaction to two articles:
Udacity offers free or cheap courses. My understanding is that it has a completion rate of 10% (only 10% of the students who start, finish) and a pass rate of 5%. This is not a surprising number. Before there were MOOCs, numbers like this were reported for other types of online education in which students set their own pace or worked with little direct interaction with the instructor. For example, I heard that Open University (a school for which I have a lot of respect) had numbers like this.
I am not sure that 10% (or 5%) is a bad rate. If the result is that thousands of people get opportunities that they would otherwise not have had, that’s an important benefit—even if only 5% find the time to make full use of those opportunities.
In general, I’m a fan of open education. When I interviewed for a professorship at Florida Tech in 1999, I presented my goal of creating open courseware for software testing (and software engineering education generally). NSF funded this in 2001. The result has been the BBST course series, used around the world in commercial and academic courses.
Software testing is a great example of the need for courses and courseware that don’t fit within the traditional university stream. I don’t believe that we will see good undergraduate degree programs in software testing. Instead, advanced testing-specific education will come from training companies and professional societies, perhaps under the supervision/guidance of some nonprofits formed for this purpose, either in universities (like my group, the Center for Software Testing Education & Research) or in the commercial space (like ISTQB). As I wrote in a recent post, I believe we have to develop a better credentialing system for software testing.
We are going to talk about this in the Workshop on Teaching Software Testing (WTST 13, January 24-26, 2014). The workshop is focused on Teaching Advanced Courses in Software Testing. It seems clear from preparatory discussions that this topic will be a springboard for discussions of advanced credentials.
Back to the MOOCs.
Udacity (and others) have earned some ill-will in the instructional community. There have been several types of irritants, such as:
- Some advocates of MOOCs have pushed the idea that MOOCs will eliminate most teaching positions. After all, if you can get a course from one of the world’s best teachers, why settle for second best? The problem with this is that it assumes that teaching = lectures. For most students, this is not true. Students learn by doing things and getting feedback. By writing essays and getting feedback. By writing code and getting feedback. By designing tests and getting feedback. The student activities—running them, coaching students through them, critiquing student work, suggesting follow-up activities for individuals to try next—do not easily scale. I spent about 15 hours this week in face-to-face meetings with individual students, coaching them on statistical analysis or software testing. Next week I will spend about 15 hours in face-to-face meetings with local students or Skype sessions with online students. This is hard work for me, but my students tell me they learn a lot from this. When people dismiss the enormous work that good teachers spend creating and supporting feedback loops for their students—especially when people who stand to make money from convincing customers and investors that this work is irrelevant—those teachers sometimes get annoyed.
- Some advocates of MOOCs, and several politicians and news columnists, have pushed the idea that this type of education can replace university education. After all, if you can educate a million students at the same time (with one of the world’s best teachers, no less), why bother going to a brick-and-mortar institution? It is this argument that fails when 95% of the students flunk out or drop out. But I think it fails worse when you consider what these students are learning. How hard are the tests they are taking or the assignments they are submitting? How carefully graded is the work—not just how accurate is the grading, though that can certainly be a big issue with computerized grading—but also, how informative is the feedback from grading? Students pay attention to what you tell them about their work. They learn a lot from that, if you give them something to learn from. My impression is that many of the tests/exams are superficial and that much of the feedback is limited and mechanical. When university teachers give this quality of feedback, students complain. They know they should get better than that at school.
- Proponents of MOOCs typically ignore or dismiss the social nature of education. Students learn a lot from each other. Back when I paid attention to the instructional-research literature, I used to read studies that reported graduating students saying they learned more from each other than from the professors. There are discussion forums in many (most? all?) MOOCs, but from what I’ve seen and been told by others, these are rarely or never well moderated. A skilled instructor keeps forum discussions on track, moves off-topic posts to another forum, asks questions, challenges weak answers, suggests readings and follow-up activities. I haven’t seen or heard of that in the MOOCs.
As far as I can tell, in the typical MOOC course, students get lectures that may have been fantastically expensive to create, but they get little engagement in the course beyond the lectures. They are getting essentially-unsupervised online instruction. And that “instruction” seems to be a technologically-fancier way of reading a book. A fixed set of material flows from the source (the book or the video lecture) to the student. There are cheaper, simpler, and faster ways to read a book.
My original vision for the BBST series was much like this. But by 2006, I had abandoned the idea of essentially-unsupervised online instruction and started working on the next generation of BBST, which would require much more teacher-with-student and student-with-student engagement.
There has been relentless (and well-funded) hype and political pressure to drive universities to offer credit for courses completed on Udacity and platforms like it. Some schools have succumbed to the pressure.
The political pressure on universities that arises from this model has been to push us to lower standards:
- lower standards of interaction (students can be nameless cattle herded into courses where no one will pay attention to you)
- lower standards of knowledge expectation (trivial, superficial machine grading of the kind that can scale to a mass audience)
- lower standards of instructional design (good design starts from considering what students should learn and how to shepherd them through experiences that will help them achieve those learning objectives. Lecture plans are not instructional design, even if the lectures are well-funded, entertaining and glitzy.)
Online instruction doesn’t have to be simplistic, but when all that the public see in the press is well-funded hype that pushes technoglitz over instructional quality, people compare what they see with what is repeated uncritically as if it was news.
The face-to-face model of instruction doesn’t scale well enough to meet America’s (or the world’s) socioeconomic needs. We need new models. I believe that online instruction has the potential to be the platform on which we can develop the new models. But the commoditizing of the instructor and the cattle-herding of the students that have been offered by the likes of Udacity are almost certainly not the answer.
Quality – which I measure by how much students learn – costs money. Personal interaction between students and instructors, significant assignments that get carefully graded and detailed feedback – costs money. It is easy to hire cheap assistants or unqualified adjuncts but it takes more than a warm body to provide high quality feedback. (There are qualified adjuncts, but the law of supply and demand has an effect when adjunct pay is low.)
The real cost of education is not the money. Yes, that is hugely significant. But it is as nothing compared to the years of life that students sacrifice to get an education. The cost of time wasted is irrecoverable.
In the academic world, there are some excellent online courses and there has been a lot of research on instructional effectiveness in these courses. Many online courses are more effective—students learn more—than face-to-face courses that cover the same material. But these are also more intense, for the teacher and the students. The students, and their teachers, work harder.
Becky Fiedler and I formed Kaner Fiedler Associates to support the next generation of BBST courses. We started the BBST effort with a MOOC-like vision of a structure that offers something for almost nothing. Our understanding evolved as we created generations of open courseware.
I think we can create high-quality online education that costs less than traditional schooling. I think we can improve the ways institutions recognize students’ preexisting knowledge, reducing the cost (but not the quality) of credentials. But cost-reducing and value-improvement does not mean “free” or even “cheap.” The price has to be high enough to sustain the course development, the course maintenance, and the costs of training, providing and supervising good instructors. There is, as far as we can tell, no good substitute for this.