Feathercap Blog

The failure of on demand courses

Apr 23, 2019 12:57:00 PM / by The Feathercap staff



Back in 1984, Educational Psychologist Benjamin Bloom posed a challenge right in the title of his research. “The 2 Sigma Problem: The Search for Methods of Group Instruction as Effective as One-to-One Tutoring”. He contended that one-on-one tutoring produces two standard deviations – 2 sigma – of improvement to students learning a given subject versus those in a classroom. That means if they got a “C” previously while in a classroom setting they would get an “A” after learning the same subject from a one-on-one coaching approach. Where he defined a “classroom” akin to a group of learners all receiving the same training without as much back and forth interaction as a one-on-one tutor. This is suspiciously like our existing on demand courses we have all experienced. Why are on demand courses failing? Put another way, the conventional course or static on demand experience we’ve known achieves on average 20% of the entire learner audience attaining highest achievement of the presented material. Contrast this with Bloom’s 2 sigma improvement through one-on-one tutoring, giving 80% of the entire learner audience attaining that highest achievement. A one-on-one tutoring approach yields 60% more of the learners achieving mastery!

Can we replicate one-on-one tutoring but without the tutors? What Bloom didn’t know is how relevant this would be to workplace training and to be able to predict our current content curation, learning and knowledge sharing capabilities and how that could affect replicating his 2 sigma improvement. Bringing us back to today there are so many new possibilities in terms of being able to quickly transform any existing organizational documents and content (MS Powerpoints, videos and images, standard operating procedures (SOPs), etc) and share with most anyone. Or very rapidly creating the content from scratch itself as a form of shared knowledge content. Then bringing any of this to the right learner exactly when they need it. But not just in the form of one long on demand or classroom course; hand build by subject matter expert and instructional designers. But having the spirit and skill of those instructional designers imparted into learning and knowledge sharing systems by gathering vast amounts of metrics and data which assists these systems to react to a learners situation. How long have they been on the job, what their peers found useful to achieve a desired job behavior and what content has both the highest regard by the expertise of the author along with how current the learning content is. These systems rely on the learning content to be more atomic and small for more relevancy and reusability. After all the smaller the content the more precise the interaction can be for a given learner. Just like one-on-one tutoring. These are just a few examples of ways and metrics modern learning and knowledge sharing systems are starting to gather. They become a feedback loop with learning departments gaining a visibility to what’s working and what’s not as never before. 

One recent development we do have to watch out for is when our learning audience feels overly compelled or feel coerced to interact with personalized on-demand learning systems in a single modality, such as solely through computer screen versus being able to freely interact with colleagues or subject matter experts. One unfortunate example comes from a well meaning learning company trying to improve efficiencies in teaching middle and high-school students. Though perhaps more efficient for saving teachers time, the data to support better student outcomes versus a normal classroom is still not clear. The students and parents themselves have objected to the negative effects of using the company's on-demand learning system by using it for multiple hours a day. Ailments have been widely reported such as wrist pain, eyestrain, loneliness and more through the use of such systems. Certainly this is a cautionary tale. 

However, if we imagine these systems working hand in hand with instructional designers and subject matter experts able to replicate the one-on-one tutor experience Bloom envisioned above but perhaps involving multi-modal learning combined with content, interactions, on the fly assessments and checklists by managers at exactly the right moment based on the learners mastery. Even Slack messages, email messages and suggested peer or manager interactions automatically scheduled and brought in as a learning conversation between learner and teacher - in this case the teacher is the learning system itself. This is the ultimate expression of the one-on-one tutor experience and frankly if anywhere near the above results hold a huge improvement over our existing long form on demand courses.

In our latest AI and the augmented workforce primer, we talk about how the above technologies will enhance the capabilities of so many organizations around the world. Instead of only having time to manage existing learning systems, LMSs, content repositories, 3rd party learning content and compliance training, organizations will be able to focus on departmental learning required by all employees to do their day to day jobs. We don’t see the above capabilities as an end to learning teams but rather a diminishing of static on demand learning courses which of course is not a new idea, at least according to Benjamin Bloom.

See our AI and the Augmented Workforce primer on how a workforce and technology can effectively work on tasks together. 

A big driver for us at Feathercap is our search and workplace learning content curation solutions. Feel free to contact us for more. 

Topics: Blog, workplace learning, classroom learning, Instructor led training, ILT, 3rd party course, ondemand course