The Dispatch
The innovation trap
What happens when technological innovation in higher education doesn't add real value?
By Claudia Civinini
“I used to get geeks calling me up in the early 2000s claiming they’d solve the language learning issue by next Thursday,” recalls Melanie Butler, Consultant Editor of the EL Gazette in the UK.
“They didn’t, of course.”
At the helm of a magazine for English language teachers for over 30 years, Butler has witnessed her fair share of overhyped innovations come and go. Video materials were an improvement over audio for language learners, she says: “It's much harder to understand what's going on if you've only got an aural input and not a visual input as well.”
However, other innovations didn’t add any real value. Some of the new tools and methods, Butler observes, ended up repackaging old methodologies – some still effective, some outdated.
“Often, in tech, if the company doesn’t have the specialists on board, they end up reinventing the wheel. That’s a bit of a failure, for an innovation,” she explains.
It’s a well-known fact, at least according to the internet, that education needs innovation. There are several areas that need innovative solutions: for example, widening access, reducing outcome inequality and special needs education. But a future characterised by on-demand, hyper-flexible education for all hasn’t really materialised yet, nor have new tools completely changed and optimised the way we teach and learn. At the same time, some basic principles of good teaching, and some old tools, are still effective, and don’t necessarily need to be replaced.
And when it comes to tech, one may wonder whether, in some instances, costs justify the added value. However, as tech is part of everyday life and hard to keep out of the classroom, the key is to take sensational claims with a pinch of salt.
As Dr Arran Hamilton and Professor John Hattie wrote in a 2022 paper: “We do not advocate turning back the clock to the time before the integrated circuit board existed, but we do advocate selecting technology wisely and not assuming it is a silver bullet that will transform everything.”
When considering new innovations in their teaching and learning practice, teachers should look to the evidence to really interrogate their potential impact.
When considering new innovations in their teaching and learning practice, teachers should look to the evidence to really interrogate their potential impact.
"Our current practices around innovation can sometimes set educators up for failure"
The devil in the caveats
While some innovations are successful, many are not, or don’t deliver as intended.
“Many approaches framed as innovation may not have a beneficial impact over and above what schools are already doing,” Professor Becky Francis CBE, Chief Executive of the Education Endowment Foundation in the UK, explains.
“Our focus at the Education Endowment Foundation is on education approaches built on evidence. We’ve evaluated many programmes or approaches and found they don’t lead to any improvements in pupil learning. Some even have a negative impact.
“This is one of the many reasons that taking an evidence-based approach in education is so important. When considering new innovations in their teaching and learning practice, teachers should look to the evidence to really interrogate their potential impact. If they decide to adopt new approaches, then the focus needs to be on how the innovation will be properly implemented to maximise its chances of success.”
It is not uncommon for innovations to not deliver: innovations and interventions are born, tested out, and evaluated – and some will not survive the process. In 2018, an impact evaluation of the innovations and interventions founded by the Education Department in the US discovered that only 18 out of 67 had a positive impact on student achievement.
There are many reasons behind this. Some of these reasons begin at inception in the idea behind the innovation itself, or its design, while others come out of the woodwork during implementation or adoption.
Some innovations require teachers and students to adapt to a totally new way of teaching and learning, but they are introduced too fast. Others don’t take into account what the evidence shows about pedagogy. Some tools and methods will work well in some contexts and not in others. Others are not implemented correctly. Others still are so overhyped they couldn’t possibly live up to the promise.
It would be wrong to say that education doesn’t need innovation. But the opposite statement is too vague – several caveats should be added, encompassing implementation process and speed, evidence, teacher training, context, culture, and many others. The devil is in the details, and the reason innovations fail to deliver is in these caveats.
The hype trap
“Online and technology-enabled delivery has been a mixed bag. It was overhyped because it promised that ‘we’ll just put the content out and it will solve everything, including access’,” comments Dr Michael Horowitz, Chancellor of the Community Solution Education System in the US.
One of the traps innovations can fall into is overhype. This is something of a widespread problem with tech tools and digital learning. As the president of the Chicago School from 2000 to 2010, Dr Horowitz recalls that he implemented online learning early on. However, he says: “We found that we had to spend a lot of time developing the courses and training faculty to teach in that way. The cost-saving element was also overhyped to some extent. And there were a lot of for-profit education companies misapplying it by doing mass courses in a kind of robotised way.
“As you got massive adoption without the attention to the student and the teacher, you predictably got higher and higher dropout rates.”
One of the extremes, he says is the MOOC concept. “In MOOCs, you had a small group of people proceeding and getting a lot out of it, but most were clicking and not getting much done. It’s like putting a massive library out there, but it doesn’t mean that people are going to try, persist and learn,” Dr Horowitz says.
Accidentally hybrid
Hybrid learning was also somewhat overhyped, Steve Muylle, Professor of Digital Strategy and Business Marketing and Associate Dean for Digital Learning at Vlerick Business School in Belgium, explains. “Hybrid learning was overpromised in the sense that it was going to be the future. And it offers flexibility and convenience – that’s what students are looking for.”
However, he adds: “We have learned that that can come at the expense of effectiveness.”
The tech required for hybrid learning, as well as IT support, can have a substantial price tag. How that technology is used determines its effectiveness: as for most other innovations, the key is in the pedagogical approach.
Muylle explains that there can be two hybrid learning modalities: hybrid by design, and hybrid by accident. The first option involves careful pedagogical design and the right tech. It worked well in Muylle’s institution; however, he says, the cost goes up “tremendously” as human moderation and facilitation are required to effectively support the online learner experiences.
The second, hybrid by accident, doesn’t work as well, but is also the most common. “What we see today is that lots of schools, especially in Europe, have those installations in their classrooms but they are only using them if someone can’t join the class in person and wants to join remotely. That’s hybrid by accident,” he says.
“What we have noticed there is that the learning experience is not that great for the online audience. They appreciate the fact that they can join the class, but they're not really engaged like the in-classroom audience.”
The best approach, he explains, is for participants to be together in the same space, whether in the classroom or online – preferably with a blended approach. Blended learning, he explains, offers the best of both worlds.
“On the one hand, you have the advantages of digital in the sense that you can offer participants or students self-paced learning materials. And then, in class, you can have experiential learning. So, there is no silver bullet, but I think the best is a combination of the two,” he says.
“And it’s scalable. Once you develop and design the module, it’s extremely scalable.”
Screen inferiority
Another common innovation that can bring a hefty price tag and its dose of hype is one-to-one device programmes, either with laptops or tablets. These programmes have been around for a while, but their adoption was accelerated by the pandemic.
According to Dr Hamilton and Professor Hattie, laptops are ‘likely to have a small positive impact on student achievement’. Research on the overall effectiveness of one-on-one devices on learning outcomes yields mixed results – this is possibly because it all depends on a range of different factors, for example, age, setting, pedagogy, task design, access to support.
To Ayman Abdel Rahman, CEO at Egypt-based ebook services provider, Kotobee, this is another example of an innovation that overpromised and underdelivered: “While the idea was to enhance learning through technology, it often overlooked the fact that traditional methods like pen and paper can be just as effective, if not more so, for certain tasks. For instance, studies have frequently shown that students retain information better when taking handwritten notes compared to typing,” he explains.
“Additionally, the overuse of devices can lead to distractions and reduce face-to-face interactions, which are crucial for developing communication and social skills."
Some research, published in Frontiers, found that pen-and-paper note-taking may be more beneficial as it requires deeper processing of the information and can support better retrieval and accuracy.
On-screen reading is also a much-scrutinised field. “We now know that reading digitally can result in weaker reading comprehension than reading on paper,” explains EL Gazette’s Butler.
The literature on the so-called screen inferiority effect is quite extensive. While some studies have found more mixed results, depending on the context and students’ reading ability, for example, several findings suggest that reading on paper rather than on a screen supports reading comprehension and learning.
The speedy revolution trap
Another consideration is the speed in which innovations are introduced, and whether students and teachers to are expected to adapt to new tools and methods in unrealistic timeframes. Especially with technology, the process of adaptation required of students, teachers and systems is complex.
The step between a blackboard and a whiteboard, for example, can arguably be less complex than that between a book and a tablet. “A whiteboard is an adaptation and an improvement of something we were already used to,” Butler explains.
“If we take digital reading instead, we have been reading on paper for a very long time, and it might take us a while to learn to read on screen and retain information in the same way.”
It’s plausible it may take a while for humans to read on screen as well as they do on paper, as it’s certain that it takes a while for teachers to adapt to using new tools. Dr Justin Reich, author, among other things, of a book titled Failure to disrupt: Why technology alone can’t transform education, explains in a 2021 paper how the process of adapting to teaching with tech is slow.
“At first, teachers tend to use new technologies to extend existing practices. Only with time, practice, experimentation, and support do they move on to more novel applications,” he writes. “Thus, every technology solution is also a human capital problem: new education technology tools are only as powerful as the communities that support their use.”
The constant novicehood trap
Innovation can also come in the form of new curricula, methodologies, approaches and philosophies. But the same principles apply as with technology: some innovations are bound to fail from the beginning because they don’t have a sound evidence base (for example, approaches based on neuromyths); other innovations are based on evidence, but can still fail during the implementation phase.
For Jess Davis, a Montessori teacher, Assistant Head of School, teacher trainer and consultant with Montessori Minds in the US, innovation in education reflect trends that come and go, and that is problematic. The first problem is in the inception: innovations, she explains, are usually driven by a new body of research in a specific area, generating a “hyperfocused” all-or-nothing approach that brands all old practices as worthless. “This is an oversimplistic view of education. We serve all sorts of learners and need a deep and varied toolbox to meet their needs,” she explains.
The second problem is in the implementation. “In my experience, new innovations are underfunded and are implemented with only minimal training, and then they are judged for effectiveness in a short period of time.
“Our current practices around innovation can sometimes set educators up for failure,” she says.
Davis taught in the public sector, but moved to the private Montessori system partly in search of more stability. During her time in the public sector, she recounts, the reading curriculum was changed twice and the maths curriculum three times. “Each [of these changes] included a workshop that was one to three days long. And then we were supposed to completely throw out our old curriculum and replace it,” she reflects.
“I found that it was speed, more than any particular curriculum system by itself, that caused there to be ill will among teachers who were constantly having to navigate the changes without a lot of support. This keeps teachers in a state of constant novicehood. This is the biggest concern to me, versus any particular trend or innovation.”
The childminder trap
Ultimately, there are several basic pedagogical principles and methodologies that have been found to be effective, and ignoring them can set an innovation up for failure.
“We know there are some broad principles that underpin effective pedagogy, such as high-quality feedback and strategies to support metacognition. While most innovations are unlikely to improve teaching and learning in their own right, they can be effective if used to support pedagogy and practice,” explains Professor Francis.
“For example, if we take technological innovations, providing a tablet for each pupil is unlikely to improve learning outcomes. But if used in ways to help improve existing practices, such as increasing the precision with which feedback on misunderstandings is provided, they stand a much better chance of doing so.”
Dr Horowitz observes that some of the keys to success for students can be considered as traditional, such as motivation and time spent on the task, and that human relationships are still key.
“The Community Solution Education System… brings six distinct universities together in one collaborative system. More exchange of ideas unmediated by technology is a fundamental thing in higher education and education. We need to preserve space for that,” he says.
For Butler, some innovations seem to be driven by a desire to save money by replacing trained teachers “with a laptop and a childminder”. But sound pedagogy and teachers are still the key factors to make any innovation work. “As yet, I still need to see any research anywhere that doesn’t have good teaching as the top factor, or one of the top ones,” she says.
Hopefully, AI-powered innovations won’t fall into the trap of trying to replace teachers.