The latest spat of conferences has seen a lot of hand waving in favour of “evidence based” ELT. Among leading figures there’s a growing consensus that, like motherhood and being kind to animals, evidence based teaching is a “good thing”. No need to get excited, though, because,
- All the evidence that challenges current ELT practice is still routinely ignored.
- Even the evidence that’s being aired is quickly adulterated.
Carol Lethaby’s talk at the recent Pavillion ELT Live! conference is a case in point. In this post I want to suggest that it’s the second link in an ongoing chain in the Broken Telphone” game, where scholarly respect for the careful presentation and use of evidence is slowly but surely abandoned.
Original Text
The story starts in 2013, with the publication of a paper by Dunlosky et al. This paper reports on work done by cognitive and educational psychologists, and its aim is to evaluate findings of studies on 10 learning techniques. It’s important to note that this is not about L2 language learning (SLA) research, or instructed SLA research; it’s a meta-analysis of research into the explicit learning strategies used by students of content-based subjects across the curriculum. See the Tables below for details.
The paper divided the 10 techniques into 3 categories, as follows:
Techniques with “low utility” (little or no evidence supporting their efficacy)
- highlighting and underlining content as you read;
- writing a summary of a to-be-learned text;
- visualising what you are reading in order to learn it;
- using mnemonic devices such as the keyword technique to learn vocabulary;
- re-reading a text to learn its contents.
Techniques with “high utility”
At the other extreme, distributed practice and practice testing are well known to be efficacious in certain types of explicit learning, and this is confirmed by the authors.
Techniques with “moderate utility”
- elaborative interrogation,
- self-explanation,
- interleaved practice.
These got “moderate utility” assessments because the evidence is far too limited to warrant any firm claims for them. As the authors say,
these techniques have not been adequately evaluated in educational contexts.
adding that
the benefits of interleaving have only just begun to be systematically explored.
The conclusion is that
the ultimate effectiveness of these techniques is currently unknown.
First Round
Fast forward to the 2018 IATEFL conference, where Patricia Harries and Carol Lethaby gave a talk called
Use your brain! Exploiting evidence-based teaching strategies in ELT.
A summary of the talk, by the authors, is included in the Conference Selection publication, edited by Tania Patterson. Having reminded everybody that there’s no evidence to support claims about learning styles (neuro-linguistic programming – NLP) or about putative left and right hemesphere learners, Harries and Lethaby summarised the findings of Dunlosky et al’s 2013 paper. The important thing for my argument is how they presented the findings on the 3 strategies which got a “moderate utilty” evaluation.
Elaborative interrogation: Learners explain things to themselves as they learn, for example, saying why a particular fact is true. The technique is thought to enhance learning by helping to integrate new and existing knowledge. The Dunlosky et al (2013) paper stresses that that there’s little evidence so far, and that more research is required.
Self-explanation: This is another questioning strategy. Here, students use questions to monitor and explain features of their own learning to themselves. Again, Dunlosky et al’s paper stresses that further research is needed before we can say there’s a good evidence base.
Interleaved practice: Interleaved practice involves the mixing up of difference practice activities in a single session and has been found to be effective in problem solving and categorisation. Yet again, more research is needed before any firm conclusions about its efficacy can be drawn.
Implications for ELT
When they went on to discuss the implications of these findings for ELT, Harries and Lethaby said that there were a number of ways that these techniques could be adapted. And here, I think, is where the limitations of the evidence starts to get lost. In this, the eagerly awaited climax of the talk, the presenters slip away from “What if..?” to “This is what we can do now”. Particularly enthusiastic is their endorsement of the use of “prior knowledge” (see van Kesteren et al, 2014) to design pre-tasks in receptive skills development, spiral syllabuses, and exploit L1 and L2 previous knowledge in vocab learning. This, despite the fact that research into the acquisition of new knowledge which might be integrated into pre-existing conceptual schemas has so far led to no firm findings. The presenters also talk up incorporating elaborative interrogation into teaching grammatical rules and structures, and using self-explanation to ask learners about how they found their answers in language and skills tasks. They conclude:
By spacing the practice of language items and structures using increasingly greater time intervals and mixing up language practice and skills development, distributed and interleaved practice can help integrate elements, build automaticity and aid memorisation.
All of these claims can give the impression that they’re supported by strong and robust research evidence, whereas, in fact they’re not. What Harries and Lethaby should have said was: “If we ever get convincing evidence that these techniques work in L2 learning, then we could use them in the following ways”.
To be fair, the presenters concluded:
Evidence-based strategies exist and language teachers need to be aware of them. Language teachers and learners already use many of these and it is beneficial to know that research supports this. There is, however, a danger in a whole-scale adoption of findings from research on content-based subject areas often done in inauthentic teaching situations.
Furthermore, there’s a video of J. Harmer interviewing Lethaby at the conference. When asked to give an example of how evidence from neuroscience can help more efficacious ELT practice, Lethaby replied
“There seems to be a place in the brain where new and old information connect. We can, perhaps, in the future, when we know more about how this is done, use it to help learners assimilate new information. But we have to be very cautious, we have to be careful. This is still only potential”.
In brief, this strikes me as an interesting talk, which points to the possible future potential of three techniques and reaffirms the evidence for the efficacy of spaced practice and practice testing. We should note that the talk extrapolates from studies of content-based school courses where the focus is on the teaching of explicit knowledge. In the case of ELT, those of us who question a PPP, coursebook-driven approach to ELT consider such a focus to be fundamentally mistaken. Most importantly, what worries me is the discussion at the end, particularly the last bit I quoted, because it could easily be misinterpreted as saying that research evidence to date gives strong support to the claim that elaborative interrogation, self-explanation, and interleaved practice are efficacious tools for classroom use.
Second Round
We come now to Lethaby’s presentation at the recent 2019 conference. Photos of slides from the talk and tweets during and after it suggest that Lethaby more or less repeated her 2018 IATEFL talk. But did she repeat all the caveats, I wonder? Did she make it abundantly clear that conjectures based on “a place in the brain where new and old information connect” are not evidence for the efficaciousness of anything, and that there’s still no good reason to believe that interleaving and elaborative interrogation are good L2 learning strategies? Or did she, as in 2018, rather over-egg the pudding again? What were the take aways from Lethaby’s talk, I wonder, and did she do enough to prevent people from misinterpreting what she was saying?
Third Round
Well she certainly didn’t do enough to prevent Hugh Dellar from getting it all wrong. Soon after the start of the talk, Dellar tweeted
@clethaby laying into some neuromyths that still blight our profession: left brain right brain nonsense, we only use 10% of our brains, students learn better when presented with info via preferred learning style etc. Great stuff.
Then:
Really great talk by @clethaby unpicking the truths and lies behind neuroscientific research in ELT. It’s a real cry for deeper engagement with the actual research and for a healthy degree of scepticism, as well as plea for real evidence-based teaching.
So Dellar has already forgotten what I’m sure Lethaby told him at the start of the talk, namely that she wasn’t reporting on neuroscientific research in ELT at all. Still, it’s good to see such a sincere endorsement of cries for deeper engagement with research.
And then, we read this:
@clethaby suggests that far more effective is spaced practice and interleaving, practice testing and eleborative interrogation – explaining to someone else how/why you got the answer to something (esp. in a guided discovery manner). It helps embed new in old.
I assume that, apart from the last sentence, this is a garbled attempt to paraphrase what Lethaby actually said, and it resembles what she said in her 2018 talk, as quoted above. Still, it shows what happens when you discuss implications of unconfirmed findings without constantly repeating that the findings are unconfirmed.
And what does that last sentence – “It helps embed new in old” – mean? It sounds as if it ought to mean something and as if Dellar thinks he knows exactly what it means. Perhaps it means “Using these four techniques helps embed new knowledge in old knowledge”. But what does “embedding new knowledge in old knowledge” mean with regard to learning an L2? This is, I presume, whether Dellar knows it or not, a reference to the van Kesteren et al (2014) article, but it’s hopelessly mangled. How is one supposed to tell when “new knowledge” gets “embedded” in “old knowledge”? What happens to the new and old knowledge? Do the two types of knowledge move from the different places they were to that place in the brain where new and old knowledge connect? How do they connect, or rather “embed”? Does the new knowledge sort of snuggle in with the old knowledge, or does it all become some different kind of knowledge? How does this process differ from learning something new? And then, since we’re interested in evidence, how do we test whether, in any particular case where a student explains to someone else how/why they got the answer to something, the “new knowledge” is “embedded” in the “old knowledge”?
We started with a 2013 meta-analysis of studies on learning techniques aimed at explicit learning in content-based courses like biology and educational psychology. We end, six years later, with someone tweeting a paraphrase of what they thought they heard someone else say about neuroscientific research in ELT. The hearer jumbles together four different techniques, two of which we know little about, and confidently asserts that teachers can use them to “help embed new in old”, whatever that means.
Fourth Round
The next step is likely to be that the tireless globetrotter will continue on his travels, now able to assure gullable young teachers all over the world that his views on ELT are backed by “actual research” and that they represent the highest form of “real evidence-based teaching”.
Finally
The tweets that followed the conference, include this one
They suggest that Carol Lethaby should re-double her efforts to avoid being misunderstood, and that all of us should re-double our efforts to carefully scrutinise what the so-called ELT experts tell us.
References
Dunlosky, J., K. A. Rawson, E. J. Marsh, M. J. Nathan and D. T. Willingham.( 2013) Improving students learning with effective learning techniques promising directions from cognitive and educational psychology. Psychological Science in the Public Interest, 14/1: 4−58.
Lethaby also ment sources in her 2018 IATEFL presentation:
Roediger, H. L. and Pyc, M. A. (2012) Inexpensive techniques to improve education: applying cognitive psychology to enhance educational practice. Journal of Applied Research in Memory and Cognition, 1/4: 242−248.
van Kesteren, M. T. R., M. Rijpkema, D. J. Ruiter, R. G. M. Morris and G. Fernandez. (2014) Building on prior knowledge: schema-dependent encoding processes related to academic performance. Journal of Cognitive Neuroscience 26/10: 2250−2261.