Broken Telephone

The latest spat of conferences has seen a lot of hand waving in favour of “evidence based” ELT. Among leading figures there’s a growing consensus that, like motherhood and being kind to animals, evidence based teaching is a “good thing”. No need to get excited, though, because,

  1. All the evidence that challenges current ELT practice is still routinely ignored.
  2. Even the evidence that’s being aired is quickly adulterated.

Carol Lethaby’s talk at the recent Pavillion ELT Live! conference is a case in point. In this post I want to suggest that it’s the second link in an ongoing chain in the Broken Telphone” game, where scholarly respect for the careful presentation and use of evidence is slowly but surely abandoned.

Original Text 

The story starts in 2013, with the publication of a paper by Dunlosky et al. This paper reports on work done by cognitive and educational psychologists, and its aim is to evaluate findings of studies on 10 learning techniques. It’s important to note that this is not about L2 language learning (SLA) research, or instructed SLA research; it’s a meta-analysis of research into the explicit learning strategies used by students of content-based subjects across the curriculum. See the Tables below for details.

The paper divided the 10 techniques into 3 categories, as follows:

Techniques with “low utility” (little or no evidence supporting their efficacy)

  • highlighting and underlining content as you read;
  • writing a summary of a to-be-learned text;
  • visualising what you are reading in order to learn it;
  • using mnemonic devices such as the keyword technique to learn vocabulary;
  • re-reading a text to learn its contents.

Techniques with “high utility”

At the other extreme, distributed practice and practice testing are well known to be efficacious in certain types of explicit learning, and this is confirmed by the authors.

Techniques with “moderate utility”

  • elaborative interrogation,
  • self-explanation,
  • interleaved practice.

These got “moderate utility” assessments because the evidence is far too limited to warrant any firm claims for them. As the authors say,

these techniques have not been adequately evaluated in educational contexts.

adding that

the benefits of interleaving have only just begun to be systematically explored.

The conclusion is that

the ultimate effectiveness of these techniques is currently unknown.

First Round  

Fast forward to the 2018 IATEFL conference, where Patricia Harries and Carol Lethaby gave a talk called

Use your brain! Exploiting evidence-based teaching strategies in ELT.

A summary of the talk, by the authors, is included in the Conference Selection publication, edited by Tania Patterson. Having reminded everybody that there’s no evidence to support claims about learning styles (neuro-linguistic programming – NLP) or about putative left and right hemesphere learners, Harries and Lethaby summarised the findings of Dunlosky et al’s 2013 paper. The important thing for my argument is how they presented the findings on the 3 strategies which got a “moderate utilty” evaluation.

Elaborative interrogation: Learners explain things to themselves as they learn, for example, saying why a particular fact is true. The technique is thought to enhance learning by helping to integrate new and existing knowledge. The Dunlosky et al (2013) paper stresses that that there’s little evidence so far, and that more research is required.

Self-explanation: This is another questioning strategy. Here, students use questions to monitor and explain features of their own learning to themselves. Again, Dunlosky et al’s paper stresses that further research is needed before we can say there’s a good evidence base.

Interleaved practice: Interleaved practice involves the mixing up of difference practice activities in a single session and has been found to be effective in problem solving and categorisation. Yet again, more research is needed before any firm conclusions about its efficacy can be drawn.

Implications for ELT

When they went on to discuss the implications of these findings for ELT, Harries and Lethaby said that there were a number of ways that these techniques could be adapted. And here, I think, is where the limitations of the evidence starts to get lost. In this, the eagerly awaited climax of the talk, the presenters slip away from “What if..?” to “This is what we can do now”. Particularly enthusiastic is their endorsement of the use of “prior knowledge” (see van Kesteren et al, 2014) to design pre-tasks in receptive skills development, spiral syllabuses, and exploit L1 and L2 previous knowledge in vocab learning. This, despite the fact that research into the acquisition of new knowledge which might be integrated into pre-existing conceptual schemas has so far led to no firm findings. The presenters also talk up incorporating elaborative interrogation into teaching grammatical rules and structures, and using self-explanation to ask learners about how they found their answers in language and skills tasks. They conclude:

By spacing the practice of language items and structures using increasingly greater time intervals and mixing up language practice and skills development, distributed and interleaved practice can help integrate elements, build automaticity and aid memorisation.

All of these claims can give the impression that they’re supported by strong and robust research evidence, whereas, in fact they’re not. What Harries and Lethaby should have said was: “If we ever get convincing evidence that these techniques work in L2 learning, then we could use them in the following ways”.

To be fair, the presenters concluded:

Evidence-based strategies exist and language teachers need to be aware of them. Language  teachers and learners already use many of these and it is beneficial to know that research supports this. There is, however, a danger in a whole-scale adoption of findings from research on content-based subject areas often done in inauthentic teaching situations.

Furthermore, there’s a video of J. Harmer interviewing Lethaby at the conference. When asked to give an example of how evidence from neuroscience can help more efficacious ELT practice, Lethaby replied

“There seems to be a place in the brain where new and old information connect. We can, perhaps, in the future, when we know more about how this is done, use it to help learners assimilate new information. But we have to be very cautious, we have to be careful. This is still only potential”. 

In brief, this strikes me as an interesting talk, which points to the possible future potential of three techniques and reaffirms the evidence for the efficacy of spaced practice and practice testing. We should note that the talk extrapolates from studies of content-based school courses where the focus is on the teaching of explicit knowledge.  In the case of ELT, those of us who question a PPP, coursebook-driven approach to ELT consider such a focus to be fundamentally mistaken. Most importantly, what worries me is the discussion at the end, particularly the last bit I quoted, because it could easily be misinterpreted as saying that research evidence to date gives strong support to the claim that elaborative interrogation, self-explanation, and interleaved practice are efficacious tools for classroom use.

Second Round  

We come now to Lethaby’s presentation at the recent 2019 conference. Photos of slides from the talk and tweets during and after it suggest that Lethaby more or less repeated her 2018 IATEFL talk. But did she repeat all the caveats, I wonder? Did she make it abundantly clear that conjectures based on “a place in the brain where new and old information connect” are not evidence for the efficaciousness of anything, and that there’s still no good reason to believe that interleaving and elaborative interrogation are good L2 learning strategies? Or did she, as in 2018, rather over-egg the pudding again? What were the take aways from Lethaby’s talk, I wonder, and did she do enough to prevent people from misinterpreting what she was saying?

Third Round 

Well she certainly didn’t do enough to prevent Hugh Dellar from getting it all wrong. Soon after the start of the talk, Dellar tweeted

@clethaby laying into some neuromyths that still blight our profession: left brain right brain nonsense, we only use 10% of our brains, students learn better when presented with info via preferred learning style etc. Great stuff.

Then:

Really great talk by @clethaby unpicking the truths and lies behind neuroscientific research in ELT. It’s a real cry for deeper engagement with the actual research and for a healthy degree of scepticism, as well as plea for real evidence-based teaching.

So Dellar has already forgotten what I’m sure Lethaby told him at the start of the talk, namely that she wasn’t reporting on neuroscientific research in ELT at all. Still, it’s good to see such a sincere endorsement of cries for deeper engagement with research.

And then, we read this:

 @clethaby suggests that far more effective is spaced practice and interleaving, practice testing and eleborative interrogation – explaining to someone else how/why you got the answer to something (esp. in a guided discovery manner). It helps embed new in old.

I assume that, apart from the last sentence, this is a garbled attempt to paraphrase what Lethaby actually said, and it resembles what she said in her 2018 talk, as quoted above. Still, it shows what happens when you discuss implications of unconfirmed findings without constantly repeating that the findings are unconfirmed.

And what does that last sentence – “It helps embed new in old” – mean? It sounds as if it ought to mean something and as if Dellar thinks he knows exactly what it means. Perhaps it means “Using these four techniques helps embed new knowledge in old knowledge”. But what does “embedding new knowledge in old knowledge” mean with regard to learning an L2? This is, I presume, whether Dellar knows it or not, a reference to the van Kesteren et al (2014) article, but it’s hopelessly mangled. How is one supposed to tell when “new knowledge” gets “embedded” in “old knowledge”? What happens to the new and old knowledge? Do the two types of knowledge move from the different places they were to that place in the brain where new and old knowledge connect? How do they connect, or rather “embed”? Does the new knowledge sort of snuggle in with the old knowledge, or does it all become some different kind of knowledge? How does this process differ from learning something new? And then, since we’re interested in evidence, how do we test whether, in any particular case where a student explains to someone else how/why they got the answer to something, the “new knowledge” is “embedded” in the “old knowledge”?

We started with a 2013 meta-analysis of studies on learning techniques aimed at explicit learning in content-based courses like biology and educational psychology. We end, six years later, with someone tweeting a paraphrase of what they thought they heard someone else say about neuroscientific research in ELT. The hearer jumbles together four different techniques, two of which we know little about, and confidently asserts that teachers can use them to “help embed new in old”, whatever that means.

Fourth Round  

The next step is likely to be that the tireless globetrotter will continue on his travels, now able to assure gullable young teachers all over the world that his views on ELT are backed by “actual research” and that they represent the highest form of “real evidence-based teaching”.

Finally

The tweets that followed the conference, include this one

They suggest that Carol Lethaby should re-double her efforts to avoid being misunderstood, and that all of us should re-double our efforts to carefully scrutinise what the so-called ELT experts tell us.

References

Dunlosky, J., K. A. Rawson, E. J. Marsh, M. J. Nathan and D. T. Willingham.( 2013) Improving students learning with effective learning techniques promising directions from cognitive and educational psychology. Psychological Science in the Public Interest, 14/1: 4−58.

Lethaby also ment sources in her 2018 IATEFL presentation:

Roediger, H. L. and Pyc, M. A. (2012) Inexpensive techniques to improve education: applying cognitive psychology to enhance educational practice. Journal of Applied Research in Memory and Cognition, 1/4: 242−248.

van Kesteren, M. T. R., M. Rijpkema, D. J. Ruiter, R. G. M. Morris and G. Fernandez. (2014) Building on prior knowledge: schema-dependent encoding processes related to academic performance. Journal of Cognitive Neuroscience 26/10: 2250−2261.

5 thoughts on “Broken Telephone

  1. hi Geoff very nice post thanks;

    this tendency to mis-report what studies orginally say, or when translating from one field to another is widespread problem

    been reading a nice thesis concerning the text coverage model in reading comprehension.

    the author uses a layer analogy where – the lower layers are more ‘reliable’ than the upper layers of newer work and “previous research in text coverage studies tends to be the victim of careless recasts and synonym substitutions which, at best, obfuscates and, at worst, misrepresents the original findings.” [https://researcharchive.vuw.ac.nz/xmlui/handle/10063/6117]

    similarly i am currently trying to get my head around how SLA has been using implicit-explicit and declarative-procedural – it’s confusing!

    do you know any good texts critiquing implicit-explict and/or procedural-declarative as used in SLA?
    ta
    
mura

  2. Hi Mura,

    Thanks for this. I’ve downloaded the thesis but haven’t read it yet.

    As for how SLA has been using the 2 dichotomies implicit-explicit and declarative-procedural, it is indeed confusing, but worth trying to get to grips with because it’s at the heart of the debate about synthetic & analytic syllabus types, which in turn crucially affects ELT practice.

    And this actually relates to Carol Lethaby’s rather disappointing discussion of neuroscience, because, as you probably know scholars like Ullman and Rastelli are working on what they call a neuro-cognitive approach to SLA,which suggests that the declarative / procedural distinction is redundant.The work starts from ERP studies which record electrical fluctuations on the scalp (measured in terms of microvolts), which are time-locked to the presentation of visual or auditory stimuli, typically sentences containing errors of various kinds (syntactic or semantic anomalies). Different errors elicit qualitatively different brain responses called ‘components’ and 3 of these components are the most studied in SLA.

    Well anyway, Ullman’s declarative/procedural model, a neurocognitive framework that’s been around for some time and was last updated in 2018, suggests that declarative and procedural memories are redundant in L2 acquisition, because the same knowledge can be acquired twice by learners: statistically as chunks, and grammatically (computed by a rule). There was a Special Issue of the Second Language Research journal (2018, Issue 1) devoted to neurolinguistics where Rastelli (the main mover), Ullman and others laid out their views. I got very excited about all this after Mike Long’s review of Rastelli’s book (Discontinuity in SLA), but I’ve cooled off. Actually, to be honest, while Rastelli’s work is interesting – and a lot better than anything Lethaby mentions! – I don’t think it stacks up.

    I doubt this helps! 😦

  3. hi Geoff
    thanks for the pointers;

    apparently another example i was reading regarding the pitfalls of translating from one field to another is the case of procedural-declarative knowledge;

    originally this distinction was used in AI work to explain how knowledge is +generated+ and not as two types of knowledge +states+ – i.e. it was a distinction in AI as to was a particular item of knowledge +derived+ by procedures or from sentence-like representations.

    psychology then picked this up this distinction and in some cases it was transformed from a difference in implementing knowledge to a difference in states of knowledge.

    anyway thought that may be a relevant comment to your blog post
    ta
    mura

  4. Hi Mura,

    Thanks for this.

    It’s important to stay in the right field as the shepherd said to the dog. “The field” is often referred to in academic work as “the domain”. The domain in question is SLA. In pscholinguistic attempts to explain SLA, the terms implicit and implicit knowledge and declarative and procedural knowledge are constructs used in the service of theories. The constructs are, in themselves, theory-laden – they serve the theory that’s being constructed. For example, the construct “implicit learning” means, for those trying to explain SLA, unconscious learning of particular, well defined parts of a second language. That, and only that – nothing else – is what’s being referred to, a vital point which over-eager, clever and inquisitive people like ELT Tim, for example, fail to grasp. When Doughty (2003) says

    “In sum, the findings of a pervasive implicit mode of learning, and the limited role of explicit learning …, point to a default mode for SLA that is fundamentally implicit, and to the need to avoid declarative knowledge when designing L2 pedagogical procedures”

    she’s talking about a particular domain. Likewise, when Loewen (2015) says:

    “The fact that explicit knowledge is relatively easy to learn but difficult to use for spontaneous L2 production, and that, conversely, implicit knowledge is relatively difficult to learn but important for L2 production is, I feel, one of the most important issues in ISLA and L2 pedagogy”

    he too is talking about a particular domain. Inside that domain, the constructs “implicit”, “explicit” etc., are coherent.

    And, of course, inside that domain, there’s controversy. I think it’s fair to say that these days the controversy centres around cognitive theories of SLA which see language learning as a unique psychological process relying on innate hard wiring and those who see it as just one more example of a general learning mechanism in the brain which uses very basic mechanisms to compute statistical power laws of frequency of input.

    We must, following Gregg, start with a property theory: what is it that is acquired? I agree with Gregg that Chomsky still has the best construct: linguistic competence. And he still has the best theory of how it’s acquired as an L1: implicitly, with help from a unique human capacity to make sense of linguistic input.

    When we come to SLA, we either stay close to this view or we embrace the usage based theorists. What’s interesting is that both camps agree that language learning in general, and SLA in particular, is basically a matter of implicit, ie., unconscious learning.

  5. one of the confusing points i see is that we would like learners to +end up+ with an “implicit” use of a second language so does it follow that such ends requires a means that is similar?

    i see your point about constructs being coherent in a domain, my understanding of implicit/explicit is that the criteria in SLA is more or less the same as in psychology?

    and further that researchers such as Rod Ellis have written that they assume that the distinction is dichotomous even if there is debate in psychology over this (which i think Tim’s tweet was evoking?)

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s