John Hattie: your research is now a con

In this article I am going to place considerable demands on Simon Collins’ flexibility of thinking, on his ability to imagine the unimaginable, his willingness to recognise a wrong universally accepted right by the media as an incontrovertible right, and the naïve trust he no doubt shares with his media colleagues in the truthfulness of education authority.

The prompting for this posting is a follow-up article Simon Collins wrote in the New Zealand Herald  (March 18, 2017) to an earlier major one detailing New Zealand’s declining international test results.

Simon Collins asked, as the media always do, an academic – described as a top educator – why he thought this was happening.

The academic was adamant: ‘We have more streaming than any other country in the world.’

He must be dreaming!

Well he does now live in Australia.

I have come to know this academic very well and am confident in declaring that he was winging it.

The colossal irony is that it was the actions and words of the academic being asked, more than any other academic, more than any other New Zealander, who caused the decline in international test results.

A decided motivation to distract.

How this academic must have laughed.

Oh what a humble amanuensis Simon Collins proved to be.

How this academic, this top educator, must have enjoyed Simon Collins falling for it.

The top educator went on to say, we used to be at the top but now we weren’t, and the reason is ‘streaming children into top, middle, and bottom classes’. (The top educator, as ever, has it wrong: the huge preponderance of streaming doesn’t occur in classes but within them, a fact that would be second nature to a true top educator.)

What a fiddle the top educator was getting away with.

I have long written about the advantages of unstreamed learning and taken courses on how to organise classroom learning without it (for it to work at its best, holistic practice is required) but its use has only miniscule harmful effects on learning compared with the widespread structural effects of the introduction of national standards, one of them, not incidentally, being an increase in streaming.

In an article in the Herald (February 6, 2010), Andrew Laxon writes ‘Bill English sought Hattie’s views when he originally developed the party’s national standards policy and Key took the same route, drawing inspiration from Hattie’s advice that a standards-based approach could work wonders in even the poorest schools.’

In that same article Andrew Laxon writes: ‘Hattie’s sworn enemy in all educational matters, former school inspector-turned-blogger Kelvin Smythe is far more forthright.’

‘He believes that Hattie has worked hand in glove with the Government on the system and claims his influence is so great that other academics are too scared to speak up against him.’

John Key said John Hattie was the architect of the introduction of national standards into New Zealand. He became the overwhelmingly dominant academic supporting government policies. To those of us who wanted truth in education, who wanted progressive policies, who wanted genuine discussion, Hattie was an academic miasma. He was everywhere. His research was the rationale for government policy and in combination with the government and a bewitched media, a devastatingly formidable force to confront.

But the research, as I hope to convincingly demonstrate, is false.

Now this is the challenge to Simon Collins, instead of investigating declining test results and interviewing an academic (always the same one), I suggest he investigate the academic.

I know what Simon Collins will think – Kelvin Smythe is he unhinged? Hattie, a former University of Auckland professor, producer of the Holy Grail research, still dominant with the National government, presently head of the Melbourne Education Research Institute, the dominant academic voice in Australia, consultant to Pearson, the go-to academic world-wide.

Just as Fukuyama declared the end of history so, in a way, has Hattie’s research been declared the end of education research.

But that would be strange wouldn’t it given the name of the book of his research: Visible Learning? A book of research that concerns itself only with that which is visible? A perfect model for the neoliberal commodification of education but a gargantuanly imperfect one for research on children.

Despite all that, Hattie’s education research remains just as false – all it goes to show is Hattie’s fantastic charismatic, testosterone driven-ability to sell trombones and employ his cunning political skills.

I viewed him once talking to a group of students, nearly all women, and Hattie standing at the board saying: When you become a teacher, would you like to teach with the absolute certainty of being right? A swoony intake of breath was the response.

But his research is demonstrably false.

Hattie’s refusal to acknowledge the falseness of that research – undertaken as a young researcher, harmful to hundreds of thousands of children – means it has become akin to academic fraud. I don’t believe the research was intended as an academic fraud but after getting the research so wrong, and having it pointed out to him, his refusal to disown it, means he has to wear its falseness.

A number of British academics have exposed him and breaking through the fear of Hattie’s close relations with the government including his involvement in PBRFs, a few New Zealand academics.

(I have written a number of lengthy articles tracking the directions Hattie took and was a great nuisance to him. On one occasion he said he would only sign an academic petition if the academics could get me to shut up. I agreed on the condition I could open up if he went against the arguments of the petition signed, which I knew he would – and which he did. To a visiting English academic he complained about harassment and there were vague references to legal action.)

The falseness of Hattie’s research is not difficult to explain; it is not technical or abstruse. But everything about Hattie’s research is false except for some opinions which, while they may be true, are also false, because he claims them to be evidence-based. One reason Hattie has an appeal to teachers (along with his assertive sense of rightness) is that he talks about ideas and approaches which are at teachers’ level of interest. His tactic is to mix, amongst his mainly right-wing education ideas, a few liberal ones – that is Hattie the education politician at work, bringing teachers in, making him all the more valuable to corporates and neoliberal governments. Hattie always recognised that his status with authority and corporates was partly dependent on his hold over teachers, hence they tolerated the occasional radical swing in support for certain policies, in particular, national standards. He signed a petition, for instance, against national standards even though he was the ‘architect’ of them, soon to change tack of course; and just before he left for Australia he said the national standards were the wrong ones – a very odd response for the architect.

Hattie took his idea for meta-analysis from the medical world and applied it to the value-laden one of education in gigantic style.

The difference between, say – a study of hospital operations involving surgical mesh and complications, and a study of the learning effects of, say, whole language involving many countries and thousands of children; children of different ages; children of different genders; children ranging from those with disabilities to able ones to university students; children in classrooms to clinical situations; teachers and cultures with different understandings of  whole language; studies over different time periods (but always short); lessons that are formal to less so (definitely a huge bias towards the formal) – is vast.

No-one else in education has tried a similar meta-analysis since, for an obvious reason, in education it throws up rubbish – there is no-way the variables can be controlled:

  • Fifty-thousand studies with the estimated number of 236 million students across many countries, though with a USA bias (every country to give a different meaning to the same words – for instance, whole reading and whole language meaning something very different in the USA to New Zealand).
  • Age (children who are younger are capable of much more rapid improvement than older children; and what is the relevance of studies of university students to the teaching of school children?)
  • Ethnicities (single ethnicity countries as against highly diverse).
  • Schooling systems (authoritarian countries as against democratic; technocratically advanced countries as against developing ones).
  • Learning contexts (classrooms or laboratories – laboratory and clinical contexts play a substantial role in the research results).
  • Student characteristics (a significant number of the results are based on children with learning disabilities).
  • Teaching styles (dominant characteristics from country to country).
  • Vocabulary (teaching practices and their names have different meanings in different countries).
  • Parts of the curriculum (certain parts of the curriculum suit different styles of teaching; what is being taught is not always made clear but mathematics, which can be taught more formally, is a definite emphasis).
  • Variation in aims from country to country (certain aims in education which might be of high importance to children and to particular societies might be more complex to teach and therefore take longer – for instance, whole reading is superior to phonics in the longer term but requires patience, the same with problem-based mathematics).
  • The affective (largely avoided as the name of the book states).

To take one of Hatties’s influences, feedback, there is an absence of easily accessible research information on:

  • The learning contexts used.
  • The institutional level involved (pre-school to university).
  • Whether the students were in a clinical or classroom context.
  • The curriculum areas involved.
  • Whether the thinking was straightforward or complex, skill-based or cognitive, formal or affective.
  • How it was taught, closed or open-ended, individual discovery or class led.
  • The age of the children, ethnicity, socio-economic status, or gender balance.
  • How long after the teaching the testing was done (of overwhelming significance).
  • The influence of the Hawthorne effect.
  • Whether the children were performing in comparison with where they were at the beginning or with a control group (ponder the enormous research significance of that – yet the results were all tumbled together).
  • How each influence has an independent effect entirely separate from the context (a more technical area of research error but pervasively significant) as he claims.

The crucial point is not that most of the information couldn’t, eventually, be identified but the impossibility of ever bringing the variables validly together. My assumption, given Hattie’s personality, is that he felt he could get away with it by deus ex machina and he largely has. Hattie, as demonstrated below, sometimes fragmented the data to often farcical effect and wickedly misleading outcomes but, it seems, we fell for deus ex machina again. Where is the outrage?

An academic dug deep down into the research information and came up with the following (there are dozens more just as grotesque):

  • Hattie’s ranking of quality of teaching (to the question: how important an influence is it – want to drive a bus through what quality means?), which he ranked 56th out of 138 in his league table (56th is he for real?), came solely from student ratings by college and university students. Can you believe this rubbish? How could Hattie not have known it was a con?
  • The micro-teaching influence was ranked 4th as an influence, but only came from pre-service teachers.
  • Professional development was ranked 19th as an influence but drops to 48th when only schools research is included.
  • Formative evaluation was ranked 3rd as an influence but there were only two sets of results and both to do with special education children.
  • Comprehensive interventions were ranked 7th as an influence but they only concerned learning disabled children.

British academics proved half his results were mathematically incorrect due to a school-level howler, a demonstration of his amateurism. It took him years to acknowledge the error even though it was self-evident. What chance then of acknowledging the falsity of his research as a whole. It is important to point out, though, the other way he worked out the results produced ‘correct’ results.

I searched all his writing to see if there was any kind of apology, any kind of counter, and I found only one semblance:

Hattie wrote: ‘No worry, it all balances out.’

Our professional lives are enriched when academics create education ideas greater than their research. Hattie, sadly, has managed the considerable feat of producing education ideas even more dismal than his specious research. There is a nimble but superficial prolixity to his writing that indicates the possession of a critical intelligence which can operate with no fixed connection to the reality of classrooms or their social context. There is, though, one exception to this, his ability to connect to the reality of the academic market. His tactically adroit research is angled and presented in such a way as to draw teachers in with its certainty, the media with its glibness, corporates with its marketability, and governments with its promise of increased control at the cheapest rates. I have written hundreds of pages about this academic, an academic who has played such major part in shredding the beautiful holistic education that is our heritage and culture. I have not written about him for some time and intend never to write about him again.

John Hattie: your research is now a con.

As for Simon, as eminent, skilled, and good-hearted as he is, taking on Hattie is beyond his reach, anyone’s, because Hattie is an embodiment of where school education is today he has, it seems, a free pass to any neoliberal outrage that takes his fancy, with truth and children the ultimate victims.

This entry was posted in Hattie and tagged , , , . Bookmark the permalink.

8 Responses to John Hattie: your research is now a con

  1. Thanks Kelvin for another detailed analysis of Hattie.
    Professor John O’Neill also identifies Hattie’s other motives:

    “public policy discourse becomes problematic when the terms used are ambiguous, unclear or vague” (p1). The “discourse seeks to portray the public sector as ‘ineffective, unresponsive, sloppy, risk-averse and innovation-resistant’ yet at the same time it promotes celebration of public sector ‘heroes’ of reform and new kinds of public sector ‘excellence’. Relatedly, Mintrom (2000) has written persuasively in the American context, of the way in which ‘policy entrepreneurs’ position themselves politically to champion, shape and benefit from school reform discourses” (p2).

    I think the analysis of Hattie needs to continue and the public need to know about all the major mistakes and misrepresentations that Hattie makes. I’ve set up a blog for people to contribute here –

  2. Kelvin says:

    Thanks George. Your website is a gold mine. I am getting older, and all those years of chasing Hattie around are over for me and seeing the way he slides away. I went hard to flush him out and to get the opposition in full pursuit. Thank-you ever so much. A couple of decades ago I felt it was only me – I feel a burden lifted.

    • jen says:

      Regardless of what you deem to be inaccurate research methodologies/findings, Professor Hattie has developed a most compelling call to educators to actually focus on “learning” in schools, how about that…schools actually seek to develop “learners” as opposed to churning out students who invariably are killing time passively in silent classrooms!!? Professor Hattie has been a revelation in drawing attention to teachers to use powerful instructional approaches that put students at the heart of their learning, to own it and for teachers to consistently orchestrate environments that deliberately and actively position students as their own teachers. The man is an inspiration to educators everywhere. He has revolutionised education and importantly as a result improved the learning lives of so many students. People want to pick at the nuances in the research…? Go beyond the numbers to the compelling narrative, it’s gold.

      • Ted Lynch says:

        Most teachers have always been about what works in the classroom, Hattie did not start that revolution.
        Kelvins (and Prof John O’Neill and others) analyses are not about nuances in research, they a major issues. Misrepresentation of research is major and can’t be justified by a ‘compelling narrative’.

  3. Thank you Kelvin Smythe

  4. Mike says:

    As a pre-service Teacher i can tell you that his findings are taught as gospel. “We now have conclusive scientific proof” and then you head out to a multi-ethnic, low socio economic school – come back with questions and look to Hattie (which completely ignores background) and the Lecturers just nod their heads and wait for someone else to ask a question.
    It is interesting to note that if you watch Hattie on Youtube he always starts with Class sizes dont matter. He is a very good salesman who knows where to sell his research – governments looking for a policy grab bag.

  5. Kelvin says:

    As you can read his research is so bad and has done so much harm as to transition to evil. Yet, there are all the ‘evidence-based’ academics too scared to speak up. Teachers and children last in this education system. Well done Mike. keep in touch. Talk to your pre-service colleagues. Introduce them to the site.

  6. there’s been a couple of good peer reviews this year if you want a copy let me know –

    Kelvin has been saying this for some time now-

    ‘When taking the necessary in-depth look at Visible Learning with the eye of an expert, we find not a mighty castle but a fragile house of cards that quickly falls apart.’ Canadian statistician- Prof Pierre-Jérôme Bergeron.

    Also, in his excellent article ‘School Leadership and the cult of the guru: the neo-Taylorism of Hattie’, Professor Scott Eacott says,

    ‘The uncritical acceptance of his work as the definitive word on what works in schooling, particularly by large professional associations such as ACEL, is highly problematic’ (p11).

    ‘The reconstruction of Hattie’s approach in detail using examples thus shows that the methodological standards to be applied are violated at all levels of the analysis. As some of the examples given here show, Hattie’s values are sometimes many times too high or low. In order to be able to estimate the impact of these deficiencies on the analysis results, the full analyzes would have to be carried out correctly, but for which, as already stated, often necessary information is missing. However, the amount and scope of these shortcomings alone give cause for justified doubts about the resilience of Hattie’s results’ Wecker, et al (2016, p31).

    ‘the methodological claims arising from Hattie’s approach, and the overall appropriateness of this approach suggest a fairly clear conclusion: a large proportion of the findings are subject to reasonable doubt’ Wecker, et al (2016, p35).

    I’ve looked at the 3 meta-analyses Hattie used for class size here, their conclusion is the opposite of Hattie’s. –

    also, I’ve just looked at Welfare policies here, once again dodgy interpretation by Hattie-

    Virtually, every meta-analysis Hattie used for a particular influence Hattie has misrepresented.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.