John Hattie: your research is now a con

In this article I am going to place considerable demands on Simon Collins’ flexibility of thinking, on his ability to imagine the unimaginable, his willingness to recognise a wrong universally accepted right by the media as an incontrovertible right, and the naïve trust he no doubt shares with his media colleagues in the truthfulness of education authority.

The prompting for this posting is a follow-up article Simon Collins wrote in the New Zealand Herald  (March 18, 2017) to an earlier major one detailing New Zealand’s declining international test results.

Simon Collins asked, as the media always do, an academic – described as a top educator – why he thought this was happening.

The academic was adamant: ‘We have more streaming than any other country in the world.’

He must be dreaming!

Well he does now live in Australia.

I have come to know this academic very well and am confident in declaring that he was winging it.

The colossal irony is that it was the actions and words of the academic being asked, more than any other academic, more than any other New Zealander, who caused the decline in international test results.

A decided motivation to distract.

How this academic must have laughed.

Oh what a humble amanuensis Simon Collins proved to be.

How this academic, this top educator, must have enjoyed Simon Collins falling for it.

The top educator went on to say, we used to be at the top but now we weren’t, and the reason is ‘streaming children into top, middle, and bottom classes’. (The top educator, as ever, has it wrong: the huge preponderance of streaming doesn’t occur in classes but within them, a fact that would be second nature to a true top educator.)

What a fiddle the top educator was getting away with.

I have long written about the advantages of unstreamed learning and taken courses on how to organise classroom learning without it (for it to work at its best, holistic practice is required) but its use has only miniscule harmful effects on learning compared with the widespread structural effects of the introduction of national standards, one of them, not incidentally, being an increase in streaming.

In an article in the Herald (February 6, 2010), Andrew Laxon writes ‘Bill English sought Hattie’s views when he originally developed the party’s national standards policy and Key took the same route, drawing inspiration from Hattie’s advice that a standards-based approach could work wonders in even the poorest schools.’

In that same article Andrew Laxon writes: ‘Hattie’s sworn enemy in all educational matters, former school inspector-turned-blogger Kelvin Smythe is far more forthright.’

‘He believes that Hattie has worked hand in glove with the Government on the system and claims his influence is so great that other academics are too scared to speak up against him.’

John Key said John Hattie was the architect of the introduction of national standards into New Zealand. He became the overwhelmingly dominant academic supporting government policies. To those of us who wanted truth in education, who wanted progressive policies, who wanted genuine discussion, Hattie was an academic miasma. He was everywhere. His research was the rationale for government policy and in combination with the government and a bewitched media, a devastatingly formidable force to confront.

But the research, as I hope to convincingly demonstrate, is false.

Now this is the challenge to Simon Collins, instead of investigating declining test results and interviewing an academic (always the same one), I suggest he investigate the academic.

I know what Simon Collins will think – Kelvin Smythe is he unhinged? Hattie, a former University of Auckland professor, producer of the Holy Grail research, still dominant with the National government, presently head of the Melbourne Education Research Institute, the dominant academic voice in Australia, consultant to Pearson, the go-to academic world-wide.

Just as Fukuyama declared the end of history so, in a way, has Hattie’s research been declared the end of education research.

But that would be strange wouldn’t it given the name of the book of his research: Visible Learning? A book of research that concerns itself only with that which is visible? A perfect model for the neoliberal commodification of education but a gargantuanly imperfect one for research on children.

Despite all that, Hattie’s education research remains just as false – all it goes to show is Hattie’s fantastic charismatic, testosterone driven-ability to sell trombones and employ his cunning political skills.

I viewed him once talking to a group of students, nearly all women, and Hattie standing at the board saying: When you become a teacher, would you like to teach with the absolute certainty of being right? A swoony intake of breath was the response.

But his research is demonstrably false.

Hattie’s refusal to acknowledge the falseness of that research – undertaken as a young researcher, harmful to hundreds of thousands of children – means it has become akin to academic fraud. I don’t believe the research was intended as an academic fraud but after getting the research so wrong, and having it pointed out to him, his refusal to disown it, means he has to wear its falseness.

A number of British academics have exposed him and breaking through the fear of Hattie’s close relations with the government including his involvement in PBRFs, a few New Zealand academics.

(I have written a number of lengthy articles tracking the directions Hattie took and was a great nuisance to him. On one occasion he said he would only sign an academic petition if the academics could get me to shut up. I agreed on the condition I could open up if he went against the arguments of the petition signed, which I knew he would – and which he did. To a visiting English academic he complained about harassment and there were vague references to legal action.)

The falseness of Hattie’s research is not difficult to explain; it is not technical or abstruse. But everything about Hattie’s research is false except for some opinions which, while they may be true, are also false, because he claims them to be evidence-based. One reason Hattie has an appeal to teachers (along with his assertive sense of rightness) is that he talks about ideas and approaches which are at teachers’ level of interest. His tactic is to mix, amongst his mainly right-wing education ideas, a few liberal ones – that is Hattie the education politician at work, bringing teachers in, making him all the more valuable to corporates and neoliberal governments. Hattie always recognised that his status with authority and corporates was partly dependent on his hold over teachers, hence they tolerated the occasional radical swing in support for certain policies, in particular, national standards. He signed a petition, for instance, against national standards even though he was the ‘architect’ of them, soon to change tack of course; and just before he left for Australia he said the national standards were the wrong ones – a very odd response for the architect.

Hattie took his idea for meta-analysis from the medical world and applied it to the value-laden one of education in gigantic style.

The difference between, say – a study of hospital operations involving surgical mesh and complications, and a study of the learning effects of, say, whole language involving many countries and thousands of children; children of different ages; children of different genders; children ranging from those with disabilities to able ones to university students; children in classrooms to clinical situations; teachers and cultures with different understandings of  whole language; studies over different time periods (but always short); lessons that are formal to less so (definitely a huge bias towards the formal) – is vast.

No-one else in education has tried a similar meta-analysis since, for an obvious reason, in education it throws up rubbish – there is no-way the variables can be controlled:

• Fifty-thousand studies with the estimated number of 236 million students across many countries, though with a USA bias (every country to give a different meaning to the same words – for instance, whole reading and whole language meaning something very different in the USA to New Zealand).
• Age (children who are younger are capable of much more rapid improvement than older children; and what is the relevance of studies of university students to the teaching of school children?)
• Ethnicities (single ethnicity countries as against highly diverse).
• Schooling systems (authoritarian countries as against democratic; technocratically advanced countries as against developing ones).
• Learning contexts (classrooms or laboratories – laboratory and clinical contexts play a substantial role in the research results).
• Student characteristics (a significant number of the results are based on children with learning disabilities).
• Teaching styles (dominant characteristics from country to country).
• Vocabulary (teaching practices and their names have different meanings in different countries).
• Parts of the curriculum (certain parts of the curriculum suit different styles of teaching; what is being taught is not always made clear but mathematics, which can be taught more formally, is a definite emphasis).
• Variation in aims from country to country (certain aims in education which might be of high importance to children and to particular societies might be more complex to teach and therefore take longer – for instance, whole reading is superior to phonics in the longer term but requires patience, the same with problem-based mathematics).
• The affective (largely avoided as the name of the book states).

To take one of Hatties’s influences, feedback, there is an absence of easily accessible research information on:

• The learning contexts used.
• The institutional level involved (pre-school to university).
• Whether the students were in a clinical or classroom context.
• The curriculum areas involved.
• Whether the thinking was straightforward or complex, skill-based or cognitive, formal or affective.
• How it was taught, closed or open-ended, individual discovery or class led.
• The age of the children, ethnicity, socio-economic status, or gender balance.
• How long after the teaching the testing was done (of overwhelming significance).
• The influence of the Hawthorne effect.
• Whether the children were performing in comparison with where they were at the beginning or with a control group (ponder the enormous research significance of that – yet the results were all tumbled together).
• How each influence has an independent effect entirely separate from the context (a more technical area of research error but pervasively significant) as he claims.

The crucial point is not that most of the information couldn’t, eventually, be identified but the impossibility of ever bringing the variables validly together. My assumption, given Hattie’s personality, is that he felt he could get away with it by deus ex machina and he largely has. Hattie, as demonstrated below, sometimes fragmented the data to often farcical effect and wickedly misleading outcomes but, it seems, we fell for deus ex machina again. Where is the outrage?

An academic dug deep down into the research information and came up with the following (there are dozens more just as grotesque):

• Hattie’s ranking of quality of teaching (to the question: how important an influence is it – want to drive a bus through what quality means?), which he ranked 56^th out of 138 in his league table (56^th is he for real?), came solely from student ratings by college and university students. Can you believe this rubbish? How could Hattie not have known it was a con?
• The micro-teaching influence was ranked 4^th as an influence, but only came from pre-service teachers.
• Professional development was ranked 19^th as an influence but drops to 48^th when only schools research is included.
• Formative evaluation was ranked 3^rd as an influence but there were only two sets of results and both to do with special education children.
• Comprehensive interventions were ranked 7^th as an influence but they only concerned learning disabled children.

British academics proved half his results were mathematically incorrect due to a school-level howler, a demonstration of his amateurism. It took him years to acknowledge the error even though it was self-evident. What chance then of acknowledging the falsity of his research as a whole. It is important to point out, though, the other way he worked out the results produced ‘correct’ results.

I searched all his writing to see if there was any kind of apology, any kind of counter, and I found only one semblance:

Hattie wrote: ‘No worry, it all balances out.’

Our professional lives are enriched when academics create education ideas greater than their research. Hattie, sadly, has managed the considerable feat of producing education ideas even more dismal than his specious research. There is a nimble but superficial prolixity to his writing that indicates the possession of a critical intelligence which can operate with no fixed connection to the reality of classrooms or their social context. There is, though, one exception to this, his ability to connect to the reality of the academic market. His tactically adroit research is angled and presented in such a way as to draw teachers in with its certainty, the media with its glibness, corporates with its marketability, and governments with its promise of increased control at the cheapest rates. I have written hundreds of pages about this academic, an academic who has played such major part in shredding the beautiful holistic education that is our heritage and culture. I have not written about him for some time and intend never to write about him again.

John Hattie: your research is now a con.

As for Simon, as eminent, skilled, and good-hearted as he is, taking on Hattie is beyond his reach, anyone’s, because Hattie is an embodiment of where school education is today he has, it seems, a free pass to any neoliberal outrage that takes his fancy, with truth and children the ultimate victims.