Hattie’s research is false: Vice-Chancellors asked to investigate Part 2

The falseness of Hattie’s research is not difficult to explain; only a few aspects are complex. But everything about Hattie’s research is false except for some opinions which, while they may be true, are also false, because he claims them to be evidence-based. One reason Hattie has an appeal to teachers (along with his assertive sense of rightness) is that he talks about ideas and approaches which are at teachers’ level of interest. His tactic is to mix, amongst his mainly right-wing education ideas, a few liberal ones – that is Hattie the education politician at work, bringing teachers in, making him all the more valuable to corporates and hierarchical governments. Hattie always recognised that his status with authority and corporates was partly dependent on his hold over teachers, hence they tolerated the occasional radical swing in support for certain policies, in particular (as discussed above), national standards. 

  • Hattie took his idea for meta-analysis from the medical world (as referred to above) and applied it to the value-laden one of education in massive style. The difference between, say, a study of hospital operations involving surgical mesh and complications (as instanced above), and a study of the learning effects of an education concept involving many countries and millions of children; children of different ages and genders; children with learning disabilities; students from universities; children in classrooms to clinical situations; teachers and cultures with different understandings of that concept; studies over different time periods (but nearly always short); lessons that are formal to less so (definitely a huge bias towards the formal); comparisons but never making clear the basis for those comparisons; particular classroom influences but not acknowledging that other influences are always present in classrooms and need to be accounted for – is vast. Hattie put all those variables together without distinction, and called it a synthesis. 

The variables are out of control:

  •  Fifty-thousand studies with the estimated number of 236 million students across many countries, though with a USA bias (every country giving a different meaning to the same words – for instance, ‘whole reading’ meaning something very different in the USA to New Zealand).
  • Age: children who are younger are capable of a much more rapid improvement than older children. Many of the studies, it seems involve university students. This variable of the age of the children is of particular research significance.
  • Ethnicity: single ethnicity countries as against highly diverse exhibit particular learning characteristics.
  • Gender: no information supplied on this.
  • Schooling systems: authoritarian countries as against democratic; technocratically advanced countries as against developing ones. Authoritarian countries always have narrow, non-democratic education systems, and developing ones more formal practices.
  • Learning contexts: classrooms or laboratories – laboratory and clinical contexts, it seems, play a substantial role in the research results. 
  • Timing: how soon after the teaching was the testing was done. This variable is of particular research significance. If the testing is done shortly after the teaching, that provides bias to Hattie’s more formal, visible learning emphasis. 
  • Basis of comparison: whether the children were performing in comparison with where they were at the beginning or with a control group. Being clear about this is the foundation research building block. 
  • Influence and effect: Hattie claims that each influence has an independent effect entirely separate from context is inexplicable to orthodox researchers.
  • Student characteristics: a significant number of the results, it seems, are based on children with learning disabilities.
  • Numbers: Hattie was fantastically unreliable with numbers with the most egregious of them being how he worked out the numbers for class size.
  • Teaching styles: dominant characteristics vary from country to country but I suspect that most studies were American-based which would have provided a particular bias.
  • Teaching practices: their names have different meanings in different countries, for instance, ‘whole language’.
  • Numbers of children involved: with all the gigantic numbers involved overall, some studies involve only a handful of children (see above).
  • Parts of the curriculum: certain parts of the curriculum suit different styles of teaching; what is being taught is not made clear, but mathematics, which is often taught more formally, seems a definite emphasis.
  • Variation in aims from country to country: certain aims in education which might be of high importance to children and to particular societies might be more complex to teach and therefore take longer – for instance, ‘whole reading’ is superior to phonics in the longer term but requires patience, the same with ‘problem-based’ mathematics.
  • The affective: largely avoided as the name of the resulting book affirms. 

One of Hatties’s highly ranked influences is feedback, and like all of Hattie’s studies, is characterised by an absence of easily accessible research information detail. Hattie and his supporters put a lot of weight on this concept – but it is really just another name for teaching, made simplistic and trite in the name allocated. From my observation, the attention to Hattie-inspired feedback has led to greater teacher dominance and learning intrusion. The metaphorical characteristics of feedback are disconcerting: feed is consistent with the humanly forlorn idea of only visible learning being worthwhile, and back has the sense of haste, teacher control, and lack of subtlety, characteristic of Hattie’s education philosophy. 

We are not informed of:

 The learning contexts used

 The institutional level involved (pre-school to university)

Whether the students were in a clinical or classroom context

 The curriculum areas involved

 Whether the thinking was straightforward or complex, skill-based or cognitive, formal or affective

 How it was taught, closed or open-ended, or individual discovery

The age of the children, ethnicity, socio-economic status, or gender balance

 How long after the teaching the testing was done 

 The influence of the Hawthorne effect

 Whether the children were performing in comparison with where they were at the beginning or with a control group 

 How Hattie can explain how an influence can have an independent effect entirely separate from the context.The following paragraphs attend to the way Hattie and the education power elites worked together, to mutual advantage, to dominate school education and, in the case of primary education, left it broken and bereft of direction. 

In the 1940s, the New Zealand education system, by a series of circumstances, struck on the truth of education in a democracy which, of course, revolved around the education of the whole child. The social and political contexts for the gaining of that truth were favourable but its discovery was not inevitable, it took leadership; it took people to bring it together, to inspire, to effect clarity. Clarence Beeby and Peter Fraser, made good use of those circumstances and the advice of a number of idealistic educators, to formulate the education philosophy now called the holistic. 

The holistic organises children’s learning through dynamic main aims (to avoid fragmentation); is the interaction of the cognitive with affective; and pays sincere attention to all parts of the curriculum. My argument in The File (brought together in Attack! 132) is that the holistic, from its beginnings in the 1940s, though often criticised by politicians in the decades following, retained, until Tomorrow’s Schools, a definite and inspiring presence in classrooms. With Tomorrow’s Schools, that was to change, the structure became hierarchical, philosophy non-democratic, and curriculum formalistic. The individual and institutional leadership of Tomorrow’s Schools were to draw heavily on Hattie’s formalistic ideas in indivisible partnership with politicians, the education bureaucracy, national standards (of which Hattie was the declared architect), the education review office, and corporate advisory services. 

The holistic main aim for school education in a democracy is that school education should prepare children for life in a democracy and to support and to protect it. This main aim points to its basis in humanism which affirms the dignity of the individual and advocates democracy as a way of establishing and furthering that. It rejects authoritarian beliefs, emphasising individual freedom, responsibility, compassion, and the need for tolerance and co-operation. And it affirms that we must take responsibility for our own lives and the world in which we live. (It may be that this is the kind of education Vice-Chancellors want or wanted for their children or grandchildren. We did our best but as described we were under siege.)

The implications inherent in the holistic make clear the kind of curriculum schools should act on and the kind of experiences children should be provided with. If the aim is to prepare children for life in a democracy and to support and protect it, the holistic is the truth for school education. The aims of the holistic curriculum can be furthered, but the principles, to be holistic, remain the same. 

An example of the way the holistic is open to furthering the holistic while the principles remain the same is the way the holistic has encompassed the neuroscience-informed mindset philosophy.  This philosophy is largely consistent with the holistic – Beeby, Fraser and the holistic teachers through the following decades (mainly in the junior area) got it right, got the neuroscience right without knowing the neuroscience, got it right from knowing the children.

Hattie’s research is structured in favour of the immediate, learning steps, objectives, structure, the formal, the hierarchical, the measurable, the closed, and the visible – and against reflection, taking time, fluency, aims, the whole, variety, problem-solving, power-sharing, evaluation, openness, and the affective. Hattie not only harms in his obsession with hugeness but also in his research totality – everything is included whether for hosannas or varying degrees of damnation. I have commented above on Hattie’s propensity for incorporating ideas from other philosophies to control or corrupt those ideas. I am emphasising here the way his research totality, backed by power elites, takes oxygen from philosophies whether established or recent, to the point of suffocation. This is what happened to the holistic  – and school achievement plummeted. 

Hattie is dependent for his position on the government, ministry, education review office, universities, and corporates for protection against those who would criticise him, also to provide an unassailable platform from which to promote himself. In return, those institutions used Hattie to promote their interests. 

The interdependence between Hattie, the government, the ministry, education review office, universities, and corporates is unsavoury. Let me take you back a number of years when Hattie, with the University of Auckland, set up the Visible Learning Laboratory. The intention was to have the Laboratory travel around the country. Hattie and the University, however, had made a strategic mistake – the Laboratory had the appearance of being on its own, more reliant on the genuine willingness of schools to support him. In being, or apparently being, less protected by his political, bureaucratic, and review office support team, without their fear-inducing power over schools, the Visible Learning Laboratory was vulnerable. My website was to play a major part in destroying it. Visible Learning Laboratory went to Cognition where it became central to that corporate’s advisory services, backed, of course, by the government, ministry, review office, and national standards requirements. 

There has been a quadrangle process of malign influence on professional development in schools. The process began with the education review office – principals knew that the professional development box would be ticked off if a corporate advisory was used and double-ticked if it was related to national standards. So it was the review office, schools, corporate advisory courses, and national standards in collusion. To use independent advisers was to take a risk – so why do that? The corporates had their programmes de facto approved by the ministry and review office, and looked to communities of learning as a new source of wealth. Life for corporates and Hattie looked good.

Now it has changed: a different government, no national standards, an independent advisory to be established, review office under control for the moment, communities of learning in doubt. Without the backing of the power elite for imposition, Hattie’s status is in state of flux, but his durability and chameleon-like qualities should not be underestimated. However, things do look less promising for Hattie and his backers. Which brings me to the telling announcement by Cognition Education that Visible Learning plus (largely the Visible Learning Laboratory) has been sold to Corwin, a giant American corporate. 

Separated from his political and bureaucratic support, Cognition knew Hattie’s Visible Learning Laboratory and associated ideas were likely to be greatly reduced in value. 

But the ill-effects of Hattie’s dominance remain pervasive in system structures, school organisation, and the classroom curriculum, and need to be decisively dealt with. For the health and confidence of our democracy and our schools, a telling blow needs to be struck against Hattie’s process of taking control (based, in the first instance, on false evidence) – a process anathema to both education and democratic ideals. 

My hope is that it won’t be long before Hattie’s bubble bursts. New Zealand played a major part in the aeration, how fitting to would be for us to be involved in the de-aerating.

 I rest my argument with the Vice-Chancellors.


Our professional lives are enriched when academics create education ideas greater than their research. Hattie, sadly, has managed the considerable feat of producing education ideas even more dismal than his specious research. There is a nimble but superficial prolixity to his writing that indicates the possession of a critical intelligence which can operate with no fixed connection to the reality of classrooms or their social context. There is, though, one exception to this, his ability to connect to the reality of the academic market. His tactically adroit research is angled and presented in such a way as to draw teachers in with its certainty, the media with its glibness, corporates with its marketability, and governments with its promise of increased control at the cheapest rates. I have written hundreds of pages about this academic, an academic who has played such a major part in shredding the beautiful holistic education that is our heritage and culture. I have not written about him for some time and intend never to write about him again.

New Zealand academics have criticised Hattie’s research and findings. They are Ivan Snook, John Clark, Richard Harker, Anne-Marie O’Neill, John O’Neill, and Roger Openshaw.

Kelvin Smythe 2018



visiblehat, an Australian educationist, contributed this to the Comment section of Part 1, providing a compelling link to the wrongness of Hattie’s research. It is included as component part of my case to the Vice-Chancellors

 One Response to Hattie’s research is false: Vice-Chancellors asked to investigate Part 1

  1. visiblehat says:

April 15, 2018 at 12:45 pm

Thanks Kelvin, the best place for the Vice Chancellors to start is to read the 3 Meta-analyses Hattie used for the 2009 version of Visible Learning. These show, Hattie can’t average and gross misrepresentation. I contacted Gene V Glass, the inventor of Meta-analysis and author of the major study Hattie used for Class size, he commented about Hattie’s averaging – ‘Averaging class size reduction effects over a range of reductions makes no sense to me’.

Summaries of the 3 meta-analyses are here – https://visablelearning.blogspot.com.au/p/class-size.html

This entry was posted in Hattie and tagged , , , , , , , . Bookmark the permalink.

3 Responses to Hattie’s research is false: Vice-Chancellors asked to investigate Part 2

  1. Thank you for your resilience, integrity and continuing expression of truth. We as educators lived this and it was painful. Your work is healing.

  2. Helen says:

    Brilliant work, Kelvin. There are so many things I wanted to add as I was reading, but you covered them all. I posted a comment on a previous article of yours about Hattie where I had unpacked totally irrelevant studies that he had used to determine that class size was unimportant. (I used a penname because I was so afraid of the then Ministry.)
    I cannot ever see purely quantitative research working in such a people-centred field as education. It is just fraught with uncontrollable variables, as you list in Part 2.
    It is shameful that this ‘research’ is in any way associated with New Zealand. I can’t help but liken his study to the doctor who announced a link between vaccinations and autism based on no plausible evidence whatsoever.

  3. Susan Bearing says:

    You might be interested that Simpson has just recorded an entertaining podcast about effect size, explaining why Hattie (and others) are just getting it wrong. Listen at http://www.ollielovell.com/errr/adriansimpson/

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.