NMSSA results and NS: more evidence of high stakes inflation

The National Monitoring Study of Student Achievement (NMSSA) project began in 2012 and assesses student performance across the y. 4 and y. 8 alignments as set in the New Zealand Curriculum.

In 2013, the NMSSA assessed mathematics (also some other curriculum areas), and in 2014 reading (also social studies). The NMSSA results for reading (2014) and for mathematics (2013) are recorded below as are the national standard results for comparison.

I have little doubt the researchers (to protect themselves from the government) will cast doubt on the validity of this comparison; I have little doubt, however, the comparison is valid. Anyway, the NMSSA marks just judged on their own are pretty terrifying and in one case transcendentally so.

NMSSA is produced by government contract out of Otago University by a team of quantitative academics some of whom I have, over the years, given a hard time, but in this case they have mainly produced the goods, especially in reading and mathematics (in social studies and some other areas to manage the research they have imposed distorting conceptual frameworks).

NMSSA results invite comparison with national standards results, but few have undertaken this, in fact, nobody I know: many (reporters, for instance) because they didn’t know NMSSA existed, and many (in education) because they didn’t want to.

A further comparison I’m interested in is with the postings linked to this posting (see below) which detail the inflationary effects of high stakes on teachers reporting of national standards and NCEA and the subsequent lack of signals to the need to change policies and structures.

If I went into the variables between NMSSA and national standards and between the groups of children, this posting could be a very long – I have chosen not to, and have plunked for a very short one.

READING

2014

National standards: y. 4

83.10% of students achieved ‘at’ or ‘above’ the national standard.

NMSSA: y. 4

58% of students scored above the minimum score associated with achieving curriculum level 2 objectives.

‘Additional’ to NS: 23.10

National standards: y. 8

77.64% of students achieved ‘at’ or ‘above’ the national standard.

NMSSA: y. 8

59% of students scored above the minimum score associated with achieving curriculum level 4 objectives.

‘Additional’ to NS: 18.64

MATHEMATICS

2013

National standards: y. 4

76.72% of students achieved ‘at’ or ‘above’ the national standard.

NMSSA: y. 4

81% of students scored above the minimum score associated with achieving curriculum level 4 objectives.

‘Additional’ to NMSSA: 3.28 ***

National standards: y. 8

68.90% of students achieved ‘at’ or ‘above’ the national standard.

NMSSA: y. 8

41% of students scored above the minimum associated with achieving curriculum level 2 objectives.

‘Additional’ to NS: 37.90 ***

On the basis of information gained from data and directly from classrooms, my estimation of high stakes inflation was 12.5% for national standards and 20% for NCEA level 2. I still hold to that. Though with the clusters and Maori plan now in play, I predict high stakes inflation will soon rise to 15% and above. The ‘additional’ to NS in reading is more than I would have predicted, somewhat exaggerated, I believe, because  reading is a very personal matter between teacher and child – meaning the way the data was gained could well have skewed the results. However, substantial reading inflation is confirmed as occurring.

The mathematics results are topsy-turvy almost beyond belief.  To go in four years from a 3.28 NMSSA ‘additional’ to a 37.90 NS ‘additional’ is a remarkable turnabout. The best reasoning I can come up with is that the level 2 curriculum objectives are softer than the level 4 ones meaning that the incentive to inflate, relative to other achievement assessment levels, is lower (considerably lower). I know we could point to better teaching at particular levels but, am loathe to do that, and find that unconvincing anyway, why in mathematics and not in reading?

I will listen carefully to what readers and academics have to say or write to me. Some may say my comparison is invalid – well good on you: explain why. And such matters are always complex which is why I have kept it simple for you to fill in the gaps.

https://networkonnet.wordpress.com/2016/06/29/one-day-someone-in-the-media-will-read-these-three-links-and-it-will-be-all-over-for-parata/

https://networkonnet.wordpress.com/2016/02/26/ive-had-enough-the-truth-about-high-stakes-assessment-and-inflation-part-2/

https://networkonnet.wordpress.com/2016/04/07/ive-had-enough-the-truth-about-high-stakes-assessment-and-inflation-part-3-the-primary-school/

https://networkonnet.wordpress.com/2016/05/06/ive-had-enough-secondary-ncea-broken-wide-open-part-4/

Advertisements
This entry was posted in Education Policy, National standards, NCEA and tagged , , . Bookmark the permalink.

4 Responses to NMSSA results and NS: more evidence of high stakes inflation

  1. Thank goodness for Kelvin. Someone giving us a clear picture of what is really happening.

  2. russell1 says:

    The thing I am most concerned about is that one of our great white leaders will decide that national testing will solve the problem.

    • Kelvin says:

      A fair point but unlikely because madcap ideas like clusters and the Maori plan will be shown as having failed. The government above all doesn’t want to spend money on actual things like reducing teacher-child ratios and teacher aides. But whatever, we can’t go on as we are. School education is hardly raised as an issue. Labour, except for computers and now careers, never makes the news because national standards and NCEA are seen as sort of working. We have to stir things up (even at the testing risk you refer to Russell) to stop the slide. It is really away of flushing out Little to get back to Labour core values and to get rid of Hipkins. Their party started this mess and misery – they can darn well start to put it right. As well, there’s the performance by NZEI top brass (especially the secretary). NZEI is, in effect, complicit with the government and national standards.

      But there are two points of hope: Tracey Martin (a continuing hero) and we need Kelvin Davis (he’s tough, I’ve worked with him, but once set on a course, indomitable).

  3. principalatuni2016 says:

    I agree russell1, it will be the great ‘consistency tool’ that they desired the great consistency tool to be. Mandated national testing for consistency sakes…yeehaaaaa!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s