Last week the DfE published its document on Primary School Accountability in 2016, a technical guide to the new key performance measures that will form the basis of the performance tables and Ofsted data this year. With this key document finally out in the public domain, it seemed like an ideal time to summarise the key data changes heading our way, and speculate on some known unknowns. Let’s start at the beginning.
Reception
We now have a reception baseline (you may have noticed). This will eventually provide the start point for VA measures but not until 2022 when the first of the baseline cohorts reach the end of KS2. This cohort will have VA measured from baseline and from KS1 and the school will benefit from the better of the two. From 2023, VA will only be measured from baseline and schools choosing not to use an approved baseline provider will be measured on attainment alone. Any takers?
The DfE could of course also use the baseline to measure VA between reception and KS1 (something they have previously aluded to) and even from reception to the phonics test. As with all other VA measures, pupils are essentially grouped by prior attainment (baseline scores), which indicate their ability on entry. A pupils’ attainment (at KS1 or KS2) is then compared against the national average attainment for pupils with the same start point. It is important to understand that pupils are compared against the national average for pupils with the same start point, not the overall national average for all pupils, or the school average (if you did that you’d always end up with a VA score of zero).
In an attempt to provide a simple analogy of VA I recently tweeted this:
I think some people thought I was making a serious suggestion. I was actually just trying to show how VA worked, how it does not require data in the same format at either end, and that each pupil is compared against pupils with similar prior attainment nationally.
Anyway, back to the baseline. We have three baseline providers: Centre for Evaluation & Monitoring (CEM), NFER, and, of course, Early Excellence, who have a whopping 75% market share. Those that lost out were GL Assessment, Hodder Education and Speech Link. I’m intrigued to see how this pans out over the next year or so: will there be much movement between providers? and will an observation-based, ‘non-invasive’ approach provide accurate and consistent baseline data? The SQA are currently carrying out a comparability study of baseline providers, which is due to report back soon. This may result in further shake up of the market and the word on the street is that the DfE really just wants one provider, which of course makes sense. Why they didn’t do this in the first place is beyond me.
Phonics
Not going anywhere!
Key Stage 1
This year we have the introduction of externally set, internally marked tests at KS1 to ‘inform’ teacher assessments. This will include a grammar, punctuation and spelling test (which everyone seems to be thrilled about) to ‘inform’ the teacher assessment of writing. The tests will be marked internally and schools will be provided with a conversion table to convert the raw score to a scaled score, where 100 indicates that the pupil has met the expected standard in the test. The reason for the inverted commas, is because I’m not sure how rigidly the test score will inform the teacher assessment. Can a pupil who achieves 92 in the SPaG test still be assessed as having met the expected standard, for example?
Most pupils will be assessed as ‘working towards the expected standard’ (WTS), ‘working at the expected standard’ (EXS) or ‘working at greater depth within the expected standard’ (GDS), the latter being the new catch phrase for ‘mastery’. Since the publication of the Rochford Review we now know that pupils who are unable to access the tests or working below the level of the tests can be assessed as either ‘foundations for the expected standard’ (PKF) or below (BLW) and that p-scales are also still in that mix.
Bizarrely (well, it seems bizarre to me), the DfE will not be collecting the KS1 test scores this year, which means that the VA for this cohort (who do not have reception baseline scores) will be based on these broad assessments when they reach the end of KS2 in 2020. It’s less differentiated (although arguably more robust) than the outgoing system of sublevels at KS1, which makes me concerned about the validity of VA measures for this cohort. Essentially, a pupil that was EXS at KS1 will have their KS2 scores compared against the national average KS2 score for pupils that were EXS at KS1. What exactly does this tell us? If the DfE collected the test scores the measures could be a lot more refined, with pupils placed into smaller, more meaningful prior attainment groups.
Of course, these are all interim arrangements, which begs the question: what will happen next year? I bet the DfE start collecting KS1 scores and it wouldn’t surprise me if we end up with externally marked KS1 tests, which makes me wonder about the future of teacher assessment.
And I’m willing to wager a Mars Bar that we end up with floor standards for KS1 within 3 years. Anyone willing to bet against it?
Key Stage 2
Not much change here.
Kidding.
First up, the floor standards and an admission. Michael Tidd was right: there will be three separate progress measures, not the single measure I’d hoped for. The DfE had alluded to a single VA measure in the Education Regulations 2015, which suggested that there would be one ‘column’ for progress. This would fit with the single Progress 8 measure used at KS4 and be less harsh on schools, but that is evidently not going to be the case here. So, a school is above floor if 65% or more of its pupils achieve the expected standard in reading, writing and maths combined. If a school falls below this 65% threshold then the progress measures kick in. Unlike now, when a school will be above floor if they are above just one of the medians for expected progress, in 2016 they will need to be above all three ‘sufficient’ progress measures. We don’t know what these thresholds are at the moment but they’ll almost certainly be negative, much as the Progress 8 floor standard is at KS4, or we’ll end up with a lot of schools below floor.
The DfE suggests that they will fix the progress thresholds for future years so schools will know in advance the line they need to cross, which will be a step in the right direction. But then comes the fun bit: trying to predict VA so schools have a reasonable idea of the position they’re in. I am hoping that i’ll be able to continue to produce my VA calculator to help schools with this but it will rely on schools’ ability to predict pupils’ KS2 scaled scores with some degree of accuracy.
And just to reiterate, if a school is above the 65% expected standard threshold, then they are above floor even if VA is low in all subjects.
So, according to the Primary Accountability document (see link at top of this post) the key performance measures for 2016 are as follows:
1) the percentage of pupils achieving the ‘expected standard’ in English reading, English writing and mathematics at the end of key stage 2
2) the pupils’ average scaled score:
- in English reading at the end of key stage 2
- in mathematics at the end of key stage 2
3) the percentage of pupils who achieve a high standard in English reading, English writing and mathematics
4) the pupils’ average progress:
- in English reading
- in English writing
- in mathematics
A few observations: 1) SPaG still does not form part of these key performance measures (bet that changes). 2) it doesn’t look like there will be a combined average scaled score, just average scores for reading and maths separately. Writing teacher assessments will be assigned a nominal value for the purposes of VA but these scores will not be published. 3) We have no definition yet of what constitutes a pupil achieving a high standard. I assume that this will require some norm-referencing once all the results are in and could be based on standard deviations. Question is: will it be fixed at a specific score in future or will it change every year?
Testing arrangements continue as now: pupils will sit externally set, externally marked tests in reading, maths and SPaG, and results will be reported as scaled scores. Schools will receive raw scores, scaled scores and an indication of whether the pupil has met the expected standard (i.e. achieved a score of 100 or more in their tests). Scores will be reported to parents.
Teacher assessments also continue but are far more limited for reading, maths and science, in which pupils are simply assessed as having met the expected standard or not. Writing has the same assessments as at KS1: working towards the expected standard, working at the expected standard, and working at greater depth. It’s where pupils are working below the expected standard or unable to access the tests that things get rather bizarre (and reduce people to tears and hysterics). In science all very straightforward: if they have not met the expected standard then they are HNM – there is no further differentiation – but in reading and maths HNM is a specific assessment that appears to align with ‘working towards’ in writing, and below that we have:
- Growing development of the expected standard
- Early development of the expected standard
- Foundations for the expected standard
And below that is below (BLW) and below that are the p-scales.
Well, that’s my interpretation but I recommend you read the Rochford Review and make up your own mind. Call me a conspiracy theorist but I do wonder if they have deliberately made this so painful that no one will complain when they ditch statutory teacher assessment.
Oh, and I suppose I’d better mention coasting schools. These are schools that fall below attainment and progress measures across 3 years. The first coasting schools will be identified this autumn and these will be schools that have fewer than 85% (yes, 85%!) achieving L4+ in reading, writing and maths in 2014 and 2015, and fewer than 85% achieving the expected standard in 2016. In addition they will need to be below the 2014 and 2015 medians for expected progress in reading, writing and maths (2014: 94%, 96%, 93%; 2015: 94%, 97%, 93%), and below just one of the three, yet to be defined, progress thresholds for 2016. Based on data for the last 3 years, Education DataLab suggested around 5% of primary schools will find themselves caught in this net but considering the expected standard is tougher than L4+RWM, it could end up being higher than that. And it remains to be seen if the DfE employ the same progress thresholds for coasting as they do for the in year floor standards. They’ll probably be tougher just to add a further layer of excitement and jeopardy.
Right, that’s enough (until I remember something important and edit this post).
I doubt that anyone has read this far anyway.
Certainly not without the aid of gin.