For primary schools, there are two main issues with the current system of measuring progress: 1) it is high stakes, 2) it involves teacher assessment. Whether we are talking about our own internal tracking systems, or those official end of key stage DfE measures, the former clearly influences the latter. Imagine you are told that you have to run a marathon in under 3 hours (yes, I do like a running analogy) and that if you fail to do so, the consequences will be dire. Of course, there are some good marathon runners out there for whom this is feasible, and a handful of really good ones for whom this is no problem at all, but for the majority of us this is near impossible. Then we are told that no one will be monitoring your efforts; they will just be basing their judgement on your time, which you alone are responsible for keeping. Now how many sub-3 hours marathon times will we see?
This is the weird situation we find ourselves in: an incompatible mix of high stakes and self-evaluation. Like fire and duvets, they make for strange and dangerous bedfellows, and clearly it’s not working. We all know that really.
Consider that weird, near vertical cliff that occurs at 32 marks every year in the national phonics screening check results; or the impact that current KS1 measures will have on the EYFSP now that the profile is used to establish prior attainment groups for those pseudo-progress measures in the dashboard. Then there are those well documented problems with using KS1 results as the baseline for progress measures. The DfE are attempting to solve this by implementing a reception baseline, but this is highly contentious and unpopular; and even if it’s implemented in 2019 as planned, we won’t see the results until 2026. Then there is the thorny Infant/junior/middle school problem that the reception baseline won’t adequately solve; it will just blur the issue so no one really knows where the problems lie, or if there are any problems anyway.
And then, of course, there are those highly variable and suspiciously high KS2 writing results, and associated issues around moderation. Why do you think that KS2 writing was ditched from the baseline for the Progress 8 measure the minute we ran out of secondary pupils with KS2 writing test results? (That was last year by the way; the 2017 GCSE cohort were the first to have writing TA at KS2 and that won’t be used in their progress 8 measures).
These are some of the well known issues relating to statutory teacher assessment, assessments that are done almost entirely for the purposes of accountability. It is little wonder that no one seems to have much faith in progress measures that rely on such data. But these issues are not restricted to statutory assessment; they also exist in the various tracking systems schools use, tracking systems which still rely on teacher assessment for measuring progress.
The final report ofthe Commission on Assessment without Levels warns about the risks of using teacher assessment for multiple purposes, yet this is still the norm in many schools. Teacher assessment is commonly used not only for formative purposes, but also for measuring pupil progress, monitoring standards, reporting to Governors, evaluating teacher performance, and even comparing schools. These multiple purposes exert conflicting pressures on the data that can and will lead to its distortion; and let’s face it, teacher assessment is too subjective anyway. One teacher’s ‘secure’ is another teacher’s ‘greater depth’, so even if there were no high stakes attached, we still wouldn’t have an accurate picture. This is made even more complicated by the various methods of the tracking systems themselves: different steps, bands and point scores; varying lists of key objectives; and contrasting definitions of ‘age related expectations’ based on spurious algorithms and arbitrary thresholds. And let’s be honest, progress measures based on teacher assessment pretty much always involve reinventing levels anyway. No one is talking the same language; no one knows what’s going on. No one can measure and compare progress.
That’s the point of Progress Bank.
This is something I’ve been thinking about for a long time: a system that wouldn’t rely on teacher assessment to measure progress and wouldn’t rely on end of key stage results either. A system that is powered by whatever standardised tests the school chooses to use, and can measure progress from any point to any point, benchmarked against other pupils with similar start points.
This is obviously a rather ambitious project and not something I’m capable of building, which is why I approached the people at Insight Tracking. I like their current system – it’s intuitive and highly customisable – and they tend to build things quick. I needed their expertise and thankfully they agreed.
The system will work like this: schools upload standardised scores from the various tests they use, pick a start point (i.e. previously uploaded data, say at the start of Y1 or Y3) and an end point (probably the most recent upload), and they receive zero-centred VA scores for cohorts, key groups and individual pupils in whatever subjects are tested. The methodology is essentially the same as that used by the DfE to measure progress, so the data is in a common format, but the system is far more flexible in terms of start and end points, and is based on more regular, lower stakes testing. Schools will be able to interrogate their data using simple, interactive reports, which will focus not only on progress, but on attainment gaps, too.
The neat thing is it doesn’t matter what tests you use, as long as they’re standardised; and if results aren’t standardised, the Progress Bank can standardise them if enough data is uploaded by enough schools to provide a suitable sample. If you use tests from multiple providers, that’s fine; and if you change test provider, your old data will be stored in Progress Bank, so you won’t lose it and can continue to use it. We can, if permission is given, even transfer test results when a pupil changes school. And if you decide to leave, the data is deleted. You own it.
The Progress Bank will be especially useful to junior and middle schools, which have a particular issue when it comes to progress measures, and this project has been expedited by the JUSCO Conference and conversations with members including Chris McDonald, the chair of the group. Obviously, the ideal solution for junior schools is to enable them to measure progress from a Year 3 on-entry baseline, not from KS1 results or from a reception baseline as proposed. Progress Bank will allow them to do that.
But it’s not just for junior schools; it is aimed at any school that is interested in alternative, benchmarked progress measures. The system can even measure progress from KS1 scaled scores instead of KS1 teacher assessment used in official measures, or from any current standardised reception baseline assessment. We hope that the data will help schools challenge the flawed measures that is currently used to hold them to account. And by using standardised scores to measure progress, hopefully we can protect the integrity of teacher assessment by ensuring it is used solely for formative purposes, and perhaps reduce workload in terms of tracking and analysis too.
Now we just need lots of schools to get on board.
Find out more about the project and register your interest here:
And follow us on twitter @theprogressbank