Differential Learning Loss by Barnaby Lenon

Some children have learnt little since March, others have learnt more than they would have done had they been in school.

Understandably, parents and teachers’ unions are calling for some adjustment to public exams in June 2021 to recognise this fact. Most people would agree with me that, if adjustment could be done it should be done. After all, one purpose of exams is to rank pupils and ranking will be pretty unfair if different pupils have had different levels of learning loss.

But this is unlikely to happen for a number of reasons.

1) Every pupil has been affected by COVID-19 learning loss in a different way. Although some parts of the country have been hit more than others, even within individual schools the impact has been very different for different pupils. After all, some have access to online lessons when others do not. Some have parents who help them catch up. Some children are simply more diligent than others. It would simply not be possible to adjust exams fairly to cope with such a wide range of experience.

It was suggested the other day that pupils in cities most affected by COVID-19 should have reduced exams. But this implies that every child in any given area was affected in a similar way. It implies someone dividing the whole country up by level of COVID-19 so those on one side of a line do x questions in an exam while those on the other side of the line do y. Far too simplistic.

2) The most obvious suggestion is that GCSE and A-level exams, where these days many questions and topics are compulsory, should introduce a level of optionality. For example, if, in GCSE Geography, you were expected to answer questions on coastal, glacial and river landforms, this could be adjusted so that you only had to answer questions on two of the three. So, pupils who have missed a lot of school would only need to be taught two of the three and would only need to revise two of the three.

But there are difficulties with this proposal.

First, many children – possibly most – have already covered coastal, glacial and river landforms.

Secondly, if you cut the syllabus down in this way pupils will know less than their predecessors and might suffer if they go on, for example, to take Geography A-level.

Thirdly, assuming that the pupils now had to choose two of coastal, glacial and river landforms, it is unlikely that the questions in all three would be equal difficulty.  If the coastal questions were easier than the river questions, the river question group would suffer. What is more, research summarised by Ofqual shows that disadvantaged and less able pupils faced with options tend to make bad decisions – they are less good at spotting the easier questions.

Fourthly, although the example I have given seems quite straightforward, this is not as true for all school subjects. Exam boards have limited capacity and to ask them to rewrite syllabuses in this way and at this late stage may be asking too much. They would make mistakes.

Finally, teachers started this academic year with a clear purpose and trajectory.  If we shift the goal posts now, the disruptive impact could be worse than not making adjustments.

3) One purpose of exams and exam grades is to signal what it is that pupils know. You want to be reasonably sure that someone who passes English GCSE, for example, has a basic level of literacy. If you start fiddling with exams to make them easier, you lose the meaning of the grades.

4) Compensating for learning loss can be achieved in other ways. Here are four:

1. on August 3 Ofqual cut parts of the syllabuses of most GCSEs.
2. those people who use exam results (schools and colleges using GCSEs for sixth form entry, universities and colleges using A-levels for admissions) can make the adjustment. They simply need to be given the schools’ predicted grades, they need the school to tell them the level of learning loss each applicant suffered, and they can then make an offer of a place based on a fair assessment of this data. This is often called ‘contextual admissions’ i.e. taking into account the context of the student. All good admissions systems do this anyway.
3. we already have a special circumstances system for pupils who miss an exam because they were ill. This system could be extended to those who missed a great deal of teaching because of COVID-19.
4. the Government is funding catch-up tutoring for those who have fallen behind.

Schools in Christchurch, New Zealand, were closed for weeks following the 2011 earthquake. They did not have access to the range of online learning that is available now. However, evidence from the New Zealand Qualifications Authority showed that student results in the final exams actually improved.

According to Professor John Hattie, then the adviser for the New Zealand Qualifications Authority, the difference was teachers focused on “what has to be learned” instead of getting through a lot of curriculum. Professor Hattie suggested that it was a similar story after Hurricane Katrina in New Orleans in 2005, when students missed up to seven weeks of school. Again, students “recovered quickly and actually began to see gains in test scores”.

This may not be true of our pupils who have had learning loss, it is too early to say.  But this story does remind us (as if we needed reminding) that the exam system is only one moving part in the system.  The teachers and pupils can behave in unexpected ways.

Ofqual is bound to impose generous grading in 2021. This will be some sort of compensation, but it is probably impossible for the exam system itself to compensate for differential learning loss.

Barnaby Lenon

Dean of Education, University of Buckingham