England’s PISA File Keeps Leaving Weaker Pupils Out

A UCL study found England’s 2018 PISA maths score sat about 10 points above a sample that matched real GCSE grades. Professor John Jerrim, who wrote England’s official 2015 country report, showed lower-attaining 15-year-olds were thin in the sitting file, and later Department for Education reports recorded the same pattern.

The OECD’s Programme for International Student Assessment is the ranking ministers quote. The 2018 England maths score was 504. Jerrim’s illustration, built from GCSE grades obtained by freedom of information, put a more complete sample near 494, around the industrialised-country average of 489.

A Sample Light on Grade 3 and Below

Jerrim’s peer-reviewed study, published online on 22 April 2021 and later in Review of Education, compared the GCSE maths grades of England’s PISA 2018 cohort with the national 2019 mix. The bottom of the scale was light, and the top was heavy.

GCSE MATHS GRADES, PISA 2018 SAMPLE VERSUS ENGLAND 2019

Across the UK, about 40% of the target group never entered the weighted PISA file, among the highest shares of any participating country. Jerrim asked the Office for Statistics Regulation to review how the figures are published, and he told UCL the study was meant to cover 15-year-olds, with serious flaws with children left out, schools declining, and pupils not turning up.

In England and Wales, there is clear evidence that some lower-achieving pupils have been systematically excluded. While what has happened in Scotland is, frankly, a bit of a mess.

John Jerrim, professor of social statistics, UCL Social Research Institute

Wales showed the same tilt. Official data put 41% of young people failing to reach a GCSE C in maths in 2018/19, against 31% in the Welsh PISA sample. Jerrim put Wales’s official maths score of 487 near 473 once that gap was priced in, which would sit with countries such as Malta and Belarus rather than on the OECD average of 489.

Heads at Weaker Schools Opted Out

England did not fail one OECD gate. It failed several, stacked. Schools can be left out of the frame. Sampled schools can refuse. Pupils inside a participating school can be marked ineligible, excluded, absent, or unwilling. Each cut removes a different slice of the age group, and together they shrink the file that produces the headline score.

HOW 15-YEAR-OLDS DROP OUT OF THE FILE

• School frame: England left out special schools, hospital schools, secure units, international immersion schools and pupil referral units before the sample was drawn.
• School refusal: About 27% of UK schools declined in 2018, against an OECD comfort zone of 15% or less; England’s response before replacements was 72%, Northern Ireland’s 66%.
• Pupil day: About 17% of sampled UK pupils in participating schools were absent or refused, and only Portugal had a weaker pupil response; England’s rate was 83.2% against an OECD average near 90%.
• Unpublished check: Because school response missed the bar, officials ran a non-response bias analysis, then did not publish it; Jerrim obtained it by FOI and found schools with weaker historic GCSE results were more likely to refuse.

That last point is the mechanism, not a side note. A two-hour test in Year 11 is easy to decline when results season is close, and the schools with the most to lose from a bad snapshot are the ones already living with weak GCSE history. The OECD later said the largest remaining gap was school and pupil behaviour, not a national plan to drop candidates, which is another way of locating the decision on the headteacher’s desk.

Paris Called the Bias Small

The Department for Education said the 2018 England file met OECD standards and was accepted for the international reports. The National Foundation for Educational Research, which recruited the 2018 sample, said it followed the same procedures as other countries and that the sitting schools and pupils met those standards, while adding it would be willing to look at Jerrim’s points.

Paris did not sign Jerrim’s 10-point illustration. An OECD reply said his estimate rested on assumptions it could not confirm, and that GCSE-style national exams around age 16, which let him run the check, do not exist in most PISA countries. It also said the United Kingdom had overstated the target population by about 6% (pupils later found ineligible) and had removed about 5% through deliberate school or pupil exclusions.

It would not rule out a small upward bias from exclusions and pupil non-response, and it said that kind of tilt might have sat in earlier rounds, or in other countries, where nobody had auxiliary exam grades to look. Jerrim’s open 2021 Review of Education paper still put the UK among the worst for total non-participation, and he argued the adjudication rules were too weak to carry the claim of a truly comparable cross-country file.

The OECD rulebook is not a mystery. Its sampling standards set 85% school and 80% pupil targets after weights. Miss the school bar and replacements are allowed, with a higher after-replacement target. Miss those, and a bias study is supposed to show the remaining schools still look like the country. Between 2000 and 2015, Jerrim noted, only three of 23 such studies led to a country being dropped.

Two of 82 Standards Missed in 2022

The next cycle did not bury the warning. It printed it. England’s PISA 2022 national report, covering 4,763 pupils in 165 schools tested in November and December 2022, told readers the sitting group did not meet 2 of 82 standards. Both misses were response rates, in a year when Covid made recruitment harder and some schools pulled out late.

ENGLAND’S 2022 SAMPLING FILE

• Schools: Weighted response started at 66% and finished at 82% after replacements, against an after-replacement target of 94.3%.
• Pupils: Final weighted response was 75%, under the 80% floor.
• Who sat: The final pupil group had higher key stage 2 attainment than the age group, and a lower share eligible for free school meals in the past six years.
• OECD pricing: Paris put a small upward bias at about 7 or 8 points after non-response adjustments, and 7 to 10 points in some other systems that also missed sampling rules.

The report still published scores, as the OECD does not revise a country’s number when standards are missed. It warned that trends and league tables needed care, and that the bottom and top tails were the worst place to lean, because lower-performing pupils looked under-counted. Australia, Canada, Ireland, the Netherlands, New Zealand, the United States, and the other UK nations were among those that also missed sampling thresholds that round.

England’s 2022 maths score was 492, the figure the 2025 report later used as the prior-cycle comparator, not significantly different from 493 in 2015 and well below 504 in 2018. Jerrim’s 10-point 2018 illustration and the OECD’s 7 or 8 point 2022 estimate are different exercises on different years. Both point the same way: the sitting room was stronger than the year group.

Why the 2025 SEN Share Barely Moved

PISA 2025 results came out on 8 September 2026. Science was the lead domain. More than 90 education systems took part. In England, 5,619 pupils in 183 schools sat a two-hour computer test between February and May 2025, most of them heading into GCSEs that summer. Pearson ran the study for England, Wales and Northern Ireland.

The PISA 2025 national report for England said a school-level bias study found the participating schools looked like the national mix on several traits. Inside those schools, non-responding pupils were slightly more likely to have been eligible for free school meals in the past six years, and slightly more likely to receive special educational needs support. Maths came in at 492, above an OECD average of 463, and in line with 2022, while the OECD trend average fell from 490 in 2018 to 465 in 2025.

Nine systems scored clearly above England in maths, including Singapore, Japan, South Korea, Estonia and Switzerland. Fourteen percent of England’s pupils reached proficiency levels 5 or 6, against an OECD 8%. A quarter sat below level 2, against an OECD 35%. Those tail figures are the ones a thin SEN share would move first.

THE SAME HOLE, THREE CYCLES

1. December 2019: PISA 2018 lands. England maths is 504. Jerrim flags unpublished bias checks and a 17% within-school drop-off.
2. 22 April 2021: His Review of Education paper, using GCSE FOI data, puts a representative England sample near 494 and Wales near 473.
3. 5 December 2023: The 2022 England report says two sampling standards were missed and higher attainers may be over-counted, with OECD bias around 7 or 8 points.
4. 8 September 2026: The 2025 England report again says the sitting pupils may not match the age group, with non-respondents more likely to carry FSM and SEN markers.

Appendix Table A.17 in that 2025 file is the sharp end. Pupils on SEN support made up 10.7% of those who sat the test and 25.0% of those sampled who did not. After the non-response adjustment, the sitting share was 10.8%. Paul Frijters, a visiting professor of wellbeing economics at LSE, read the same sampling table and noted pupils with an education, health and care plan were 1.4% of the final sample against 7.1% among those who did not sit. The weights barely moved the mix. They cannot invent pupils who never entered the room.

The February Window and the Empty Back Row

Two other design choices now sit on top of the old non-response problem. Christian Bokhove, professor of mathematics education at the University of Southampton, pointed out that England tested from February to May 2025, a mix of Year 10 and Year 11, while Wales, Scotland and Northern Ireland tested in September and October, after the summer break, mostly in the year below, with GCSEs still a year away. Jerrim wrote on 14 September 2026 that from 2006 to 2022 England’s PISA sample had been about 99% Year 11s, and that 2025 looked more like one-third Year 10 and two-thirds Year 11.

Exclusions are the other gate. The OECD treats a 5% exclusion cap as the point at which average scores might move by about five points. In a 27 September 2026 discussion of the new results, Jerrim was asked about a 7.6% exclusion rate, above that cap, and said a higher figure makes a larger tilt on the mean credible, which is why the bias studies matter. The pupils most likely to be out of the building, off the roll, absent on the day, or excluded for special educational needs are the lowest attainers. Estimates of the bottom tail move first.

England has lived with this since the first cycles. Response was so weak in 2000 and 2003 that results are only treated as comparable from 2006. Other systems with thin participation, including Canada, New Zealand, Portugal and Hong Kong, have faced similar arguments. England is unusual in having GCSE grades, FSM flags and SEN markers to show who is missing. That is extra rope, as Jerrim has put it, not proof that every other country is clean.

The 2018 paper was sold as a cheating row. The files since then read as a repeating sample. Weaker schools decline. Lower-attaining pupils inside the schools that do take part go missing. Weights shuffle the remaining tickets and the bottom row stays thin. Ministers still get a number they can hold up. The age group that number is supposed to describe is not fully in the room.

Frequently Asked Questions

What is the PISA test and who sits it?

PISA is an OECD study of 15-year-olds, run every three years, covering reading, maths and science on a two-hour computer test plus questionnaires. Science led the 2025 round. More than 90 education systems took part that cycle, up from about 80 in 2018 and 81 in 2022, and headteachers also fill in a school questionnaire on resources and intake.

How does a country get flagged for a weak PISA sample?

The OECD wants at least 85% of sampled schools and 80% of sampled pupils, after weights. If original school response sits between 65% and 85%, replacement schools can be used, with a higher after-replacement target. Miss those bars and a non-response bias analysis is required; Jerrim found that between 2000 and 2015 only three of 23 such studies led to a country being excluded from the international results.

Which pupils can be left out of PISA on purpose?

National centres may exclude some schools before sampling, with an OECD cap of 2.5% of schools for those frame cuts. In England that has covered special schools, hospital schools, secure units, international immersion schools and pupil referral units. Separate pupil exclusions, often tied to severe special educational needs that block access to the test, sit under a 5% yardstick the OECD links to about five points of possible bias on the mean.

Who ran England’s PISA fieldwork in 2018 and 2025?

The National Foundation for Educational Research recruited the 2018 England sample under OECD procedures and wrote the national report. Pearson delivered the 2025 study in England, Wales and Northern Ireland. Scotland runs a separate national centre, which is why its test window and eligibility rules can diverge from the rest of the UK in the same cycle.

What did Jerrim ask the statistics regulator to do?

He called for the Office for Statistics Regulation to run an independent review of how UK PISA data are reported, with clearer rules for later rounds. He also argued the OECD should tighten technical standards, how strictly they are applied, and the adjudication process, which he said could not carry the claim that PISA files are fully comparable across countries.