punk.science Halifax, NS · Canada
Research · STORM · 2026
Verification-first research · grounded assessment

Is AI Making Us Stupid?

PISA 2025 recorded the lowest scores in its history, and AI chatbots are getting the blame. We checked the OECD's own data and the controlled experiments. The evidence says how students use AI matters, and that the decline started before the chatbots arrived.

putting AI to work on reliable web research since 2026… [ ok ]
20 sources 22 verified 2 single-source 3 contested 0 fabricated
punk.science/Tech Briefs/Brief 04
Answer up front

Not on this evidence — and the PISA data cannot say so.

The OECD's PISA 2025 results, released in September 2026, recorded the lowest OECD-average performance ever observed in science, reading and mathematics.14 A recent piece in The Conversation cites those results under the headline that the release of AI chatbots “has harmed teens' learning”.3 The data does not support that causal claim. PISA 2025 is a single snapshot of self-reported AI use, the OECD states that its AI–score gaps “do not necessarily imply a negative impact of AI use”,1 and the score declines began before chatbots were publicly available.157

What the evidence does support is narrower and more useful. How a student uses AI matters more than whether they use it. Students who have chatbots summarise or draft for them score lower; students who use AI weekly to help them learn score about the same as non-users.126 Controlled experiments show the same split from the other direction: unrestricted AI acts as a crutch that leaves students worse off once it is taken away, while AI built to teach produces real gains.91213 The evidence does not show that AI is making us stupid. It does show that using AI to skip the thinking undercuts the learning.910 [ ok ]

01 · What PISA 2025 actually measured

A real decline that started before the chatbots.

PISA 2025 was the largest edition to date: more than 760 000 students, representing around 33 million 15-year-olds in 91 countries and economies.1 The OECD reports that reading has declined since peaking around 2012, science has declined more gradually over the past decade, and mathematics has fallen especially sharply since 2018.1 Across 35 OECD countries with comparable data, mean reading performance dropped 11 points between 2018 and 2022 and a further 14 points between 2022 and 2025.1 The share of “hasty readers” — students who rush through texts and give quick but incorrect answers — almost doubled between 2018 and 2025.1

The timing is what undercuts the chatbot explanation. Most education systems sat the PISA 2022 test between March and August 2022,7 and the OECD notes that AI chatbots only became publicly available in late 2022.1 The 11-point reading drop between 2018 and 2022 was therefore measured before chatbots were publicly available.17 The OECD's own foreword says that “a decline in reading was already visible well before the pandemic hit.”1 Writing in The Conversation's French edition, Margarida Romero reaches the same conclusion: the PISA data do not allow the decline to be attributed to AI, because it began before AI spread.5

The cohort that sat PISA 2025 was already behind earlier in childhood. In the PIRLS study of fourth-graders, 21 countries had lower average reading achievement in 2021 than in 2016 — two-thirds of participants.8 The OECD reports that, for most countries, the 2022–2025 PISA declines were “foreshadowed by the results of similar cohorts at a younger age” and “do not necessarily reflect post-2022 phenomena.”1 Adults show the same pattern. The OECD's 2022/23 Survey of Adult Skills found that adult literacy had stagnated or declined over the previous decade in most participating countries.16 Its data were collected in 2022–2023,16 a window that began before chatbots were publicly available.1

“People quickly turn to explanations such as the impact of the COVID pandemic, but a decline in reading was already visible well before the pandemic hit.”

— OECD, PISA 2025 Results (Volume I), foreword [1]
02 · What PISA says about AI

The answer depends on what the AI is doing.

PISA 2025 is the first cycle to ask 15-year-olds how they use AI chatbots for schoolwork.16 There is no earlier PISA measurement of AI use to compare against, so the data cannot show change over time. Use is widespread: on average across OECD countries, only 13.7% of students said they never or almost never use AI for any of the schoolwork purposes PISA asked about.2 Regular, almost-daily use was reported by about one student in five.1

PISA asked about four purposes, and they do not behave alike. The OECD-average science scores below are adjusted for socio-economic status:2

Uses AI to…NeverWeeklyDailyWeekly vs never
Summarise a text508490477−18
Draft writing509489481−20
Research a new topic505497486−8
“Help me learn”499501494+2

“Help me learn” is the most commonly reported purpose,1 and weekly users in that category score about the same as non-users.12 The OECD says so directly: weekly users and non-users of AI for general learning purposes “perform similarly.”1 The gaps are concentrated where the chatbot produces the work — summarising and drafting.12

An independent reanalysis of the same microdata by economists at Yale and the Universidad de Chile reaches the same split with stronger controls. Comparing schoolmates and holding total AI use fixed, it finds that drafting and summarising with AI are associated with lower scores, while research and learning go with higher ones.6 Its authors note that if weak students simply turned to AI because they were struggling, every purpose should show lower scores, and the data rejects that prediction.6

Both analyses find that richer students use AI more. The OECD reports that students who use AI more frequently tend to come from more advantaged backgrounds.1 The reanalysis finds that controlling for socio-economic status makes the negative gradients steeper, not flatter.6 Neither analysis turns the association into a cause, and the OECD says so plainly:

“These relationships do not necessarily imply a negative impact of AI use on science performance, but may reflect a complex mix of who adopts AI and how they use it.”

— OECD, PISA 2025 Results (Volume I), chapter 4 [1]

Teaching appears to make a difference. Among students who use AI to help them learn once a week or more, those who were taught in school to assess the quality of AI-generated information slightly outperform both non-users and infrequent users.1 Those learning opportunities are reported more often by socio-economically advantaged students.1

03 · Checking the Canadian numbers

Most of the figures check out. The headline doesn't.

Canadian students scored 510 in science, 490 in reading and 485 in mathematics in PISA 2025. That is above the OECD average, but down 24, 37 and 42 points respectively since 2006.4 The Conversation's article, by McGill learning scientist Adam Dubé, draws its Canadian AI figures from the OECD tables.3 We checked them against the published data:2

Claim in The Conversation [3]OECD table [2]
Daily AI summarisers score 38 points lower in science−37 (SES-adjusted)verified
Canadian teens using AI to draft score 42 points lower−42 (SES-adjusted)verified
50% of Canadian teens never use AI to generate text50.2% never use it to draftverified
38% of Canadian teens use AI every week to learn37.7% weekly or dailyverified

The numbers are largely accurate. The problem is what is built on them. The article's own text concedes that the decline since 2022 coinciding with ChatGPT's release “is not evidence of causality”, yet its headline and text state that the release of AI chatbots “has harmed teens' education.”3 The data it cites supports an association, by purpose, in a single year.12 It does not support the causal headline.

Over-association guard
“Years of learning” figures rest on the OECD estimate that 15-year-olds gain about 20 points over one school year. The OECD calls this useful context that “should be interpreted cautiously”.1 The AI gaps are differences between groups of students that, in the OECD's words, may reflect “a complex mix of who adopts AI and how they use it”.1 They are not measured losses of school time.
04 · What controlled experiments show

Crutch or tutor — the design decides.

Surveys cannot separate cause from selection, but randomised experiments can. The strongest school-age evidence comes from a field experiment with nearly a thousand high-school maths students, published in PNAS.9 Access to a plain ChatGPT-style interface improved practice grades by 48%. When access was later removed, those students performed 17% worse than students who never had it.9 A second version with prompts designed to safeguard learning raised practice grades by 127% and largely mitigated the later harm.9 The authors' summary: without guardrails, students use GPT-4 as a “crutch” and then perform worse on their own.9

A randomised study of 91 university students points the same way. Students who researched a topic with ChatGPT experienced significantly lower cognitive load than those using Google, but produced lower-quality reasoning and argumentation.10

The same tools, built differently, help. In a randomised trial with college students, those using an AI tutor designed around the same pedagogical practices as the in-class lessons learned significantly more, in less time, than students in an active-learning class.12 A World Bank randomised trial in Nigeria, using Microsoft Copilot (GPT-4) as a virtual tutor for first-year senior secondary students over six weeks, raised overall scores by 0.31 standard deviations and English scores by 0.23.13 The authors estimate that as equivalent to 1.5 to 2 years of business-as-usual schooling.13

A 2024 meta-analysis of 69 experimental studies found that ChatGPT improves academic performance and higher-order thinking propensities, and reduces mental effort.11 Its authors add two caveats that matter here. Methodological limitations such as the lack of power analysis call for caution, and future studies should separate the quality of ChatGPT's output from the student's own performance.11 The PNAS experiment shows why that matters: grades rose while students had the tool and fell below the control group once it was removed.9 Most of the meta-analysed studies were at university level.11

AI does the workAI supports the learner
PISA 2025 associationLower scores (summarise, draft)26Similar or higher (learn, research)26
Randomised experimentsWorse once the tool is removed; shallower reasoning910Measurable gains1213
What made the differenceUnrestricted, answer-giving access9Guardrails, pedagogy, structure91213

“If we get this right, AI becomes a scaffold, not a crutch. A tool for thinking, not a substitute for it.”

— OECD, PISA 2025 Results (Volume I), foreword [1]
05 · Adults and cognitive offloading

Less effort, less checking — reported, not yet proven.

The adult evidence points the same direction as the experiments but is weaker. In a survey of 319 knowledge workers sharing 936 real examples, published at CHI 2025, higher confidence in generative AI was associated with less critical thinking, while higher self-confidence was associated with more.14 A separate study of 666 participants found a significant negative correlation between frequent AI-tool use and critical-thinking ability, mediated by increased cognitive offloading. Younger participants showed higher dependence on AI and lower critical-thinking scores.15

Both studies are correlational or self-reported.1415 Together with the experimental finding that chatbots lower mental effort,1011 they show that people offload thinking to AI. None of these studies measures long-term change in general cognitive ability.

Bias note: every PISA-based claim in this brief — including The Conversation's, CBC's, Romero's and the Yale/Chile reanalysis — traces to one self-reported, cross-sectional OECD dataset. Independence on the causal question comes only from the randomised experiments in section 04.

What remains unverified / contested

What this brief deliberately does not claim.

This brief was built verified-only. Every claim above is corroborated by independent sources or checked against primary data, or is explicitly attributed to the single study that produced it. The items below did not clear that bar.

Contested — “two years behind.” The Conversation states that the 38% of Canadian teens who use AI weekly to learn “score two years behind.”3 The OECD table does not reproduce this. Canadian weekly “help me learn” users score 6 points below non-users unadjusted and 12 points below after the socio-economic adjustment, less than one notional school year.2 The global claim that weekly users “score one year behind”3 holds only for summarising and drafting (−18 and −20). It does not hold for research (−8) or learning (+2).2 We could not identify another table that would produce the article's figure.

Contested — the causal headline. “The release of AI chatbots has harmed teens' learning” is contradicted by the article's own caveat and by the OECD's explicit statement that the associations do not necessarily imply a negative impact.13

Contested — “Your Brain on ChatGPT.” MIT's EEG study, “Your Brain on ChatGPT,” is an arXiv preprint with 54 participants, only 18 of whom completed the final session.17 Its lead author has said it “cannot and did not show” “dumbness in the brain, no stupidity,”19 and a published comment raises concerns about its sample size, reproducibility, EEG methods and reporting.18 It is not used as evidence in the body of this brief.

Single-source. The OECD's cohort argument rests on a modest correlation (r = 0.34, 23 countries) between PIRLS and PISA trends, reported only by the OECD.1 The knowledge-worker confidence finding comes from one self-report survey.14 The size of the purpose effects comes from one working paper that has not been peer-reviewed.6

Excluded — retracted. A 2025 meta-analysis reporting large ChatGPT learning benefits (Wang & Fan, Humanities and Social Sciences Communications) was retracted on 22 April 2026.20 It is not used.

Excluded — opinion. Experts quoted by CBC attribute the decline to digital distraction, AI use and teacher shortages.4 These are informed opinions, not measurements, and they are not asserted here as causes.

Not established. No study we found follows the same students over time to measure whether chatbot use changes their learning trajectory outside a short experiment. Until that evidence exists, “AI is making teens stupid” and “AI is harmless” are both unproven.

06 · Verification summary

27 material claims, audited.

22
Verified
2
Single-source
3
Contested
0
Fabricated

Verified-only build. PISA claims were checked against the OECD's full report and its published data tables, not press coverage. Four candidate sources were dropped: one retracted, one that could not be fetched, one opinion stated as fact, and one unverified figure. Confidence: High that PISA 2025 cannot support a causal claim against AI. High that the score decline predates chatbots. Moderate–high that answer-giving AI use impairs learning while guided use does not — correlational and experimental evidence agree, but the experiments are few and short.

Sources & independence

The register.

  1. OECD — PISA 2025 Results (Volume I): Future-Ready Students (September 2026) Primary · originator · full report PDF read directly
  2. OECD — PISA 2025 Volume I data tables, Chapter 4 & Annex B1 (incl. Table I.B1.4.61; also stat.link/o1j4k0) Primary data · same origin as [1]
  3. Dubé — The release of AI chatbots has harmed teens' learning and Canada needs to act, The Conversation (14 October 2026) Secondary · claimant · derives from [1][2]
  4. Wong — Canadian teens score well on OECD tests, but sharp drops in reading, math raise concerns, CBC News (11 September 2026) Secondary · derives from [1]
  5. Romero — PISA 2025 : l'IA est-elle responsable de la baisse des résultats ?, The Conversation (10 September 2026) Secondary · expert analysis · derives from [1]
  6. Lepe & Contreras — Substitute or Complement? Generative AI Use and Student Achievement: Evidence from PISA 2025 Primary · working paper, not peer-reviewed · independent reanalysis of [2] microdata
  7. NCES — PISA 2022 Technical Notes Primary · independent · testing window
  8. IEA — PIRLS 2021: Trends in Reading Achievement Primary · independent
  9. Bastani et al. — Generative AI without guardrails can harm learning: Evidence from high school mathematics, PNAS (2025) Primary · peer-reviewed field experiment · independent
  10. Stadler, Bannert & Sailer — Cognitive ease at a cost, Computers in Human Behavior (2024) Primary · peer-reviewed randomised study · independent
  11. Deng et al. — Does ChatGPT enhance student learning? A systematic review and meta-analysis of experimental studies, Computers & Education (2024) Primary · peer-reviewed meta-analysis · independent
  12. Kestin et al. — AI tutoring outperforms in-class active learning: an RCT, Scientific Reports (2025) Primary · peer-reviewed RCT · independent
  13. De Simone et al. — From Chalkboards to Chatbots: Evaluating the Impact of Generative AI on Learning Outcomes in Nigeria, World Bank WP 11125 (2025) Primary · RCT working paper · independent
  14. Lee et al. — The Impact of Generative AI on Critical Thinking, CHI 2025 Primary · peer-reviewed self-report survey · independent
  15. Gerlich — AI Tools in Society: Impacts on Cognitive Offloading and the Future of Critical Thinking, Societies (2025) Primary · peer-reviewed, correlational · independent
  16. OECD — Survey of Adult Skills 2023: Insights and Interpretations (December 2024) Primary · same organisation as [1], separate survey
  17. Kosmyna et al. — Your Brain on ChatGPT: Accumulation of Cognitive Debt when Using an AI Assistant for Essay Writing Task (arXiv, 2025) Primary · preprint, not peer-reviewed · contested; excluded from the body
  18. Stankovic et al. — Comment on: Your Brain on ChatGPT (arXiv, December 2025) Primary · independent critique of [17]
  19. Jones — Does Using ChatGPT Really Change Your Brain Activity?, Scientific American / Nature (June 2025) Secondary · independent reporting on [17]
  20. Retraction Note — Wang & Fan, The effect of ChatGPT on students' learning performance…, Humanities and Social Sciences Communications (22 April 2026) Retraction notice · excluded from the body
← All briefs in Punk Science: Tech Briefs