What Is Model Collapse? The AI Training Problem Explained

Model collapse is what happens when an AI system is trained, generation after generation, on data that earlier AI systems generated — and it gradually forgets the rare, unusual, and nuanced parts of reality, drifting toward repetitive, generic, or outright nonsensical output. Researchers first demonstrated and named the effect in 2024, and a 2026 physics study has now shown a surprisingly simple way to stop it in a class of statistical models: keep at least one genuine, non-synthetic data point in every round of retraining.

What is model collapse?

Model collapse is a degenerative process in machine learning where a model's quality and diversity erode because it was trained, directly or indirectly, on content produced by other AI models rather than purely on original human-created data. The term comes from the 2024 paper by Ilia Shumailov and co-authors, published in Nature (volume 631, pages 755–759) and first circulated as a preprint titled "The Curse of Recursion: Training on Generated Data Makes Models Forget." A broader overview of the concept and its disputed real-world scope is also kept current on Wikipedia's model collapse page.

The researchers showed the effect across several kinds of models — large language models, variational autoencoders, and Gaussian mixture models — and found a common pattern: when a model is trained indiscriminately on the output of a previous model, the tails of the original data distribution start to vanish. In plain terms, the unusual, rare, and minority examples in the data disappear first, and what's left behind drifts toward an average, flattened, and ultimately distorted version of reality.

How model collapse actually happens

The Shumailov paper traces collapse to three compounding sources of error that appear whenever a model learns from another model's output instead of from the original data:

None of these errors is fatal on its own. The problem is recursion: feed a model's slightly-off output back in as training data for the next model, and each of those small errors compounds. Repeat that cycle across several "generations" of models, and the result is a runaway drift away from the real-world distribution the first model was trained on.

The example researchers use to illustrate it

One demonstration from the original study has become the standard way to explain model collapse to a general audience. The researchers fine-tuned a language model on a passage about medieval church architecture, then repeatedly retrained new "generations" of the model on each previous generation's own output, with no new human text added at any point. According to the paper, by around the ninth generation, a prompt about church towers produced a rambling, unrelated passage about jackrabbits — the model had entirely lost the thread of the original topic. Along the way, the Shumailov team also found that recursively trained language models tended to repeat the same phrases far more often than models trained on human-written text.

The paper describes two broad stages of this decline. In early model collapse, the model starts losing information about the tails of the distribution — rare topics, unusual phrasing, minority viewpoints — while overall performance on common benchmarks can actually look stable or even improve, which makes the problem easy to miss. In late model collapse, the damage becomes obvious: the model's outputs lose most of their variance, blend together, and increasingly bear little resemblance to the data it was supposed to learn from.

Why it matters more in 2026 than it did before

Model collapse was always a theoretical curiosity, but it has become a practical concern as AI labs run short of fresh human-written text to train on and lean more heavily on synthetic, model-generated data to fill the gap. King's College London's May 2026 summary of its own new research put it plainly: high-quality human-written text for training large language models may become scarce, pushing developers toward synthetic data generated by AI models themselves — precisely the ingredient that causes collapse if used carelessly.

That shift matters because today's internet increasingly contains text, images, and audio that were themselves produced by earlier generations of AI tools. Any model trained by scraping the open web now risks absorbing some share of AI-generated material without anyone deliberately curating it in — a slower, less controlled version of the same closed loop the original researchers built on purpose in their experiments.

The 2026 study: one real data point can break the loop

In May 2026, a team from King's College London, the Norwegian University of Science and Technology, and the Abdus Salam International Centre for Theoretical Physics published a paper in Physical Review Letters — posted as a preprint titled "Lost in Retraining: Roaming the Parameter Space of Exponential Families Under Closed-Loop Learning" — that looked at the mathematics of model collapse directly. Studying a class of statistical models known as exponential families, the researchers showed that standard maximum-likelihood training, run in a fully closed loop on a model's own generated data, will always lead to collapse — eventually degenerating into meaningless output.

Their key finding is the fix: adding even a single data point from outside the closed loop — a genuine, non-synthetic example — at each retraining round was enough to prevent collapse, and this held true even when the AI-generated data points vastly outnumbered that one real example. The team reported similar protective behavior in Restricted Boltzmann Machines, a different model family, suggesting the result may generalize beyond the exponential-family models they studied directly.

"By focusing on a simple model, we can establish why adding just one data point prevents them from generating gibberish," said Yasser Roudi, Professor of Disordered Systems in King's College London's Department of Mathematics, in the university's announcement of the study. Roudi added that the goal is for "computer scientists [to] have the tools to prevent this potentially disastrous scenario" as AI development continues. The researchers caution that their results so far rest on small, analytically tractable models rather than full-scale neural networks, and the next step is testing whether the same principle — that a trickle of real data can anchor a system against collapse — holds up at the scale of today's large language models.

Is model collapse already affecting the AI you use?

There's genuine disagreement among researchers about how much this matters for commercial, production AI systems today. The original 2024 paper and the 2026 physics study both model a fairly extreme, fully closed loop: a model trained only on the previous model's synthetic output, with no fresh real data mixed in. Some subsequent research argues that collapse is largely avoided in practice as long as synthetic data accumulates alongside real data over time, rather than replacing it generation after generation — a meaningfully less severe setup than the closed loops studied in the foundational papers.

That disagreement is exactly why AI labs now treat data provenance — knowing where training data actually came from, and how much of it is AI-generated — as a serious engineering problem rather than an afterthought. It's also why it's worth being skeptical of any claim that a specific chatbot or image generator has already "collapsed" based on a handful of odd outputs; the term describes a structural, cumulative failure mode demonstrated in controlled experiments, not a diagnosis for any one weird response.

Model collapse vs. other AI training problems

Model collapse gets confused with a few related but distinct ideas. It is not the same as training a smaller model to mimic a larger one on purpose — a legitimate and widely used technique called distillation, which has its own separate risks when it's done without the original model maker's consent (see our explainer on AI distillation attacks). Model collapse also isn't the same as a single hallucinated fact in one answer; it's a property of the training data pipeline that builds up across many retraining cycles, not a one-off mistake. And it's a different question from how a model reasons once it's trained — our guide to how reasoning models differ from standard LLMs covers that separately.

Early vs. late collapse at a glance

How researchers and AI labs are trying to prevent it

Across the research surveyed for this piece, a few mitigation strategies come up repeatedly:

• Keep real data in the loop. The 2026 Physical Review Letters result formalizes the simplest fix: don't let a training loop become fully synthetic. Even a small, consistent trickle of genuine human-generated (or otherwise verified non-model) data can anchor a system against drift.
• Accumulate rather than replace. Keeping prior generations of real data in the training set, instead of discarding it in favor of newer synthetic data, appears to slow or avoid collapse in some analyses — though this remains an area of active disagreement, as noted above.
• Track data provenance. Knowing whether a given training example originated from a human or a model is a prerequisite for applying any of the above fixes deliberately rather than by accident.
• Add priors or regularization. The King's College London team's mathematical analysis found that incorporating a prior belief during training — not just adding external data — can also help prevent the runaway drift that defines collapse.

What it means if you just use AI tools day to day

None of this means the chatbot or image generator you use today is quietly turning to gibberish. The collapse experiments in both the 2024 and 2026 papers deliberately built closed loops with no fresh real data — a worst-case scenario designed to isolate and measure the effect, not a description of how major AI labs currently train their flagship models. Commercial systems are trained on enormous, carefully filtered mixes of data, and labs have strong commercial incentives to avoid the kind of quality drift collapse describes.

That said, the research points to a few things worth watching as a reader rather than a researcher. First, as more of the open web fills with AI-generated text, images, and video, any team training a new model by scraping that web is, by default, doing a less controlled version of the same recursive loop the papers warn about — which is why data provenance and filtering keep coming up as priorities. Second, "it was trained partly on synthetic data" is not, by itself, a red flag; synthetic data is also used deliberately and carefully to fill gaps where real data is scarce, and the 2026 study's whole point is that doing so safely is possible with the right safeguards. Third, be skeptical of viral claims that a specific product has "collapsed" based on one strange answer — model collapse, as researchers define it, is a slow structural drift measured across generations of retraining, not a single glitchy reply.

What's next

The 2026 Physical Review Letters paper is explicit that its results, so far, come from small, mathematically tractable models rather than today's billion-parameter language models — the authors say testing the same "keep one real data point" principle on larger neural networks is the next step. Separately, the broader research community remains split on how urgently commercial AI developers need to worry about collapse in practice, given that most production systems already mix synthetic data with large amounts of real-world data rather than training in the fully closed loops the lab experiments used. Expect more papers over the next year trying to pin down exactly how much synthetic data a training pipeline can tolerate before the drift becomes visible — and more attention, from regulators and AI labs alike, on tools that can reliably tell human-written content apart from AI-generated content in the first place.

Written byTheo Park

Theo Park runs the AI desk at Pandromeda. He follows model launches from the frontier labs and the open-weight community, tracks the assistants and developer tools built on them, and explains what each release changes on pricing, capability and safety. His reporting leans on primary sources: model cards, technical reports, API documentation and the companies' own announcements.

More from Theo ParkAll AI coverage