What is model collapse?

Model collapse is the progressive degradation of a generative model's output quality when it is trained, directly or indirectly, on data produced by other AI models rather than on real data generated by people. The phenomenon was first described systematically by Ilia Shumailov and colleagues in a paper published in Nature in July 2024, titled “AI models collapse when trained on recursively generated data”: the authors showed that training models recursively on output generated by the previous generation narrows the statistical variety of the data at every cycle, the tails of the distribution, meaning rare events, minority opinions, edge cases, disappear first, and the final model produces text that is increasingly uniform and less representative of the real world. It is the equivalent of a photocopy of a photocopy: every pass loses definition, even when the first output looked perfect.

Two stages, not one

The mechanism unfolds in two stages. In the early stage the model stops representing the tail of the distribution well: statistical minorities, rare phrasings, uncommon styles. This is a silent error, hard to spot with aggregate metrics. In later stages the collapse spreads to the center of the distribution: the model converges toward a handful of typical answers, repeats the same lexical patterns, and loses the ability to generate variety even where it would have the competence to do so. The risk grows with the web itself: as model-generated content accumulates online unlabeled, every new model trained on generic scraping risks absorbing a growing share of synthetic data without knowing it, in a cycle the study's authors call recursive contamination.

An enterprise case

A concrete enterprise case: a company training a classification model on customer support tickets, which also folds into the training set answers already generated by a previous chatbot to boost volume, risks reinforcing the chatbot's errors instead of correcting them, narrowing the variety of answers to a handful of generic formulas over time. The same risk applies to language models trained on web corpora increasingly polluted by generated text: without a guaranteed share of verified human data, every training cycle starts from an already impoverished base.

Why it matters for decision makers

For decision makers, model collapse changes the risk calculus around training data governance: it is no longer enough to ask whether the data is good quality, one has to know whether it is AI-generated and in what proportion. This requires traceability of provenance, minimum quotas of verified human data in the training set, and the discipline of not confusing collapse with the controlled production of synthetic data, which remains a valid practice when generated data complements, rather than replaces, a validated real base. The topic also touches model monitoring in production, since a drop in answer variety is one of the signals good monitoring should catch before the damage propagates into the next training cycle.