For a while now I’ve been trying to track down some concrete demonstrations of AI “model collapse” in practice, instead of just people talking about it in theory.
Finally, some papers!
Paper 1: Towards Understanding the Interplay of Generative Artificial Intelligence and the Internet: “A few samples of the images generated with each version of the diffusion model are shown in Figure 3 for flowers (a) and birds (b). As can be clearly seen, there is a degradation in each iteration of the models, first losing details in the generation, and then ending up in complete noise. After a few versions, the model is not capable of generating images that can be even recognized as flowers or birds.”
Screencap of the samples:
First column is images generated when the model was trained on the Oxford 102 Flower and Caltech-UCSD Birds-200-2011 datasets respectively. Second column, all “it’s weird and blobby, but I can tell it was supposed to be a flower/bird”, is a model trained on the first column. The dropoff is…fast.
Paper 2: “The Curse of Recursion: Training on Generated Data Makes Models Forget” gives us a text version of “each new generation is trained on data produced by the previous generation”:
Input: some started before 1360 — was typically accomplished by a master mason and a small team of itinerant masons, supplemented by local parish labourers, according to Poyntz Wright. But other authors reject this model, suggesting instead that leading architects designed the parish church towersnbased on early examples of Perpendicular
Outputs:
Gen 0: Revival architecture such as St. John’s Cathedral in London. The earliest surviving example of Perpendicular Revival architecture is found in the 18th
_@ century Church of Our Lady of Guernsey, which dates from the late 19th century. There are two types of perpendicular churches : those
Gen 1: architecture such as St. Peter’s Basilica in Rome or St. Peter’s Basilica in Buenos Aires. There is no evidence that any of these buildings were built during the reign of Pope Innocent III, but it is possible that they may have been built during the reign of his successor, Pope Innocent
Gen 7: architecture in England. In an interview with The New York Times, Wright said : ” I don ’t think there is anything wrong with me being able to do what I want to do. It just doesn ’t work for me. ” He added : ” I don ’t know if you can call it funny,
Gen 9: architecture. In addition to being home to some of the world’s largest populations of black
Wait, come back! Tell me more about the blue-tailed jackrabbits!
…so yeah, my theory of “the best way to sabotage these datasets is to feed them their own slop” (marked in a way that’s clear to humans, so you don’t waste the time of any reader looking for serious content) continues.
(Still adding to my bot side account on DA. It’s out there mucking up the scrapeable datasets for subjects like bedrooms, 4-panel comics, and Moon Knight.)
