AI language models: The risks of textual incest

Every second panelist, almost every second moderator uses the same conciet: “I asked ChatGPT what I should say.” What was once intriguing has quickly become a stale routine. ChatGPT seems to be revitalizing our world by enabling artificial intelligence to describe it. We humans were already able to do the same thing, of course, but because it has always been that way, we apparently got bored with ourselves. Suddenly, AI makes creativity exciting again. 

If there's a trend in the technology sector, you'll notice it here at South by Southwest, one of the world's largest digital conferences, in Austin, Texas. Two years ago it was blockchain; 2022 there was no escaping NFTs, and the Metaverse was supposedly set to replace the real world altogether at some point. This year, generative AI is making its triumphant appearance in the convention center space. ChatGPT is being asked about everything and applauded like a toddler who can say “mommy.”

There is one guest who has every right to jump in and ask ChatGPT a question, but doesn't (again, the moderator already did that). Greg Brockman is one of the founders of OpenAI and the inventor of the AI application that is now re-describing the world to us. Brockman soberly analyzes the genesis and the potential future, the opportunities and risks of ChatGPT. In doing so he utters a sentence that is central to explaining the tool's success: Every business model is a language model. Put differently, the world is based on communication. And now AI is suddenly just as good at it as we are.
 
But is it really? These language models are trained with texts from the Internet. They analyze what has already been described in order to deduce what is still waiting to be described. All future communication is derived from past communication.
 
That's no problem for us, because we can learn from history. Whether that's the case for language models remains to be seen. A self-reinforcing mechanism is at work here which could potentially cause creative progress to implode at some point.
 
Fed with historical data, AI spits out more and more predictions for probable text sequences, all of which are nothing more than ever-new re-combinations of what already exists. Meanwhile, people prouce texts, pieces of music, poems that are sometimes quite original, and which in turn find their way into the growing memory storage medium of humankind. Innovation and re-combination complement one another. Forever?
 
An international research team has calculated that the pool of new high-quality text data for the AI will be exhausted by as early as 2026. After that, it will train more and more with texts that it has created itself. This is text incest, and in this case not a moral but an intellectual problem. 

Miriam Meckel: The threat of intellectual boredom

 
The AI presses all communication through a highly efficient mixer. What it generates is in turn fed into the global knowledge pool of the Internet  – a self-reinforcing mechanism that augments the remix while originality shrinks. At some point, however, there will be a tipping point. This is the dynamic of a crisis of meaning in the data society, and it is reminiscent of history.
 
Just as structured products contributed to the 2008 crash of the financial markets and the subsequent crisis, generative AI could be heading for a melting point of original communication. The AI creates complexly structured derivatives of thought – in so many variations that we can hardly recognize the original any more. Constant remixes of already-used phrases are blended in new combinations until all that's left is a thought smoothie.
 
It will be the era of continuous clichés. Automated communication will become more and more important, with AI bots answering emails that were written by AI bots. A secondary market of automated communication is emerging that will soon be larger than the primary market of human communication – an intellectually boring world of reconstituted thoughts.
 
Actually, according to Greg Brockman at SXSW, ChatGPT is supposed to help dissolve our creative blockades and assist us in coming up with new ideas. And indeed, the tool can be very useful in that way. To understand the impact of new technologies, says Brockman, you have to understand people first and foremost. They have what Hanna Arendt called the capacity for the unexpected – unless they have been rationalized away for the sake of convenience. Let's hope we are still capable of surprising ourselves, historically speaking.