AI: The postmodern engine

Why large language models are designed to deceive us

Difference Engine Number 2 by Carsten Ullrich, CC BY-SA 2.0

This article is a response to Susan Pickard’s essay Why AI Consciousness is a Feminist Issue.

It was no surprise to me that Richard Dawkins has been musing on whether large language models, or in popular parlance ‘artificial intelligence’ platforms, are conscious. For Dawkins and his followers, humans and other animals are merely flesh machines programmed by genetic and memetic code. Evolution itself is considered biological automation which constantly iterates, mostly failing, but every so often producing a spectacular success in its own terms, such as the honey bee, or a virus which spreads widely before killing its host. It’s all about replication, which is why we say a social media post or the latest smartphone app has ‘gone viral’.

For those tech bros who consider themselves the dominant species, as measured by their ability to persuade other people to invest billions of dollars in their schemes, Dawkins’ seminal ‘selfish gene’ model is an affirmation that they are not only justified in seeking world domination, but are the masters of technological innovation and therefore the pinnacle of evolution of human society.

The hype cycle is self-reinforcing; the more a geek that got lucky is convinced of their brilliance, the more investors believe in them. Just don’t mention their financial disasters and technical blunders along the way, and we can all keep the narrative going. That start-up which burned millions of dollars, producing nothing of value, is just proof that evolution works.

Dawkins has returned the favour to the neo-Darwinian tech bros by adding his considerable intellectual prestige, not only as an elite college professor but a popular non-fiction author over many decades, to the large language model hype about ‘conscious’ machines.

Did you ever have the impression that your smartphone was listening to you or reading your mind, because its predictive text keyboard knew exactly what you were going to type next? Your phone probably is listening to you, whether you activated this feature or not, but the predictive text feature is merely using a statistical model to guess what you want to type, based on your previous text input and that of millions of other users who write in your language.

Large language models are essentially an evolution of the same predictive text engine, combined with a huge library of pirated and shredded books which was used to train those models to produce natural language outputs, rather than just links to information as in the classic search engine response.

Mary Harrington recently expressed alarm that Anthropic has been buying up rare books to digitise and destroy them. If you have been following the details of the Bartz versus Anthropic settlement, this will not be news to you:

“its service providers stripped the books from their bindings, cut their pages to size, and scanned the books into digital form — discarding the paper originals.”

One thing that Ray Bradbury and François Truffaut got wrong in the novel and film Fahrenheit 451, respectively, was that in our time, books would be burned. Instead, if they don’t go to landfill, books are recycled as post-consumer waste. Books take up valuable warehouse space that could be used for e-scooters, vibrators and any number of other battery-powered pleasure devices. If the people do not wish to read books, they will cease to exist.

Of course, the publishing industry, libraries and other parties have been purposefully destroying the written word for a very long time. In 1992, I discovered that my own university library was sending many tonnes of books, known in the trade as ‘discards’, to landfill. The horny-handed labourers given the task of throwing these books into the trash found the task objectionable, while university management had no such compunction.

On another occasion, a specific book still on order from a different university library turned up in the back of a Luton van stuffed to its roof with discards, parked on wasteground at Brick Lane market in East London. This was back in the day when the neighbourhood was still a liminal space between the city and its underworld, rather than the gentrified hipster haunt of today. No doubt the van driver had been paid to dump the university’s books, but criminal enterprise saved at least a few of them.

To provide an example of how difficult rare books are becoming to obtain, I was searching the book catalogues recently for a copy of Samuel Pisar’s 1970 work Coexistence and Commerce, which was popular enough at the time to be produced in a paperback edition and several translations. The British Library has two copies in storage. Viewing either would have required advanced notice of a trip to London and several days spent in its fine Reading Rooms, since our national library does not loan out its precious works for very good reasons.

Alternatively, I could have enrolled at one of the few universities among the Russell Group which still hold a copy of this book, in the parallel universe where I would be admitted to any of these institutions. eBay had zero copies for sale worldwide, and Amazon had nothing to offer either. I turned to the rare book catalogues populated by niche booksellers and found two copies in North America, one of which was listed at an extortionate price, plus $80 for shipping. I ordered the cheaper, and realised my mistake once it did not arrive after a couple of months.

The courier couldn’t locate the book entrusted to its care, and offered a refund. The customs backlog caused for entirely political reasons has made shipping books internationally more difficult in a world of supposedly frictionless e-commerce, it seems.

Having given up on ever holding a physical copy in my hands, a saved search on eBay later turned up one hardback copy in England, which I had delivered at the same time the lost copy from the United States arrived on my door mat. Therefore out of the three copies in the world for sale in the last year, I inadvertently own two of them now.

As explained by Bender and Gebru et al five years ago, the ‘AI’ language model does not feature natural language understanding; it is a stochastic parrot. I call the large language models postmodern engines; they embody the belief that there is no truth but the power, of their owners, to mass-produce hegemonic narratives at virtually unlimited scale. That is the only value proposition behind the trillion-dollar valuation of Anthropic as it prepares to debut on the stock market. Of course, the product is marketed entirely differently.

What sets so-called ‘artificial intelligence’ apart from the routine destruction of the written word is the claim that large language models add value to the books that are shredded. It is on this basis that some people imagine human employees can be replaced with AI tokens, used to pay AI ‘agents’; perhaps a misnomer, since software bots have no agency of their own.

In the era of ‘too long; didn’t read’, what is needed is greater reading comprehension, not outsourcing our critical faculties to frequently incorrect but very confident machines. There have already been multiple legal cases in Britain featuring hallucinated case law references.

Once we understand how large language models work, asking if they are conscious makes about as much sense as asking if a photocopier has a soul, because, given appropriate inputs, it can produce works of Russian literature.

In order to maintain the fiction of artificial intelligence, it is necessary to strip attribution from the works that are ingested, implying that the machine ‘knows’ something. Otherwise, we just have a ruinously expensive, energy-hungry search engine returning matches to prompts in natural language. As Gary Marcus has noted, this business model is financially unsustainable.

It is this performance of the authorial voice in response to ‘AI’ prompts which confuses users into thinking they are interacting with a conscious entity.

This voice is a by-product of the tech bros’ need to present the AI start-up as transformationally innovative, and thereby attract billions of dollars in capital from your pension fund - the second-tier investors who pay off the venture capitalists who always planned their ‘exit event’.

Your average pension fund manager or institutional investor is looking for stock which is going to pay dividends over the long term, of course, and so the supposedly transformational aspect of the hyped technology is essential to the entire enterprise. A slightly better search engine isn’t going to do that.

As for a potential feminist ghost in the machine, Bender and Gebru et al found artificial intelligence degrades entropically by recycling its own output as input, including the content of websites such as Reddit, considered a valuable source of knowledge by the kind of person who works in an AI startup. Very few of the online posts were written by females of any kind, let alone feminists. It’s called a broligarchy for a reason, and I noticed no major players in the AI gold rush led by women.

I agree with Susan Picard that students should use large language models. The smarter of our youth will figure out the limitations of these infernal machines and see beyond the hype of the broligarchy. They will return to reading the primary sources, while these remain available.

The remainder, who attempt to pass off the work of the machine as their own, both in student life and in the workplace, will find themselves the first to be replaced by the next iteration of the postmodern engine.

Thanks to all the new readers who have subscribed following a recommendation from Brian Merchant. If you aren’t familiar with Blood in the Machine, I highly recommend its coverage of the ‘AI’ debacle. Hammers up!