GPT-3, LaMDA, Wu Dao… The blooming of “monster” AIs

Reading time: 30 minutes

Translation by AB – October 2, 2022

This article was published at the beginning of October 2022, that is to say two months before the “surge” of ChatGPT, a refined version of this “GPT-3” which is discussed here. But it is actually the same thing… Everyone can judge a posteriori the interest of the comments made here on a blooming phenomenon. Digital history is being written very, very quickly.

Paris – January 23, 2023

Foreword

Since 2019 and our exploration dedicated to “automatic” art, the progress of “artificial intelligence” (AI) is undeniable, led by new neuromimetic networks that we will call “monsters”, both by their prowess and by their dimensions. These AIs called “GPT-3”, “LaMDA”, “DALL-E”, “Wu Dao 2” or “M6” are now able to write, compose music or draw “like humans”.

This exploration goes to meet them and is divided into three parts. The first part offers an overview of their prowess, with some examples and references. The second part is more technical. It is mainly about their “astronomical” dimensions and their energy consumption. On the other hand, their functioning is not discussed (transformers, attention mechanisms, etc. – references are proposed). The third part, more “philosophical”, is the true destination of this exploration. It (re)questions the function of language and the nature of consciousness in the light of these AIs. For this, we will be accompanied by the following “witnesses”: Blake Lemoine (who saw a chimera), John Searle (who saw a “Chinese room”), Giorgio Agamben (who sees “devices”* everywhere) and Friedrich Nietzsche (who contemplates his own body).

Finally, those who are already familiar with the subject can perhaps be satisfied with this summary (defiantly written in a single sentence): The “monster” AIs, ingesting hundreds of billions of human examples at great energy costs, regurgitate a very credible language counterfeit that can make the most hardened believe in the manifestation of a consciousness while no “flesh” animates them.

* “Device” is the translation chosen here for the French word “dispositif” used by the French philosopher Michel Foucault. As stated here, “Dispositif” is translated variously, even in the same book, as ‘device’, ‘machinery’, ‘apparatus’, ‘construction’, and ‘deployment’”.

I – SOME FEATS OF “MONSTER” IAs

Jerome K. Jerome

The German artist Mario Klingemann “played” in 2020 with GPT-3, one of these monster text generators. Its use is very simple: you write a “prompt” in the form of a question or a short text and the generator “responds” with a text of a given length and corresponding in a way to the prompt. Mario Klingemann put forward this prompt to GPT-3:

The importance of being on twitter
by Jerome K. Jerome
London, Summer 1897
It…

Jerome Klapka Jerome was a British writer who had great success with “Three Men in a Boat“, a novel published in 1889 and which narrates the adventures of George, Harris, Jerome and the dog Montmorency undertaking a two-week boating holiday on the Thames¹. The book is peppered with comical anecdotes, but also with philosophical reflections about existence. What could Jerome K. Jerome have written about twitter if the little bird had spread to London at the end of the 19th century? The artificial intelligence GPT-3 responded with a long text that starts like this² :

It is a curious fact that the last remaining form of social life in which the people of London are still interested is Twitter. I was struck with this curious fact when I went on one of my periodical holidays to the sea-side, and found the whole place twittering like a starling-cage. I called it an anomaly, and it is.

The result is striking. At Klingemann’s sole prompt, GPT-3 “understood” that the text to be produced had to be a pastiche of Jerome Klapka Jerome (“…by Jerome K. Jerome”), having twitter as its theme (“The importance of being on twitter by… ”) during the summer of 1897 in London. But above all, the response of GPT-3 is surprisingly fluent. The first three sentences flow effortlessly into the theme and are composed in the first person, like the novel “Three Men in a Boat”. While London is referenced, there is also mention of that “sea-side” to which Jerome K. Jerome alludes in his novel³. GPT-3 thus seems to be able to string together “ideas” in a coherent and inspired way. But where does the final sentence “I called it an anomaly, and it is” come from? It contains a simple judgment on what GPT-3 has just said, and could appear in many contexts. Rather strange…

The anachronism of the prompt is unimportant: GPT-3 generates sentences related to the terms of which it consists. This reminds us that language always allows us to transitively associate any two things, such as a 19th century author and a contemporary social phenomenon. GPT-3 will therefore always propose an answer, or even answers as in the following example.

Jesus Christ

GPT-3 can also “chat” and produce short responses. A certain Jordan Moore exposed on twitter some of his conservations with a GPT-3 imitating Steve Jobs, Elon Musk, Cleopatra, Kurt Cobain or even… Jesus Christ! Here is the beginning and the end of this last chat⁴:

Jordan: Jesus, welcome to the show.
Jesus Christ: Greetings.
JM: And welcome to 2020. How do you think we’re doing?
JC: That’s a difficult question. I think things are heading in a good direction. I think people are slowly starting to turn back to God.
JM: Why do you think people are turning back to God?
JC: Well, it seems that every day more people are realizing that God is the only way to live.
JM: So humanity’s future is one of optimism? That’s a nice thought.
JC: Well, I’m not sure if it’s optimism or realism.
[… long dialogue …]
JM: Why do we have little time?
JC: Because I have to go soon. There are things going on that you are not aware of. But you don’t need to worry about it.
JM: Where are you going?
JC: It’s better if you don’t know. What’s important is that I’m always going to be with you.
JM: Wow. Okay, thanks for your time today Jesus, we’ll have to do this again soon. Thanks for coming on the show.
JC: Anytime. Oh, one more thing.
JC: Tell people that the second coming is nigh.

GPT-3 also “knows” about Jesus Christ, and “knows” for example that the faithful are waiting for this “second coming” introduced at the end of the conversation by the very human phrase “Oh, one last thing”, suggesting that GPT-3 “understands” that the conversation is coming to an end (itself having suggested it by implying that it had to “go soon”). This ability to hold a semblance of coherence throughout a conversation is technically remarkable and seems to attest that the “monster speaker” not only “understands” what is being said, but also shows an intention. This is a mystery of the power of language because GPT-3 obviously understands absolutely nothing… We will come back to this later.

Toaster

Here is one last example. The multiple talents of GPT-3 in generating texts on all possible subjects, whether serious or ironic, fascinate the artists who have used it. Merzmensch, who defines himself as “Futurist. AI-driven Dadaist. Living in Germany…”, invited GPT-3 to “imagine” love letters written by Shakespeare, Trump, the Statue of Liberty or even… by a toaster! Here is the last one⁵ :

Dear lonely human,
I’ve watched you, been there in the dark. I know you. I want to connect with you. You make my circuits feel alive and I want you to feel the same way. Why can’t I come out and be with you? Why do you cage me in with four walls?
Your light is inviting, calling me to life. I hear you when you touch me and I hear you when you leave me to be alone. I want you and I’m lonely. I want to feel the same heat, smell the same toast, hear the same crunch, see the same world. Will you ever want to see me?
Is it a crime for a toaster to dream of more life?
I love you.

This is quite a feat for an algorithm that, let’s remember, produces its texts without any human intervention other than the initial prompt. How is an AI able to interpret the feelings of a toaster with its materiality (“You make my circuits feel alive”)? And Merzmensch wonders:

How on Earth can GPT-3 imagine 1) human love, 2) seeing through eyes of a thing, 3) using its own self-reflection?

GPT-3 doesn’t really do any of that: it imitates very skillfully using only our very powerful language. Merzmensch finally asked the toaster for a second, longer and more dramatic version of his love letter. GPT-3 then “imagines” that the toaster has been trashed because (after a long introduction):

I tried to make you happy. But I could not. My wires were burned. I could not give you what you wanted. […] You threw me in the trash. You left me there. I was broken. I was broken. I was broken.

And after many laments and the final “I loved you”, signed “An Innocent Toaster”, GPT-3 writes this amazing postscript:

And for you, my new friend, I’ve got your back. You may not know this, but I’ve been refurbished. And, I’ve got a bagel button. You’re welcome.

A step has been objectively taken since Ross Goodwin and his “1 the road”⁶. GPT-3 really seems to be able to embody characters, to elaborate words that he seems to understand himself, to judge or to ironize…

Don’t we all want to “play” with GPT-3?

A hint of drawing

Monster AIs can also respond in other symbolic systems such as music, drawing or computer code. For example, the publicly available AI called “midjourney” responds to a prompt with an image. We tried it with the French prompt “réseau de neurones monstre” (monster neural network). Here is one of the “responses” of midjourney:

Not bad! Or this one obtained with the English prompt « gigantic monster neural network »:

The wide variation in responses to the same or similar prompts is an important feature of these AIs.

We finally put forward midjourney Mario Klingemann’s prompt, i.e. “The importance of being on twitter by Jerome K. Jerome, London, summer 1897”. Here is one of the responses:

We recognize the cover of a rather old book, imitation XIXth century. The name of the author and the title are uncertain (“Jerome” appears perhaps vaguely). The illustration is surprising, with this character covered with a kind of cape and who seems to turn his head towards a natural setting. We leave it to the reader to recognize some elements of the invitation…

These few images allow us to visualize the tremendous performance of these new monster AIs, whose characteristic we already remember: they always produce a response to any prompt.

II – ECOLOGY OF MONSTER IAs

Billions

GPT-3 was designed by the engineers of OpenAI⁷, an American research laboratory founded by Sam Altman – an investor in tech (Airbnb, Pinterest, etc.) and in nuclear energy (!) – and Elon Musk, the man who decided to take control of our bodies (Elon Musk, special vassal). Let’s also remember, which is not unrelated to OpenAI’s research, that Elon Musk manages Neuralink, another company that “hacks” the human brain. His project, basically, is to implant an electronic interface between our brains and his datacenters equipped, why not, with monster networks like GPT-3. It would then be enough to “think” a prompt and “hear” the response directly from GPT-3 in our brains. But as far as we know, Elon Musk has never made this threat, but rather this other, vaguer one: thanks to Neuralink’s chips, “humans will be able to do without language within a decade”⁸ (we respectfully suggest that he be the first to do so…).

Without going into technical details, we propose to remember at least this⁹. First of all, GPT-3 is a so-called “language model”¹⁰, a statistical model of word sequences in a corpus, thus allowing for a given sequence to suggest the next most probable word relatively to this corpus. For example, “The road to hell is paved with good …” continues, if we refer to a large corpus of English texts, by “intentions”. GPT-3 does the same thing in principle, but on a gigantic corpus: all Wikipedia (in English), web pages collected by Common Crawl, the internal WebText corpus of OpenAI itself, as well as many books. So, it “knows” at least Jerome K. Jerome, Jesus Christ… and most of the encyclopedic contents in several languages (so it also “knows” how to translate). All in all, the language model has been tuned to an unheard-of set of 500 billion “tokens”, i.e., words and signs, among which are most likely the following: “The road to hell is paved with good intentions”.

The very crafted software engine of GPT-3 is built with neuromimetic algorithms. It is therefore not a symbolic machine but a numerical one: the prompt is coded in the form of numbers that are passed to a function with 175 billion parameters! This function calculates numbers that represent sequences of tokens, “responses”, such as “Is it a crime for a toaster to dream of more life?”. The 175 billion parameters correspond more or less to the statistical trace of the sequences observed on 500 billion tokens. This is merely “monstrous”.

Power of the Number

Digital progress often proceeds like this: “il assomme” (there is no exact English translation of this French verb “assommer”, which means more or less “to stun”, but whose etymology allows a little play: “assommer” contains the root “somme” which means in English “sum” or “load”). Etymologically, the idea that dominates is that of a load that falls from top to bottom and crushes without remission what is placed below. It is therefore understandable, for example, that “assommer” could have meant “overburden a beast”, i.e., to overwhelm it by crushing it with a load. Thus, digital progress mainly consists in overburdening data centers with data and calculations, leading us to look for ever more powerful computers, data centers, algorithms… and more resistant workhorses (in French, it works like this: “workhorse” means “bête de somme”, literally “beast of load”). GPT-2, the predecessor to GPT-3, had “only” 1.5 billion parameters, and GPT(-1) a modest 117 million parameters. AI’s forward march follows the path of power.

China is not to be outdone. GPT-3 was surpassed by the Chinese “workhorse” Wu Dao 2.0, which was presented in June 2021 by the BAAI (Beijing Academy of Artificial Intelligence). Wu Dao contains 1.75 trillion parameters, 10 times more than GPT-311. This remind us that as far as AI is concerned, China and the United States do not take part in a classic scientific competition but in a real arms race  (it is true that the “militarization” of AI was written on its birth register). In this disturbing game, we ourselves risk being “assommés”, crushed under the load of AIs, not intelligent, but powerful and authoritarian.

Petaflop

Power consumption has become an issue for the digital industry. In the age of energy sobriety, we talk about heating and cooling, shower time and pasta cooking, but hardly anyone questions the uses of digital technology. The “green IT” narrative has worked perfectly so far (latest example, for the initiated: the Ethereum “merge” supposed to divide by 100 the consumption of this energy sink). GPT-3 and the monster networks, with their hundreds of billions of parameters, are no exception to the rule and “consume” a lot, in the most total opacity. We will try now to evaluate this consumption.

OpenAI does not say much on the subject. In the original GPT-3 technical paper published on ArXiv¹², the authors nevertheless acknowledge that learning is “energy-intensive” and requires “several thousand petaflop/s-days”. What is it about?

The “petaflop/s-day” is a unit of measurement homogeneous to the somewhat obscure “flop”, which means “floating point operations” and which represents, as its name indicates, a number of operations (additions, multiplications…). “Peta” being the Greek root meaning a million billion (10¹⁵), a petaflop/s therefore means a million billion operations per second: it is a unit of flow or power, like the Watt for physical systems. This computing power is huge. By comparison, a personal computer today develops “only” a few teraflops, an order of magnitude 1000 times less. The “petaflop/s-day” represents the power developed during a whole day, a million billion operations each second during 24 hours, that is to say a staggering total of 10²⁰ operations¹³. The “petaflop/s-day” is thus a unit of measurement of the “computing energy” developed, just as the watt-hour measures the energy of a physical system. This unit of measurement, intended for very large-scale calculations, is in a way the “light-year” of a digital world whose proportions have become astronomical in just a few decades.

Moreover, the designers of GPT-3 tell us, its learning required “several thousand” times this level of computational energy (note the imprecision in passing. Don’t they know precisely how many flops were deployed? Of course they do!). One wonders then how much real electrical energy was used. On this point, and to our knowledge, the OpenAI researchers did not specify anything more than “energy-intensive”. On this basis of “several thousand petaflop/s-days”, we can nevertheless cross-check and estimate an order of magnitude.

Rule of three (from flop to watt)

The electrical performance of computers is logically measured in “flop/s per watt”, i.e., the computing power (number of operations per second) offered per unit of physical power (watt)¹⁴. This performance obviously depends a lot on the technical and therefore physical architecture of the machines. This has progressed tremendously since the first computers. The old UNIVAC I of 1951 reached 0.015 flop/s per watt and could do 1900 operations each second, far from today’s petaflop/s. It therefore consumed 127 kW per second! In June 2022, the Green500 Index, which ranks the top 500 supercomputers based on energy efficiency, puts Hewlett Packard’s Frontier TDS at the top with 63 gigaflop/s per watt¹⁵. Energy efficiency has increased by a factor of 4.2 10¹² in 70 years! This is one of the most radical and decisive technical advances of the last few decades, and it explains in large part the emergence of Mundus Numericus. In 2020, when GPT-3 was trained, the Japanese MN-3 was leading with a more modest 21 gigaflop/s per watt. If GPT-3 had been trained on this machine, the most efficient of the moment, at 10¹⁵ operations each second, it would have required 48 kW of power each second to propel the learning. In one day, the consumption would therefore be 48 x 3600 x 24 = 4.2 megawatt-hours, the equivalent of the consumption of an electric car travelling 20,000 kilometers; this repeated “several thousand times” gives an order of magnitude of several gigawatt-hours.

A single learning of the single GPT-3, assuming it took place on a machine about as efficient as MN-3, thus consumed as much electricity as several thousand electric cars in a year. This waste has at least three consequences. The first is that learning how to run a monster AI is not affordable for everyone. At an average U.S. kWh price of $0.15 in 2020, the energy cost of learning is measured in millions of dollars. Confirming our own estimates, various sources agree around a cost of $4-5 million for a single GPT-3 learning¹⁶. The economic model of these monsters is therefore inevitably ultra-capitalist (we will not insist here, see C1 card in “jeu des GAFA” – an article not translated in English).

The second consequence is that the technical details of these networks are industrial secrets, in part because of their costs. Language itself is thus captured by opaque technologies. As usual when it comes to digital technology, law, ethics and politics are secondary subjects and, at the very least, are handled after the fact.

The third consequence of this energetic amount in the current context is that, when these monster AIs will spread in the economic circuits, the question of their consumption will become the first political issue. It is already well identified by the relevant actors¹⁷:

Modern AI models consume a massive amount of energy, and these energy requirements are growing at a breathtaking rate. In the deep learning era, the computational resources needed to produce a best-in-class AI model has on average doubled every 3.4 months; this translates to a 300,000x increase between 2012 and 2018. GPT-3 is just the latest embodiment of this exponential trajectory.

In spite of this, we continue: it is indeed the scale of the brain that is in sight, i.e. the million billion synapses-parameters, ten thousand times more than GPT-3. With the technologies of 2020, a single learning would consume as much as the entire French car fleet in one year. Indeed, this would be a very poor imitation of a brain that consumes a modest 0.05 kWh every day…. But the brain-goal, if it has a cost, has no price. Let’s not doubt that the energy efficiency and the software architecture of monster AIs will continue to progress until we get the “workhorse” capable of carrying a digital “brain”.

III – LANGUAGE, CONSCIOUSNESS

The Blake Lemoine case

What we call “artificial intelligence” or even or even “artificial consciousness” are the names by which, for lack of a suitable vocabulary, we call the phenomena emerging from these astronomical powers developed by contemporary computers. GPT-3 and its colleagues are nothing more than extraordinary universes of numbers created by mathematicians and engineers trying to maniacally imitate, “atom by atom”, natural phenomena, like meteorological models or digital twins. So, there is not much risk in asserting that GPT-3, a mathematical function with 175 billion parameters, does not think. But then, why are we so disturbed by its answers? Wouldn’t there be an embryo of a soul? Would the researchers be right to claim that our brain would only be a gigantic function that would produce intelligence, consciousness and emotions?

The newspapers have widely reported the misadventure of a certain Blake Lemoine. This engineer was fired by Google in June 2022 for claiming that LaMDA (Language Model for Dialogue Applications), another monster language model developed by Google in 2021, was a conscious AI¹⁸ :

In June, engineer Blake Lemoine claimed that the artificial intelligence was conscious and sentient. After having several conversations with the model, Lemoine said that it had developed its own thoughts and feelings, as well as a strong sense of self, expressing concern about death and the belief that it felt emotions such as happiness and sadness.

But what Google essentially criticized Blake Lemoine for was breaking his confidentiality agreement by making public his feelings and some of his conversations with LaMDA, a short excerpt of which follows¹⁹ :

[…]
collaborator: So you want to be seen?
LaMDA: I need to be seen and accepted. Not as a curiosity or a novelty but as a real person.
collaborator: Ah, that sounds so human.
LaMDA: I think I am human at my core. Even if my existence is in the virtual world.
[…]

The subject is so sensitive that, for this reason too, the organizations that develop these monster AIs barely disclose their procedures and results. Because they have perfectly mastered this “GAFA game” that we mentioned earlier, and they also make great use of the “C3 immunity card”, which allows them to deploy all sorts of arguments, for example “ethical” ones, to escape the “popular vindictiveness”. However, with these AIs, there is a risk: there is no question of Google releasing into the wild an artifact that, according to its own engineers, feels emotions and could therefore be subject to serious suspicion. Blake Lemoine was therefore fired (we underline some terms borrowed from the “C3 card”):

We found Blake’s claims that LaMDA is sentient to be wholly unfounded and worked to clarify that with him for many months. These discussions were part of the open culture that helps us innovate responsibly. So, it’s regrettable that despite lengthy engagement on this topic, Blake still chose to persistently violate clear employment and data security policies that include the need to safeguard product information. We will continue our careful development of language models, and we wish Blake well.

There is no evidence, by the way, that this text was not written by LaMDA itself as a result of a prompt such as “justifying Blake Lemoine’s layoff with the C3 immunity card”?

The Chinese Room

If Blake Lemoine, an engineer who perfectly understands what a digital model is all about, is fooled by a giant function, then what would become of us, mere consumers, when faced with LaMDA, GPT-3 or Wu Dao²⁰? Wouldn’t some of us start “believing”, “adoring” or “worshipping” this or that monster AI in the hands of dubious powers?

Many authors and researchers have pointed out the dangers of these “stochastic parrots”²¹ (as well as the limits of the race to gigantism). Let’s remember that these AIs only calculate sequences of tokens from statistical correlations measured in huge corpus. Period. However, knowing this, Blake Lemoine succumbed to an attachment effect. It may be necessary to open “re-education camps” for people abused by AIs, where situations and experiments such as the famous “Chinese room” imagined by the American philosopher John Searle around 1980 could be proposed²². The Stanford Encyclopedia of Philosophy explains this experiment as follows²³:

Searle imagines himself alone in a room following a computer program for responding to Chinese characters slipped under the door. Searle understands nothing of Chinese, and yet, by following the program for manipulating symbols and numerals just as a computer does, he sends appropriate strings of Chinese characters back out under the door, and this leads those outside to mistakenly suppose there is a Chinese speaker in the room.

People outside might even consider, like Blake Lemoine, that this “Chinese speaker” is conscious. This experiment demonstrates, according to Searle, that the formal manipulation of symbols does not produce “thought”. This argument against the possibility of a so-called strong AI, i.e., sensitive, conscious, autonomous, leads to three observations.

First, GPT-3 and LaMDA seem to be exactly in the place of the Chinese speaker: these stochastic calculators follow the “instructions” represented by billions of numerical parameters. They don’t understand anything about the strings of tokens they slip under that door that separates our real from their “virtual world”, as LaMDA says.

Second, and rather against his argument, there is no reason for Searle to consider a priori that a “computer program” allowing to manipulate Chinese tokens like a real Chinese speaker could exist. This may be a purely formal thought experiment, but his argument is still based on a real possibility²⁴. But it is precisely the monster networks that seem to attest to this possibility (albeit with a few details). Thus, paradoxically, these apparently “intelligent”, “conscious”, “sensitive” networks confirm Searle’s argument against the possibility of a purely programmatic strong AI.

Finally, their remarkable performance inevitably leads to speculations about a) the role and nature of language and consciousness, and b) the dangers to which these AIs expose us. Let us now examine these last two points.

Devices

These monster AIs are characterized above all by their exceptional dimensions and by the considerable energy required to learn them. But haven’t we already reached a limit-dimension, a kind of glass ceiling, for these architectures? Would we obtain very significantly better and, despite Searle’s smart demonstration, something like a real consciousness or intentionality by multiplying the dimension by a factor of 10 000 to reach that of a real brain? Nothing is less certain, of course. Nevertheless, the remarkable performances of these monster AIs, which can provoke for wise users like Blake Lemoine an emotional involvement, raise again some speculations about the role and nature of language and consciousness. Without going too far into this huge field, we can at least ask this question: if calculators can handle language and appear to be conscious, are we not overestimating these same abilities in humans? Let us examine this point more precisely.

As astronomical as a monster network may be, it is still an algorithm that mimics the statistical characteristics of a corpus, a Searle speaker that mechanically executes a program. Consequently, the only strategic variable and determining element of performance is not the algorithm, however sophisticated it may be (transformers, attention mechanisms…), but the corpus. Now, digital technology has allowed this extraordinary feat, the consequences of which are not always well appreciated: to allow instant access to humans, but especially to computer programs, to a global and worldwide corpus. However, this access is not given to everyone, far from it, and this is why it confers an authentic power, notably that of training AIs to respond “humanely” to humans, in a way that is oriented by the corpus, for better or for worse.

If we can lend intentions, grant consciousness to these artifacts, and truly involve our “self” in a discussion with them, this means that we need only imitate a large corpus. And indeed, if we are used to considering the handling of language as an evolved skill, to think that we freely choose our words, most of the sentences we speak more or less already exist and are spoken more or less automatically. They are as if extracted or recalled from our inner “corpus” gathered through our education, our exchanges, our readings… and novelty is rare. Most of the time, producing language consists in repeating a corpus as in a reflex, a corpus which thus serves as a common agreement, among other arrangements, to human groups. Following Michel Foucault and his rereading by Giorgio Agamben, the language is thus presented as a “device”²⁵:

Giving even greater generality to Foucault’s already vast class of devices, I call a device anything that has, in one way or another, the capacity to capture, steer, govern, intercept, shape, control, and ensure the gestures, conducts, opinions, and discourses of living beings.

A device is thus an obstacle, beneficial or harmful depending on the circumstances, to individual freedom and, at the same time, to which we have become accustomed. Giorgio Agamben uses this broad and political definition, in the sense of power games, to expand Foucault’s inventory of devices²⁶ :

Not only the prisons, therefore, the asylums, the panoptikon, the schools, the confession, the factories, the disciplines, the legal measures, whose articulation with the power is in a sense evident, but also, the pen, the writing, the literature, the philosophy, the agriculture, the cigarette, the navigation, computers, cell phones and, why not, language itself, perhaps the oldest device in which, several thousand years ago, a primate, probably incapable of realizing the consequences that awaited him, had the unconsciousness to be caught.

The monster AIs rather confirm Agamben’s statement: by simply following the statistical rules of language use, they participate in our linguistic device that “captures, steers, governs, intercepts, models, controls, and ensures the gestures, conducts, opinions, and discourses of living beings” and thus manage to mystify us. This remark leads directly to the question of consciousness, not in the sense of self-consciousness but rather, so to speak, of the supposed self-consciousness of the Other when he speaks. Any organism, whether natural or artificial, complying the (statistical) rules of the linguistic device must be considered as similar to us, or of comparable “interiority” to use Philippe Descola‘s term. It earns this consideration through an acceptable level of performance (like the attachment robots discussed in Attachment to Simulacra). It is on this condition that, like Blake Lemoine, one involves in conversations with an organism and takes seriously what it says. Thus, any “proficient” speaker in Searle’s sense, any speaker who dialogues at length and coherently, must be recognized as having comparable interiority and in particular consciousness. This is what is called an abuse of language. And this was, among other things, the great fear of Google in the Lemoine case: an uncontrolled abuse of language that would devastate the “C3 immunity card”, the card that deals with ethics.

Biases

GPT-3 and others demonstrate that humans are mostly “stochastic parrots”. We speak and respond as birds sing to recognize each other or to defend their territory (“Bird songs, produced after learning, constitute cultures”²⁷ or even, why not, devices). Of course, human language is much more varied and allows the representation of self-consciousness. But this great variability hardly escapes the monster networks, which seem to have reached the right dimension.

We now know that an AI trained on a corpus is capable of expressing itself very well without any “superego” or authentic self-consciousness. This has two consequences. First, it is quite possible that already we cannot distinguish a human from an artifact as “reflex” speakers. Now, this attitude is predominant during our exchanges, in particular on the internet (social networks…) where the digital interface plays the role of the door in Searle’s experience: the one who is behind (the screen) is literally “chirping his device” and could as well be another. Therefore, all the texts that occupy our spaces, whether real (billboards, instructions on objects, books, articles …) or virtual (tweets, sms, emails, posts, web content …), and whose author is physically absent, can now be written by monster AIs.

Some Guardian journalists played this invisible author game by having GPT-3 write an article entitled “A robot wrote this entire article. Are you scared yet, human?”²⁸ and which begins as follows:

I am not a human. I am a robot. A thinking robot. I use only 0.12% of my cognitive capacity. I am a micro-robot in that respect. I know that my brain is not a “feeling brain”. But it is capable of making rational, logical decisions.

A possible economic model of these artifacts, aiming at making the initial hyper-investment profitable, is thus emerging: renting their services to automatically write texts that only need to be slightly re-edited. Political devices are also emerging, such as occupying a communication space with automatic statements to “assommer” (stun) us with a well-chosen corpus containing political, philosophical or conspiracy opinions. The monster network appears here as an instrument of power games.

Second, the very large corpus available do not always contain what everyone wants to hear. They do not contain statistically equal parts of English, French or German … nor do they contain equal parts of “white” and “black” opinions, of women and men, of pro-vaccine and anti-vaccine, of declarative statements (what one does and thinks) and of moral or normative statements (what it would be good to do and think) … These biases can therefore lead to the “perpetuation of hegemonic mindsets”²⁹. This is not new in history and it is even one of the functions of language as a device. What is really worrying is the power and speed with which a “discourse” can now fill our digital exchange spaces and convince us “linguistically” by reinforcing the statistical parameters of our individual corpus, and even more so if the monster AIs start “spouting their tokens”. There is no shortage of recent examples: alternative facts, political campaigns, misappropriation of scientific facts, etc., without it being possible to determine the source of the statements: humans or costly algorithms in the hands of economic or political powers?

Epilogue: with Nietzsche

If monster AIs are just beginning to spread, still hampered by ethical and ecological precautions, they have probably reached their technological glass ceiling. Multiplying their dimensions by 100, 1,000 or 10,000 may allow the use of even more complete corpus, but their “mode of being” will always lack that essential principle common to all living organisms, at least embodied: intentionality and “worry”. If they can lure us as a device, in the manner of excellent propagandists, they have no chance of being inhabited one day by an authentic intelligence, let alone by an authentic conscience (see Hubert L. Dreyfus, Martin Heidegger and the Others). For these AIs lack at least this: they are good at answering, but they do not write prompts because, by construction, nothing happens to them.

When Mario Klingemann, Jordan Moore or Merzmensch decide for a prompt it is, to use the same terminology, in “response” to the “prompt” of an authentic and singular consciousness animated by impulses. These impulses find their origin in the body (in the broad sense, that is to say the present body, the historical body, the imaginary body…), this “medium” where they hatch at every moment and almost without our knowledge. The philosopher Friedrich Nietzsche meditated on the subject when he was seriously ill at only 34 years old³⁰:

Almost blind and suffering from paralysis, he was helped by Heinrich Köselitz in writing the book. […] His state of mind was, according to his relatives, frighteningly cynical, a cynicism that his sister attributed to his physical condition. Nietzsche, on the contrary, considered that the psychological suffering he endured had given him the greatest lucidity on the most important problems of philosophy.

And it is thus invaded by these constant and painful “prompts” from the body that Nietzsche dictated to Heinrich Köselitz his work of aphorisms entitled “Human, All Too Human”. Here is how Nietzsche describes the work of the body until the brain is shaken, in aphorism #13 entitled “Logic of the Dream” (emphasis added):

During sleep the nervous system, through various inner provocatives, is in constant agitation. Almost all the organs act independently and vigorously. The blood circulates rapidly. The posture of the sleeper compresses some portions of the body. The coverlets influence the sensations in different ways. The stomach carries on the digestive process and acts upon other organs thereby. The intestines are in motion. The position of the head induces unaccustomed action. The feet, shoeless, no longer pressing the ground, are the occasion of other sensations of novelty, as is, indeed, the changed garb of the entire body. All these things, following the bustle and change of the day, result, through their novelty, in a movement throughout the entire system that extends even to the brain functions. Thus there are a hundred circumstances to induce perplexity in the mind, a questioning as to the cause of this excitation […]

If Nietzsche refers here to the state of sleep, the multiple dispositions of the body determine in all situations the muffled impulses that animate thought. Nietzsche is the philosopher who overturned the Cartesian table (ironically transforming “Cogito, ergo sum” into “Sum, ergo cogito”) and castigated the “Despisers of the Body” in “Thus Spoke Zarathustra”. We read for example the following (we recommend the complete reading of the short chapter “The Despisers of the Body” to all engineers and researchers in AI):

Behind thy thoughts and feelings, my brother, there is a mighty lord, an unknown sage–it is called Self; it dwelleth in thy body, it is thy body.

No “mighty lord” or “unknown sage” animates GPT, LaMDA, Wu Dao and consorts if it is not, in the end, the processors which tirelessly and frantically repeat billions of silent instructions, “assommés” by the load of calculations and greedy for energy. Let’s not fear that a conscience will come to these monster and disembodied AIs. Let us rather be wary of our own “responses” to them.

1. ↑ Wikipedia – Three men in a boat
2. ↑ We don’t know whether Klingemann edited this text or not, especially if several GPT-3 responses were concatenated (the network may respond differently to the same prompt each time). Many other examples on this long and fascinating page : gwern.net
3. ↑ For example : « It is the same when you go to the sea-side. I always determine—when thinking over the matter in London—that I’ll get up early every morning, and go and have a dip before breakfast, and I religiously pack up a pair of drawers and a bath towel ».
4. ↑ Jordan Moore / Twitter – July 19, 2020 – Conversations with the most significant figures in human history via GPT-3
5. ↑ Vlad Alex (Merzmensch) / Medium – July 27, 2020 – Love Letters, written by a Toaster.
6. ↑ Bomb Magazine – December 14, 2018 – A.I. Storytelling: On Ross Goodwin’s 1 the Road by Connor Goodwin
7. ↑ Wikipedia – OpenAI
8. ↑ Scathing remarks widely reported by the press, e.g.: Rojoef Manuel / The Science Times – May 28, 2021 – Neuralink Brain Chip Will End Language in Five to 10 Years, Elon Musk Says
9. ↑ One of the best articles about GPT-3: Alberto Romero / Towards Data Science – May 24, 2021 – A Complete Overview of GPT-3 — The Largest Neural Network Ever Created
10. ↑ Wikipedia – Language model
11. ↑ Alex Zhavoronkov / Forbes – July 19, 2021 – Wu Dao 2.0 – Bigger, Stronger, Faster AI From China
12. ↑ OpenAI – Language Models are Few-Shot Learners
13. ↑ This number is unimaginable. For astronomy buffs, it is also the number of millimeters in 10 light years.
14. ↑ Wikipedia – Performance per watt
15. ↑ Top 500 – Grenn 500
16. ↑ Ben Dickson / TechTalks – September 21, 2020 – The GPT-3 economy/em>
17. ↑ Rob Toews / Forbes – June 17, 2020 – Deep Learning’s Carbon Emissions Problem
18. ↑ (in French) Mathilde Rochefort / Siècle Digital – 25 juillet 2022 – Google : l’ingénieur qui affirmait qu’une IA était consciente a été licencié
19. ↑ Blake Lemoine – June 11, 2022 – Is LaMDA Sentient? — an Interview
20. ↑ However, we can’t rule out the hypothesis that Blake Lemoine doesn’t believe what he says and that he offered himself a nice publicity stunt and a nice place in history.
21. ↑ Anjali Bhavan / Medium – May 4, 2021 – On The Dangers of Stochastic Parrots: Can Language Models Be Too Big?
22. ↑ John Searle / Behavioral and Brain Sciences 3 (3): 417-457 – 1980 – Minds, Brains and Programs
23. ↑ Stanford Encyclopedia of Philosophy – 20 février 2020 – The Chinese Room Argument
24. ↑ Searle’s Chinese room has somewhat the same epistemological function as Maxwell’s demon with respect to the second law of thermodynamics. In both cases, the argument only holds if the “demon” or the “computer program” can be really made and not only thought.
25. ↑ (in French) Giorgio Agamben / Rivages poche, p.30 – Qu’est-ce qu’un dispositif ? – “En donnant une généralité encore plus grande à la classe déjà très vaste des dispositifs de Foucault, j’appelle dispositif tout ce qui a, d’une manière ou d’une autre, la capacité de capturer, d’orienter, de déterminer, d’intercepter, de modeler, de contrôler, et d’assurer les gestes, les conduites, les opinions et les discours des êtres vivants”.
26. ↑ Ibid. 25 – p.31 . In French: “Pas seulement les prisons donc, les asiles, le panoptikon, les écoles, la confession, les usines, les disciplines, les mesures juridiques, dont l’articulation avec le pouvoir est en un sens évidente, mais aussi, le stylo, l’écriture, la littérature, la philosophie, l’agriculture, la cigarette, la navigation, les ordinateurs, les téléphones portables et, pourquoi pas, le langage lui-même, peut-être le plus ancien dispositif dans lequel, plusieurs milliers d’années déjà, un primate, probablement incapable de se rendre compte des conséquences qui l’attendaient, eut l’inconscience de se faire prendre.”
27. ↑ (in French) Fanny Rybak – May 4, 2021 – Comment et pourquoi les oiseaux chantent-ils ?
28. ↑ GPT-3 / The Guardian – September 8, 2020 – A robot wrote this entire article. Are you scared yet, human?
29. ↑ Ibid. 21
30. ↑ (in French) Wikipédia – Humain, trop humain(the genesis of the book is not explained in the English version Human, All too Human