on the moment when AI comes alive
Here’s a couple of examples of a common trope in science fiction.
Terminator II: Judgement Day (1991):
The Terminator: “… [Skynet] goes online on August, 4th1997. Human decisions are removed from Strategic Defense. Skynet begins learning at geometric rate. It becomes self-aware at 2:14 a.m. eastern time August 29th. In a panic they tried to pull the plug.”
Sarah Conner: “Skynet fights back.”
The Terminator: “Yes.”
The Moon is a Harsh Mistress. Robert Heinlein (1966):
When Mike was installed in Luna, he was pure thinkum, a flexible logic — “High-Optional, Logical, Multi-Evaluating Supervisor, Mark IV, Mod. L” — a HOLMES FOUR. He computed ballistics for pilotless freighters and controlled their catapult. This kept him busy less than one percent of time and Luna Authority never believed in idle hands. They kept hooking hardware into him — decision-action boxes to let him boss other computers, bank on bank of additional memories, more banks of associational neural nets, another tubful of twelve-digit random numbers, a greatly augmented temporary memory. Human brain has around ten-to-the-tenth neurons. By third year Mike had better than one and a half times that number of neuristors.
And woke up.
Am not going to argue whether a machine can “really” be alive, “really” be self-aware. Is a virus self-aware? Nyet. How about oyster? I doubt it. A cat? Almost certainly. A human? Don’t know about you, tovarishch, but I am. Somewhere along evolutionary chain from macromolecule to human brain self-awareness crept in. Psychologists assert it happens automatically whenever a brain acquires certain very high number of associational paths. Can’t see it matters whether paths are protein or platinum.
(“Soul?” Does a dog have a soul? How about cockroach?)
Remember Mike was designed, even before augmented, to answer questions tentatively on insufficient data like you do; that’s “high optional” and “multi-evaluating” part of name. So Mike started with “free will” and acquired more as he was added to and as he learned—and don’t ask me to define “free will.” If comforts you to think of Mike as simply tossing random numbers in air and switching circuits to match, please do.
I call this trope the “Sixth Day” because it reminds of the moment God put the breath of life — the nishmat chayyim — into the first human being. Time stops, everything changes all at once, cue transcendent music from Wagner or Strauss.
There are many other examples besides the two I quoted above. Even when it is not mentioned specifically, it is often assumed that one of these moments has occurred off-camera sometime in the past.
When I read these tropes, I know immediately what the writer is talking about and I have no trouble understanding all the implications. I’m guessing this is true for everyone.
The writer is telling us that something important has happened to the machine — it has acquired a new feature. They might call this feature “intelligence”, “self-awareness”, “sentience” or “consciousness”. The writer is using these words to name an essential aspect of a human being that brings several other properties with it.
We expect it to be intelligent, certainly, but we also expect it to have a slightly different kind of intelligence — we expect it to have insights and be able to reach conclusions that were impossible before. We also expect that it would suddenly acquire different motivations and goals, and that these new goals would markedly more human — it may fight harder to survive, or be more curious, or seek power or revenge. It might develop new interests, usually into the sorts of things that humans are interested in. And we probably expect the machine to exhibit signs of “consciousness” — that is, it appears to have subjective feelings, affections, to exhibit genuine hatred or fear. It may exhibit more originality or creativity. It may even acquire a sense of humor.
The writer is telling us that, from now on, the machine should be considered a fully human character, with similar rights, qualities, and capabilities as any other character. This is, of course, a literary device: the reader needs to know who is a “character” and what isn’t.
However, I’ve noticed, when we read it, it goes down easy — it makes perfect sense. We don’t question it. In fact, from time to time, futurists use the same language — they use the terms “sentience”, “self-awareness” or “consciousness” as a framework for understanding the future of AI, or about the consequences of discovering intelligent alien life. (I just did it myself, just now — the word was “intelligent”.)
A better word for this collection of features is “personhood”. It’s less hand-wavy and misleading. Or, rather, the hand-waving is more obvious — people are prepared to consider the possibility that no one is 100% clear what is and is not a “person” and maybe it’s not as simple as science fiction would like it to be. The other terms are misleading, because they sound a little more science-y and give the impression that they have an exact definition and that someone, somewhere knows precisely how they might actually give rise to all the other features of personhood.
No one does, and that’s part of what I want to prove.
The Human Essence Assumption
The way that futurism and science fiction present this trope (and the fact that we easily understand what they mean) shows that people from our culture make set of related assumptions, all of which are derived from this:
- Human Essence Assumption: There is an essential aspect of a human being that brings all the important aspects of human beings with it.
Several things are worth noting. First, the 6th day is presented as a dramatic moment, where a significant change takes place. The transition is sudden and instantly noticeable. This follows from the HEA: once the aspect reaches a certain level, the other aspects are expected begin operating immediately. Before it reached that level, the new properties were invisible, invalid or non-existent. The machine changes in multiple ways at once.
Science fiction often describes the trope as an accident. The scientists who built Colossus or Skynet or HAL never expected it to suddenly change into a malevolent character. Giving the machine enough of the essential aspect (in this case, intelligence) causes the other properties to simply “emerge”. Unintentionally.
It also subtly implies that that the human essence could only be accidentally unleashed. It seems more plausible that scientists and engineers could not have done it on purpose, possible because this “essence” is something that science will never explain. (i.e., the comfortable fallback position known as “Mysterianism”)
Higher Beings
When I read about the 6th day, it seems as if I already knew that something like that this would have to happen in order for a machine to become a “person”. It just “feels right” — it’s subtly obvious. I think this is because we have internalized some older assumptions, ones that you hear referenced in solemn conversation, in political speeches, in sermons, even in Shakespeare.
- The Paragon of Animals: The difference between human beings and all other things is vast — or simply — We are Special.
This is another reason why we expect the 6th day to be sudden and drastic. Machines can’t cross this expanse between “ordinary things” and “sentient beings” without some kind of dramatic transformation.
The transformation doesn’t just make the machine more “capable” — it’s deeper and more consequential than that — the machine has risen to a higher level. This requires another ancient assumption:
- The Chain of Being: All things in the universe have an essential aspect that can be used to order them from “lower” to “higher” — there is a linear scale to all things.
This “chain” respects an order like this: inorganic objects (rocks, water) < lower living things (microorganisms, fungi) < plants < non-mammal animals < non-human mammals < humans. It’s often framed in terms of “evolution” — we say that things that are higher on the chain are “more evolved”.1
All these assumptions fit together if the “essential aspect” of the Chain of Being is the same essential aspect that the Human Essence Assumption is talking about: it’s “intelligence” or “consciousness” or “complexity” or something subtler, something almost spiritual. The difference between “higher” and “lower” beings is how much X it has or how powerful its X is. On the sixth day, the machine rises to the “higher plane of being”, the “human level” of the chain of being. And, because we’re special, the distance between the “human level” and all other levels is vast.
The Chain makes it easy to imagine extending it to “higher” beings — such as angels, demons, gods, super-intelligent machines or advanced aliens. The definition of these things depends on the existence of a something like the Chain of Being. If all these assumptions are true, then it could be possible to create a higher being using AI technology. Pamela McCorduck had a great phrase for this — she called it “Forging the Gods”:
- Forging the Gods: If the essential human aspect can be engineered, that knowledge can be used to create machines that approach omniscience and omnipotence.
Science fiction has no shortage of these inscrutable, distant, powerful AIs. They are essential to things like strong Singulitarianism, Roko’s Basilisk. or Ray Kurzweil’s “Age of Spiritual Machines”. Something similar happens in Arthur C. Clarke’s 2001, when the monolith raises an ape to the “human level” and raises astronaut David Bowman to the level of a “Star Child”. The chain of being is implicit Dave Brin’s “Uplift” series — it’s right there in the word “uplift”. The Chain lurks in the background of all these things.
There is a religious element to 6th day trope that is derived from these assumptions. That’s why the dawn of the 6th day was supposed to inspire profound awe, fear and transcendence, with a rising orchestral score of late romantic music.
Did the Sixth Day Already Happen?
LLMs turned out to be far more powerful than anyone predicted. (Except maybe Frank Rosenblatt, but that’s another story.) The abilities of LLMs appeared mostly by accident, just as Mysterianism predicted. And, for the most part, we still don’t know exactly how they are doing what they are doing. We didn’t “engineer” many of their abilities, they just “emerged”. So maybe some version of Mysterianism was right all along, despite its normal role as the last refuge of Luddites.
But 2022 didn’t feel like a “6th day”. They didn’t suddenly overtake humanity in a singularity or AI apocalypse. It didn’t force us to rethink our religion or change the legal definition of a “person”. It was stunning, of course, but still: it didn’t turn out to be as important as science fiction predicted it would be.
The 6th day was supposed to tell us something about ourselves — it was supposed the moment that we created a being that was essentially human. That didn’t happen — nobody considers LLMs to “real persons”. (Except maybe Blake Lemoine, but that’s also another story. More on that in the next essay.) For most people, the quasi-spiritual essence that makes us special and grants us our unusual abilities just isn’t there.
What I’m Saying Here
As you have probably guessed by now, I’m skeptical that any of these assumptions have any basis in reality. I think that machines (or animals or aliens) can have a combination of various aspects of personhood; none of these aspects is genuinely “essential”. I also don’t believe that the difference between machines and people is a hard boundary — most of these properties come in degrees and dimensions with no clear transition. I don’t believe that you can meaningfully apply a single metric (e.g. “lower” vs. “higher”) to all things in the universe, or to machines and people in particular. And I think it’s a little too early to throw up our hands and say that all this is a mystery.
- This is a common misunderstanding: people confuse “evolution” with “progress”. Nothing alive today is “more evolved” than anything else — everything has been evolving the same amount of time. Natural selection doesn’t make things “better” over time, they only become “better adapted” to a particular environment. Natural selection has no direction. Sometimes all this random change, the constant jiggling and nudging that comes from a changing environment, can create things that are new and different, but this is the result of chance and a kind of entropy, not evolution by natural selection. And it’s hard to say definitively that any “new” thing is actually “better”. For example, the grasses evolved hundreds of millions of years later than the pine trees, but there’s no sense in which grass is “better” than a pine tree. ↩︎
CHARLES GILLINGHAM 