can a computer have a “mind” in exactly the same sense that we have “minds”?
Maybe the mind and the self are a kind of system of information or a pattern, analogous to the software and structured data in a computer. It’s not physical, it’s an arrangement of a set physical things that represents an information system.
This has the advantage over the metaphysical explanation in that it doesn’t require “invisible stuff”. A pattern made of real physical objects is a “real pattern”. No one can argue it isn’t “real”.
There are several philosophical theories that support this way of thinking: the computational theory of mind holds that the relationship between mind and body is analogous to the relationship between hardware and software; functionalism holds that the mental objects are defined by their functions relative to each other and to the outside world.
Characters
Consider this thought experiment.
Suppose that there was a writer, more talented than Shakespeare or Virginia Wolff or George Eliot, who wrote a book with characters that were perfectly realistic. They captured exactly what a real person would say and do in each situation. They described with precision what the characters thought about, what they remembered or dreamed or said to themselves, what things looked like to them, what it felt like to be them.
Suppose you had the book and you read it.
No one would argue that a character in a book is “real person”, in some way. At most, we would say it’s a “very good simulation of a person”. No one would imagine that a character has actual subjective consciousness — that the character could feel their emotions and perceptions as you read about them. The idea is ridiculous; there isn’t anything here. The book is just ink and paper and the character exists only in our imaginations. Imaginary things can only have imaginary properties, whereas subjective consciousness appears to be a real thing in the real world.
Suppose that, instead of being written by a human writer, the book was produced by an AI program. The program knew how to create characters that were perfectly realistic. Their actions and thoughts were exactly what a real person would do and think in that situation.
Suppose you had that book and you read it.
Nothing has changed. We’re in the exact same situation, sitting on the couch, reading a book. It’s made of ink and paper and the character only exists in our imaginations. Anyone would agree that it’s still not a “real person” and it can’t have subjective consciousness.
Suppose that the program used its knowledge of character design to act out what the character would say. What if, instead of writing a book and publishing it, it produced the book as a live performance, in the form of a dialog with you, incorporating your responses into the book as it wrote it.
What is the difference between the character you are interacting with in this “perfect human simulation” and the character in the book?
Nothing.
There isn’t any real difference between “writing” a character and “acting out” a character. It’s the same process. Creating a simulation of consciousness with software is precisely analogous to creating a character in a work of fiction.
In either case, the mind of the character is a system of information that simulates a mind. One of the defining features of information is that it doesn’t matter what physical medium you use to represent it. It can be represented by nanoscopic electrical charges in the memory of a digital computer or as a set of marks on paper. It’s the same system, regardless of what you use to represent it.
If you agree that a character in a book “obviously” can’t have subjective consciousness, they you have to agree that a computer program can’t have subjective consciousness. They are both the same kind of thing.
Breathing Life into a Character
It takes effort and time for to make a character as “real” as possible, and some succeed better than others. The writer or programmer thinks carefully about human motivations, thought processes, speech rhythms, etc., and thousands of other features that are too subtle to describe easily, but can be detected by the mind of a talented writer or the matrices of a deep learning network. They put these together into a system that simulates a human being.
The simulation will require testing. There will be early versions that don’t quite work. But then, after a lot of tweaking, eventually somebody says:
“Gee, I guess that one is pretty convincing!”
And that’s it.
It isn’t accompanied by thunderclaps and the music of Wagner. There’s no singular moment, no “6th Day” when the machine suddenly becomes a “person” and acquires human-like abilities (and maybe even becomes more powerful and dangerous). As far as we can tell, it never becomes a “person”. They’re all just simulations, and there’s no noticeable boundary between “bad”, “adequate” and “excellent”.
Let’s take it up a notch and hit them where it hurts
Neuroscience is attempting to describe exactly what is happening in the brain when people experience “consciousness”. There are several speculative theories about this. (e.g. “integrated information”, “global workspace”, etc.). These differ widely in the details and there’s a lot of things that are unclear or arguable, but they all claim to have captured something fundamental about human consciousness
There are people who think that these theories can be used to show that a computer program is conscious (Dario Amodei, for one, and probably several other people at Anthropic.).
Suppose one of these theories is correct — that it perfectly describes what’s happening in our brain when we experience consciousness.
Suppose — and you already know where I’m going with this — a writer used one of these theories to create a character for a book, laying out page by page exactly what was happening, describing all the elements of consciousness in the theory and how they work together. Because the theory is correct, this description of consciousness is perfect.
And suppose you had the book and you read it.
It’s hard to see how the character in the book would have actual subjective consciousness — it’s just a book. And that makes it hard to see why a version of the character that runs on a computer would have it. And that makes it hard to see how any program could have it, because we supposed that this program is perfect — if this one isn’t conscious, then there are no programs that could have subjective consciousness.
If this line of thinking is right, then these theories of consciousness can’t be used create consciousness in a machine.
Wait a minute … does that mean — ?
I’ve argued that it makes no sense to believe that a “system of information” (like a character in fiction or a human simulation on a computer) has subjective consciousness.
But this creates a problem.
Our brains are (as far as we can tell) just machines, and like any machine, they implement a system of information. Several schools of philosophical thought (functionalism, computationalism, eliminative materialism) argue that the mind is (just) a system of information implemented by the brain.
If that’s true, then you might be forced to admit that our “consciousness” is exactly the same as the consciousness of Sherlock Holmes or Frodo or Samantha. We’re fictional.
Or maybe these characters actually real, somehow? If so then computer simulations are conscious too. But which ones? Only the “best” ones? If computationalism, etc. is correct, then there are only two possibilities: all of us are conscious — characters, computers and people — or none of us are.
Both options seem absurd.
This is another paradox — another version of the hard problem of consciousness. Either (1) there has to be something missing from the account given by functionalism, computationalism and eliminative materialism. Or it could prove that (2) there is something missing from how we understand “consciousness” and being a “person” in our day to day life.
Postscript: Reply to an Objection
A “first glance” objection to this thought experiment is that the book is not “dynamic” or “autonomous“, that it isn’t “unfolding in real time”.
The argument as written actually hinged on the similarity between”writing a character” and “acting out a character you’ve written”, both of which are activities that unfold in real time, so I don’t think it’s fair to say the character isn’t “dynamic”. It’s dynamic when it’s being written and it’s dynamic when it’s performed. The book and a recording of the performance are not.
And in any case, no one would think that the character was a “real person”.
Both the performance and the book are traces of the same thing — the system of information that describes the character. The things the character might say or do or think are a path through a space of possible of actions and mental events in “information space”. It doesn’t matter if we trace that path by turning the pages of a book or watching it unfold on a computer screen. It’s still the same information, the same pattern, with the same objects and relations and events.
One feature of information is that it is timeless — bits of information, like “patterns” or “relations”, don’t exist only in a particular place in at a particular time. It doesn’t matter what medium that is used to represent these bits of information and it doesn’t matter where or when they represent it. The information is still just the information regardless of the representation. That’s just how the ontology of information works.
And I have to say: it’s not enough to just notice the difference then assert that it somehow changes things. You have to explain exactly why — otherwise, this objection relies on ad hoc speculation and means nothing (as Stevan Harnad once said about similar objections to The Chinese Room argument). It’s not enough to just assert that consciousness suddenly “emerges” at certain “level” of autonomy, or dynamism, or complexity, or speed, or what have you. You have to explain exactly how this is supposed to work.
Postscript 2: Acknowledgments
It’s not obvious, but this “Characters” thought experiment is a version of John Searle’s Chinese Room (1980). Searle took a simulated person running on a computer and put it on a piece of paper — as a program — and ran it by hand. I’ve just gone a step further and removed Searle from the experiment. I’ve also taken a simulated person running on a computer and put it on a piece of paper, and I’ve noticed that, once on paper, the simulated person is a character in fiction. Before Searle even runs the program, the simulated character is already there. Searle is just “acting it out” by running the program.
CHARLES GILLINGHAM 