The Stepford Scenario

could you love a machine the same way you love your children? 

Consider this thought experiment, which is a variation on the 1972 movie “Stepford Wives”. (This version works better for parents.)

Of course I did something heinous. If this was a movie, we expect you to fight to death to get your children back. That’s the only way this movie can go. At least, it’s the only way that would make any sense to the audience.

This thought experiment (if it worked) should convince you that you believe some version of this assumption:

• Human Essence and Human Sympathy: There is something about a person that is the unique target of empathy and love.

You can call it whatever you like: a “self” or “personhood” or a unique “consciousness”. Whatever it is, we believe it exists — or, at the very least, we understand what it’s supposed to be and how it’s supposed to work — otherwise, we couldn’t understand the thought experiment above.

In fact, anyone who doesn’t believe that other people have a unique “self” or “consciousness” has a serious psychiatric problem — they are profoundly autistic or sociopathic or have some other debilitating mental aphasia. If you sincerely don’t believe in this, then it will be difficult for you to live a normal life.

I would argue that we cannot imagine that this aspect does not exist; the idea is built into us. If your brain is functioning properly, you will find yourself unable to believe that everyone else is a mindless robot who feels nothing and has no “self”. We are behaving in the only way that makes any sense to us. 

We are behaving as if “souls” exist.

The paradox is that, although we behave as if it exists, although we assume it must exist, we have no consistent way of describing what exactly it is. The experiment explicitly eliminates several possibilities. It can’t be explained by just (1) The way a person looks. (2) Anything they might say or do. (3) Anything you might think you know about them. (4) The way you currently feel about them. Here are some other possibilities.

---

Invisible Stuff

If consciousness is a metaphysical thing, the there are actually two crimes. First, I stole the thing that you love, their spirit or “self”. Second, I have given you creatures that have no “consciousness” at all, that appear human but, just like Frankenstein’s monster or classic zombies, they are soulless creatures with no “self”. This is horrifying, at least, even if it’s not currently illegal.

This is the way I imagine the writer would expect us to understand this story.

This way of understanding “consciousness” is called “substance dualism” by philosophers. None of them (at least, none of them alive today) think it is correct. The basic problem is this: it seems very unlikely that this invisible stuff exists. If it did, physicists and neuroscientists would have found some evidence of it by now.

But, of course, feel free to believe in a metaphysical explanation if you like. As I said above, you have to believe in something in order to live a normal life. Just be aware, for now, that science will probably never find it, so there will always be a few people who will say (after a few drinks, late at night, as a way of being smart) that it isn’t “real”.

But of course, when they wake up, they will immediately go back to assuming it is real; they won’t start running around yelling that all the other people are just robots.

---

Information

Another theory is that self is a system of information, a particular kind of information processing that it is carried out by brains, similar to a computer program running on a computer.

Let’s talk a bit about how information works. If I write your address on two pieces of paper, it’s the same address. The pieces of paper are both physical representations of the same information. Similarly, two copies of a book contain the same book. You can only copy the representation of the information, you can’t copy the information itself. It’s the same information, no matter how many physical things are used to represent it.

In the Stepford Scenario, my robot copies are perfect — they contain exactly the same information as your children, they have the same pattern, they implement the same function. And we know that if I make two copies of a bit of information, then they contain the same information. So it follows that, according to computational theory of mind or functionalism, the robot copies are not just simulations of your children, they are your children.

If that’s true, then in the scenario above I have done nothing wrong — in fact, I have done nothing.

Now, maybe this is the right explanation. I haven’t disproved it. I’ve only shown that I am uncomfortable with it. No one would accept that the robot copies of their children are “just fine”. We wouldn’t say “that’s all right, you can keep my children. These are good enough for me.” This is not how we see the situation. This is not how we would act or how we would feel.

The thought experiment above shows that, in your day-to-day life, you do not accept either the computational theory of mind or functionalism as complete descriptions of what’s going on with other people. We believe there is more to our children than “a system of information that produces a particular behavior”, regardless of how accurate the simulation is. The “ghost inside the machine”, the child that we love, can’t be composed of just information.

---

Ordinary Stuff

What if the “self” we are attached to is just their bodies — every last hair on their heads, their organs, their brains, the whole machine balanced on two feet, including everything it has done and everywhere it has been. This is not as naive as it sounds. After all, this is how we talk in our ordinary, day-to-day life. We talk as if the child’s “self” and the child’s body are the same thing. No philosophy required.

Consider that, according to mechanism, your children’s bodies are just machines, their brains are just machines, and everything about them is just mechanical. It follows that your children were already robots. I just gave you new robots, indistinguishable from the old ones. The fact that they are robots isn’t the problem. The problem is they’re the wrong robots.

We can, of course, love a thing. We can love historical artifacts, prize possessions, particular places. We get angry if they are destroyed. We are offended if we find out they have been replaced with fakes. If this is the right way to look at it, then the fact that you care about these children is no different than the way your child might feel about a particular stuffed toy. Your anger is the same anger your child would feel if we replaced a favorite stuffy with a new one from the store.

A replacement isn’t enough. It’s not just the thing that matters but the provenance of the thing. Where it’s been, its origin, its history, what’s happened to it. It’s also what you feel about the things that have happened to it. Your children’s bodies have a provenance and you know what it is. There are things you know about them, things in your head like: these are your children, no one else’s, you saw them born, you’ve been with them their whole life, you imagine a future for them, and so on.

What makes a thing important is more than just the thing, it’s also what you know about the thing and what you know about it has to be true. To have the right provenance, it has to be the right thing, the actual thing.

In the scenario, you think the robots are your children and what’s in your head tells you to love the robots the same way you loved your children. I’ve tricked you into being wrong about where you target your love, and this might be the most horrible aspect of the scenario.

This account is compelling, but I can’t help feeling that something has been left out. I’m still not entirely comfortable thinking of another human being as a just a “thing”, like any other thing. I can’t get myself to sincerely believe that we love our children the same way we love a favorite guitar.

This doesn’t prove that the “Ordinary Stuff” explanation is wrong (just as I couldn’t disprove the “Information System” explanation). In both cases, I’ve only proved that I am uncomfortable with it.

---

Subjectivity and Sentience

Your children can actually feel and experience what’s happening, whereas the robots’ gears and motors and circuits don’t feel a thing. This, I’m sure I’ll agree, is a difference between them actually matters: the robots don’t have subjective consciousness and children do.

Before we go on, remember that subjective consciousness is paradoxical and we have no idea how it actually works. All we know for sure is that we think there is a difference between experiencing something for yourself and just hearing about it or knowing about it.

The next three sections here will try to dig into this, even though this is a bit outside the scope of what I’m trying to show. It goes without saying, the next few pages are going to be rough sailing. Skipping them doesn’t affect the whole argument.

---

Sentience Matters

We feel sympathy for things that can suffer and empathy for things that can feel love. We “feel for them” — our feelings mirror the feelings we think they are having. We worry that our children will get hurt. This is different than when we worry if our car will break.

Consider how we assign rights to animals based on our feelings; we care for a kitten but we step on a bug. A similar thing happens in the pro-choice / pro-life debate. The similarity between us and the thing suggests that they are at least partly built the same way as we are, which suggests that they have similar experiences, thus it feels the same way to be them.

This ability to “feel” is called “sentience”. We recognize sentience in other people (or animals or things), so we feel sympathy and empathy, and this motivates us to prevent their suffering, to reciprocate their kindness and so on. This logic is built into us — it’s part of how our brains work. Every normal brain will follow this path: similarity -> sentience -> sympathy -> morality.

In many contexts, people trust their capacity for human sympathy to make these choices — people find it more convincing than any scientific or philosophical theory. Sympathy is, in fact, the only reliable way we have to measure “personhood”. It’s the ground truth of morality and law, the first and final thing that actually matters. But, as near as I can tell, we don’t really understand why we find it so convincing and fundamental.

---

Sentience By Itself Doesn’t Exactly Explain the Scenario

The scenario breaks the first link: the robots are similar to our children, but in this case it doesn’t imply they have sentience. They’re not “built the same way” as us so it’s unlikely they “feel the same way” as we do.

The children feel nothing. They’re zombies. There is something deeply disturbing about creatures like this, but what, exactly? How are these creatures harming you?

You believe the robots have feelings when they don’t. This is awkward for you, of course — you’re showering love on them, they couldn’t care less. I’ve put you in a situation where you love something that doesn’t love you back. This isn’t a crime, but it’s definitely wrong. You will probably go to a lot of trouble to prevent the robots from suffering, and all of this is a waste of your time — they can’t feel a thing. Wasting your time is rude but not criminal. 

Meanwhile, your actual children, the ones I stole, are not suffering at all. As I said, they are in a better situation that they were with you. So as far as their welfare goes, you don’t have to worry about it. Definitely not a crime.

So you could make the case that the robots aren’t doing anything wrong. They’re not harming you and they’re just being what they are. The crime isn’t that I gave you robots or how horrifying the robots are. The crime is I took your children.

I don’t think that the children’s “sentience” is the issue here — sentience is just an ability to have an experience. We don’t love our children’s experience, or the fact that they have some experience, or their ability to have an experience — we love the thing that has the experience. The “subject” of “subjectivity”.

So we’re back to the thing.

---

Mental Experience

Another aspect of subjective consciousness is our inner life — the things that we notice happening to us when we are thinking or imagining or planning or remembering. We have a train of thought, that we can picture things in our imagination, that we can form words in our heads without saying them out loud, talking to ourselves in a way that only we can hear. We experience these things subjectively, first hand. We know they exist and we know what they’re like.

People say “I have thought in my head” or they talk about the “content of their consciousness”— as if they’re in a kind of container that holds all these things that are going on when we’re “thinking”. This is another reification — a metaphor for all the things that are happening. So here’s another “thing”-that-isn’t-actually-a-thing, that we have no way to talk about except through metaphor.

Really it’s a “process“, a series of “events” (because it takes place over time) involving “thoughts”, some of which can see or hear (that is they are “experiences”). So suppose that’s something I stole: the children’s “stream of consciousness” — the ongoing process of experience and thought that was happening in their heads.

The content of their consciousness, what’s on their mind, is part of what you love about them. Their stream of consciousness also has a provenance, just like their bodies. The things they’ve thought about, the history of their stream of consciousness matters.

What’s to stop me from copying the children’s “stream of consciousness” into the robots? What if I gave them simulated thoughts, simulated inner speech, a simulated train of thought, starting from the exact point when I took the children? If subjective consciousness is an illusion or metaphor, and all there is only information processing, then the robots would have the same minds as your children. It follows that you owe the robots your protection and love.

There’s no way you’re going to do that.

We have to believe that the children’s mental experience isn’t enough — that there’s more to the mind than the contents of the mind. We believe that there is also someone in there subjectively experiencing the contents of the mind: a “self”.

Again — the thing.

---

Best Answer I’ve Got

It may be I’ve belabored the obvious here, but here’s what I’ve got: the things that matter are (1) their bodies (2) the things they experience in their heads, their stream of consciousness. What we don’t have: the subject, the thing that experiences the stream of consciousness.

The fallback here, which I introduced in Ghosts and Other Invisible Stuff, is that the child’s “self” is a kind of metaphor. Perhaps even a metaphor that is built into our brains a priori in a way that is impossible to avoid.

This is all I’ve got: you need their actual bodies and stream of consciousness, with a built-in metaphor that convinces us that there is something there. So deeply built-in that we actually can’t imagine it any other way.