These four assumptions are almost always made in science fiction and popular commentary on AI (or alien life). They are often made by academic futurists and professional scientists (who should know better).
- There is some essential property of a human being that gives rise to all the other important properties of a human being. In science fiction, the essential property is called things like “intelligence”, “consciousness”, “sentience”, “self-awareness”.
- Related assumptions:
- The Paragon of Animals. There is a distinct and important boundary between things that have the essence (i.e. human beings) and all other things in the universe.
- The Promethean Moment Trope. A computer (or alien) that crosses the boundary and acquires the property will suddenly have new abilities and properties.
- AI Doomsday. AI will become markedly more dangerous at a particular point in the future.
- “A Machine Will Never do X” / The AI Effect. For many human abilities and traits, there is a clear difference between “real”/”genuine”/”true” forms and “simulated” forms. (e.g. “genuine creativity”, “actually thinking”)
- The Stepford Scenario. A perfect robot simulation of a human being is not a “real” person and can’t be a target of “genuine” empathy and love — the robot simulation lacks an essential property.
- “Science Will Never Explain X”. The words we use to describe the essence, “consciousness”, “sentience” and so on, are somehow “hard to define”. This is sometimes followed by the assertion that there is something inscrutable about the essential property that will make it forever inaccessible to empirical study.
- Top candidates for the essence and why they don’t work.
- Human simulation and Human Sympathy.
- Acting like a person (i.e., the Turing Test).
- Cared about by a person. (in sci-fi: Her, TheStepford Wives, Ex Machina, A.I., Frankenstein)
- Consciousness, Sentience, Self-Awareness. (In sci-fi: Ex Machina) Related traits that require subjective consciousness: sense of humor, emotions, “lived experience”, “values”, “free will”, “thinking”, “mind”, intentionality.
- Intelligence. General intelligence (“AGI”), “Human-level” intelligence, superintelligence.
- The Essence as Metaphysical. Some new age thinkers and futurists visualize “consciousness” as an invisible, energetic fluid that suffuses humans and other “higher beings”. Here, the word “consciousness” is being used a secular synonym for “spirit” or “soul”.
- Call it “Personhood”. This is my preferred term. In its normal usage, it entails free will, suffering, responsibility and rights.
- Human simulation and Human Sympathy.
The Chain of Being Assumption
- All things in the universe have a property that can be used as a linear measure. It can be described metaphorically as altitude — i.e., there are “higher” and “lower” things. It generally respects this order: inorganic objects < microorganisms, slime molds, etc. < plants < non-mammal animals < non-human mammals < humans
- Related assumptions:
- Evolution = Progress. It’s common to describe things that are “higher” in the chain of being as “more evolved”.
- Advanced Aliens, Superintellligent machines, Angels and God There are “higher” beings.
The Myth of Steady Progress
- Technological capability and human knowledge increase at a (roughly) consistent rate.
The Myth of Limitless Progress
- Technology will eventually overcome any limit. Any form of magic is possible with sufficiently advanced technology.
- Related assumptions:
- Superintelligent machines will have god-like abilities.
CHARLES GILLINGHAM 