Excerpts from "Why Robots Go Astray, or the Cognitive Foundations of the Frankenstein Complex"—part II of Strange Concepts and the Stories They Make Possible (Johns Hopkins University Press, 2008), pp. 50-85.
1. What Is the Frankenstein Complex?
Isaac Asimov defines the Frankenstein complex as "mankind's . . . gut fears that any artificial man they created would turn upon its creator." Although the particular character who offers up this definition in one of Asimov's short stories proclaims himself free of this complex and not afraid "that robots may replace human beings," at the end of the day he is proven rather spectacularly wrong. Indeed, many of Asimov's tales feature some variation of the Frankenstein complex being vindicated. His robots routinely disobey their makers, surprise them by doing things that they were emphatically not created to do, and even plot to destroy their masters. In fact, the whole genre of science fiction as we know it today is unimaginable without the recurrent motif of robots, cyborgs, animatrons, androids, virtual agents, and other artificial creatures going astray in a wide variety of shocking ways.
Let us widen the circle yet further. It seems that the Frankenstein complex was around even before the rebellion of the first official "Robot," of Karel Capek's play Rossum's Universal Robots (R. U. R.) (1921), and before the murderous rampage of the "Creature" from Mary Shelley's Frankenstein (1816). Milton's Paradise Lost (1667) comes to mind, with its humans and angels defying their "Glorious Maker." And then we are back to the Book of Genesis, in which it is God who has the Frankenstein complex. Upon learning that Adam and Eve have disobeyed him and eaten from the forbidden tree, God reasons that the "man has now become like one of us, knowing good and evil," and that he "must not be allowed to reach out his hand and take also from the tree of life and eat, and live forever." God is just as fearful of competition and displacement by the creatures that He himself has created as are Asimov's brilliant engineers at the fictional corporation "U.S. Robots." The deviant fictional robots are typically recycled, and, likewise, God banishes Adam and Eve and makes sure that they will ultimately turn to the dust of which Adam was originally made.
The recurrence of the "robot-gone-astray" motif suggests that we find perennially interesting stories about creatures who disobey their makers. We tell them to score a specific political point; to issue a vivid ecological warning; or to put a popular actress on the screen in the role of a sexy cyborg. Whatever our initial motivation and rich array of subsequent interpretations, one thing remains the same: We tell such stories and apparently cannot get enough of them.
Of course, there are plenty of explanations of why we find them so gripping. Such explanations typically invite us to see Frankenstein, R.U.R., and I, Robot, Bladerunner, He, She, and It, and Idoru as responding both to fundamental psychological needs—such as the "obsession" with our "own image" or the fascination with "our own capacity for destruction"—and to specific cultural preoccupations of their time. Thus Harold B. Segel considers Capek's R. U. R. a characteristically Expressionist reaction to the technological expansion in the wake of World War I. In this view, Capek's dystopian drama "embraces the common Expressionist vision of humans so overwhelmed by machine culture that they become subservient to and are ultimately swept away by the very robots they have created to lighten their own burden." Similarly, J. P. Telotte suggests that the sci-fi movies of the seventies, such as THX 1138 (1971), The Clones (1973), The Terminal Man (1974), Westworld (1973), and Futureworld (1976), hit a cultural nerve by contrasting the "mechanically crafted body powdered by artificial intelligence" with the "real, the genuine, the human." Telotte believes that such a contrast was particularly relevant in the era "that saw the creation of the first functional synthetic genes," the "rapid spread of reproductive technologies like the Xerox machine," and the "introduction of the videocassette recorder"—technological developments that blurred the "distinction between the originals and their copies." And Asimov himself thinks that with the widespread use of the microchip in the eighties, a "brand new variety of technophobia" uncoiled itself in evil-computer stories, in which the increasingly "compact, versatile, complex, capable," and "intelligent" machines threatened to "replace not just a person but all humanity."
These are good explanations, but for me they are not good enough anymore. Here is why. First, I am put off by the ease with which we come up with them. Indeed, it seems that any historical epoch or cultural niche can supply a whole slew of reasons why at this particular moment people would be particularly interested in the stories of artificial creatures who disobey their masters. You tell me that people loved the rebellious robot stories in the America of the 1970s because such stories both expressed and assuaged their anxiety about the blurred "distinction between the originals and their copies." I will happily counter that people also happened to love such stories in Soviet Russia in the 1970s (e.g., Prickliuchenia Electronika, 1979) because . . . let me think—ah!—because they allowed them to express their unhappiness about being treated by the State as easily replaceable parts of a huge mechanism. I will then quote you a series of everyday Soviet proverbs to the effect that nobody is unique ("nesamenimych u nas net!") and that we all are but nails, and bolts, and screws, easily replicated and easily replaced. Such wise sayings must have scarred several generations of people growing up under Soviet rule, and as such they must have intensified their hankering after fictional stories in which replicable robots turn out to be unique, irreplaceable, special, human.
Don't you find this theory compelling? I find it pat and irresponsible. It took me three seconds to come up with it, and you will never be able to either prove it or refute it. What kind of "explanation" is it?
My other quarrel with such explanations is that they leave out completely the human brain-mind which evolved to deal with natural kinds and artifacts but not with the artifacts that look and act like natural kinds. Such hybrids did not exist in the Pleistocene, when our capacities for categorization acquired their present shape. I wonder, then, what evolved cognitive adaptations we call upon to conceptualize these strange creatures, or, to put it another, better, way, what features of such hybrids may reflect the workings of our evolved cognitive categorization processes?
Keeping these two objections in mind, I would thus like to advance the three following hypotheses. First, the fictional stories of artificially made creatures tease in particularly felicitous ways our evolved cognitive adaptations for categorization, and as such they remain perennially interesting to human beings. I use this "panspecies" language on purpose to challenge you to think of any human culture that does not have fictional stories about living creatures who are not born but "made," artifact-like.
Second, certain plotlines in such stories, for example, the "rebellious robot" plotline, could be seen as expressions of our intuitive attempts to resolve the state of cognitive ambiguity that has been forced upon us by the challenge of processing the representations of such hybrid creatures. This is an important point, and in what follows, I will spend quite a bit of time on it. Here, however, is something that I want you to do in advance of that longer explanation. The next time you watch a science fiction movie or read a science fiction story, I want you to pay attention to that movie/story's use of functionalist language. I predict that science fiction narratives that develop the theme of a "rebellious creature" will foreground the functionalist rhetoric, that is the rhetoric emphasizing that a given protagonist—a robot, a cyborg, an android—was put together artifact-like, that is, it was made, or custheory of mind-tailored, or designed with a specific function in mind. I suggest that by cultivating such functionalist rhetoric to describe an entity that we at the same time cannot help but perceive as a living being, such stories tacitly render the motif of its subsequent rebellion particularly cognitively plausible or cognitively satisfying. This is not to say that our fictional man-made living beings always have to rebel: for every stray android of Bladerunner there is a perfectly satisfied with its lot 3-CPO from Star Wars. But if they do rebel, we have been prepared for it by struggling tacitly with their counterintuitive ontological status since the beginning of the story.
My third hypothesis concerns the current practices of historicizing our fascination with rebellious artifacts. I believe that the concept of the human brain becomes meaningless once you attempt to separate it from the culture in which that brain develops (and by the same token the concept of human culture becomes meaningless once you try to extract the human brain from it). As Frank Keil puts it, once we "acknowledge real structure in the causal and relational regularities of living beings, a structure that is largely distinct from other sorts of natural and artificial kinds, [we can] further see how cultural practices and beliefs interact with those structural patterns, and how each of those in turn interact with cognitive biases and constraints." So fictional stories of rebellious robots indeed touch—sometimes intentionally, sometimes not—certain cultural nerves (e.g., an anxiety about encroaching technology, an unhappiness of being thought of as replaceable and replicable), but the very reason that they can touch these nerves is that they have first "grabbed" us cognitively, that is, they have created a zone of cognitive uncertainty and possibility.
To illustrate this latter point, let us first borrow an argument from part I [of this book]. I suggested in my discussion of Speak, Memory that had its narrator claimed outright that as an adult he finally understood that the essential feature of the Mademoiselle was her helpless kindness, it might have provoked an ambivalent reaction in readers. We might have felt that this is not quite it and that Mademoiselle was more complex than this, especially after everything we learned about her through the eyes of Nabokov's younger self. This is to say that even though we intuitively ascribe essences to a given individual (for doing so allows us to make inferences on the basis of minimal data), we remain particularly open to suggestion that we have missed grasping her essential feature once more. This may sound paradoxical but it makes sense once you remind yourself that essences are bound to remain elusive because our quest for them reflects the quirks of our cognitive architecture and not the objective presence of essences in people.
This instability implicit in our essentializing may explain, at least in part, why a fictional story that features a person whose "self" is defined exclusively through his social position (e.g., "he is a lackey—that's all there is to him") implicitly prepares us for later complications: a revelation, for example, that the "lackey" actually has ambivalent feelings that transcend that initial characterization. Now, it does not mean that this later complication of the plot has to happen, merely that we have been primed for it (in case the writer does decide to develop it) by a strong statement of what the character "essentially" is.
Let me restate this cognitive claim in historical terms: a fictional story that features a person whose "self" is defined exclusively through his social position lends itself to all kinds of subversive cultural readings grounded in specific ideological concerns of the moment. As soon as the story claims that social class defines the person, we are primed to look for culturally specific contexts that would subvert that claim.
On this level of analysis, it becomes very difficult to sustain the separation between the "cultural" and the "cognitive." To quote Keil again, "Conceptual domains are embedded in cultures that have strong influences on their natures, boundaries, and levels of differentiation." We have to speak about the ways in which the cognitive reinforces the cultural and the cultural reinforces the cognitive. This inquiry into the mutual cognitive-cultural reinforcement is quite different from explanations that focus on one (arbitrarily chosen) historical circumstance and ignore cognition altogether. (I gave you an example of one such explanation when I came up with a facile theory that my peers and I loved the movie about a rebellious boy-robot, Prikliuchenia Electronika, because even at our tender age we already felt the crushing power of the Soviet state.)
But before we approach specific stories of rebellious robots from the cognitive perspective, I need to elaborate some points of evolutionary research introduced in part I [of this book]. Let us turn once more to the work of cognitive psychologists and anthropologists.
2. On Zygoons, Thricklers, and Kerpas
The best thing about figuring out a set of rules is that it allows us to make a better sense of the cases that defy those rules. If we accept that our cognitive-evolutionary heritage prods us to think of living kinds in terms of their essences and of artifacts in terms of their functions, we have a new way of approaching numerous cultural representations that violate the boundaries between living kinds and artifacts and thus challenge our primary ontological categories. In the already-discussed The Essential Child (2003), Susan A. Gelman looks briefly at such boundary-violating entities, from crying statues and works of art to children's toys and cartoons. Two other recent books, Pascal Boyer's Religion Explained: The Evolutionary Origins of Religious Thought (2001) and Scott Atran's In Gods We Trust: The Evolutionary Landscape of Religion (2002), focus on the role played by such conceptual hybrids—for example, artifacts whose special magical powers imply sentience—in world's religions. In what follows, I build on Gelman's, Boyer's, and Atran's studies to explore the categorical violations that abound in our fictional narratives. To do so, however, I first need to spell out the implicit inferences that structure those violations and make them so eminently tellable: fascinating, memorable, and always open to new interpretations.
To clarify the concept of inference, consider the two following sentences offered by Boyer: 1) "Zygoons are the only predators of hyenas," and 2) "Thricklers are expensive, but cabinetmakers need them to work wood." If I ask you what will happen to a zygoon if I cut it in two, you will tell me . . . —now take a second to consider your answer before you read the end of this sentence and then compare your answer to mine—that a zygoon will probably die. If, however, I ask you what will happen if I cut a thrickler in two, you may speculate that it will be damaged or, on the contrary, will double its value because a thrifty carpenter can now use up only one half of a thrickler to make a cabinet and save the other half for the next job. Whatever you say about the halved thrickler, you will not assert that it will die. That fate will be reserved for a zygoon. I may ask you other questions—for example, whether zygoons can be made of rubber, or whether thricklers propagate by laying their eggs in the warm sand—and I won't even bother to write out here what I think your answers will be because I am that certain that I can guess them correctly, even though this is the first time that you and I have heard about zygoons and thricklers (and it's likely to be the last, too).
Now think of kerpa. Let's say I tell you that to make a certain dish, you have to take one pound of boiled spinach, two teaspoons of butter, a pinch of salt, and a spoon and a half of kerpa. If I then ask you whether kerpas tend to smile sadly when you ignore their polite inquiries, or whether they can be folded down at the press of a button, you will say that they don't and they can't (unless, that is, we are in the world of Lewis Carroll, a subject of Part III). On the other hand, if I ask you whether you can take some kerpa on a plane in a plastic jar, you may respond with a tentative yes, a more definite answer hinging on finding out how perishable or how legal a substance kerpa is.
Are you impressed by your ability to converse with confidence about these previously unencountered entities: zygoons, thricklers, and kerpas? How do you explain this ability? To answer this question, Boyer reminds us that
Minds that acquire knowledge are not empty containers into which experience and teaching pour predigested information. A mind needs and generally has some way of organizing information to make sense of what is observed and learned. This allows the mind to go beyond the information given, or in the jargon, to produce inferences on the basis of information given.
So when we hear that zygoons are the only predators of hyenas, we immediately—though by no means consciously—place this new entity known as zygoon into the ANIMAL section of our mental encyclopedia, ANIMAL being a subcategory of a larger ontological category of living beings. Our essentializing mechanism then kicks in, enabling us to infer that "if you cut a zygoon in two it will probably die" (for it is in the nature of most animals to die when they are cut in two); that "zygoons cannot be made, they are born of other zygoons"; that "zygoons need to feed in order to survive"; and so forth.
Similarly, when we learn that thricklers are expensive, but cabinetmakers need them to work wood, we assume that thricklers are TOOLS, a subcategory of artifacts. We think of artifacts as "manmade, inanimate, and defined by their function." Moreover, we know that other, familiar items in this category, such as telephones, "cannot grow or eat or sleep," or lay eggs in the hot sand, "nor can screwdrivers or motorbikes." We will thus assume that "thricklers cannot grow or eat or sleep," or lay eggs in the hot sand.
Finally, when we are told that a spoon and a half of kerpa is required for a recipe, we classify kerpa as SUBSTANCE. Substances can be weighed and carried around—as water, sand, and rocks—but they certainly cannot be folded down at the touch of a button (that quality is reserved for artifacts) and they cannot smile sadly when you ignore their polite inquiries. Only people can do that.
3. Theory of Mind
This latter point is so important and will come up so frequently in this book that it is worth our while to interrupt our present discussion to elaborate it. When we hear that a certain entity is capable of a mental state, such as feeling disappointed and trying to cover it with a sad smile (a smile that might also be implicitly signaling that feeling of disappointment), we assume that we are dealing with a sentient being, quite likely a human being. For—and here we have yet another instance of essentialist thinking—it is in the nature of human beings to exhibit such ambivalent states of mind. (I am not saying, by the way, that other animals cannot feel ambivalent, in a broad sense of the word, but rather that no other animal is capable of consciously performing its ambivalence for the sake of the spectator.)
The mental condition of wishing both to convey and to cover one's disappointment is just one of the myriad mental stances available to us that we can recognize in ourselves and in others. Cognitive scientists have a special term, or actually several interchangeable terms, to describe the cognitive adaptation that allows us to consider a broad variety of mental states—thoughts, beliefs, desires, intentions—as underlying our own and other people's behavior. Atran and Boyer sometimes refer to it as folkpsychology; other scholars have called it theory of mind or mind-reading ability.
. . .
Cognitive evolutionary psychologists working with theory of mind think that this adaptation must have developed during the "massive neurocognitive evolution" which took place during the Pleistocene (1.8 million to 10,000 years ago). The emergence of a Theory of Mind "module" was evolution's answer to the "staggeringly complex" challenge faced by our ancestors, who needed to make sense of the behavior of other people in their group, which could include up to 200 individuals. Baron-Cohen points out that "attributing mental states to a complex system (such as a human being) is by far the easiest way of understanding it," that is, of "coming up with an explanation of the complex system's behavior and predicting what it will do next." In other words, mind reading is both predicated on the intensely social nature of our species and makes this intense social nature possible. (Lest this argument strikes you as circular, think of our legs: their shape is both predicated upon the evolution of our species' locomotion and makes our present locomotion possible.)
We engage in mind-reading when we ascribe to a person a certain mental state on the basis of her observable action (e.g., we see her reaching for a glass of water and assume that she is thirsty); when we interpret our own feelings based on our proprioceptive awareness (e.g., our heart skips a beat when a certain person enters the room and we realize that we might have been attracted to him or her all along); when we intuit a complex state of mind based on a limited verbal description (e.g., a friend tells us that she feels sad and happy at the same time, and we believe that we know what she means); when we compose an essay, a lecture, a movie, a song, a novel, or an instruction for an electrical appliance and try to imagine how this or that segment of our target audience will respond to it; when we negotiate a multi-layered social situation (e.g., a friend tells us in front of his boss that he would love to work on the new project but we have our own reasons to believe that he is lying and try to turn the conversation so that the boss, who, we think, may suspect that he is lying, would not make him work on that project and yet would not think that he didn't really want to, and so forth).
Attributing states of mind is the default way by which we construct and navigate our social environment. This does not mean that our actual interpretations of other people's mental states are always correct—far from it! For example, the person who reached for the glass of water might not have been thirsty at all, but rather might have wanted us to think that she was thirsty, so that she could later excuse herself and go out of the room, presumably to get more water, but really to make the phone call that she didn't want us to know of. Still interpreting someone's mental state incorrectly is very different from not being able to conceive that there is a mental state behind the observable behavior. . . .
We read minds constantly whether we are aware or it or not. This is a crucial point, which can be easily obscured by our imperfect terminology. The words theory in theory of mind and reading in mind-reading are potentially misleading because they imply that we go around thoughtfully communing with ourselves along the lines of, "aha, I see that she is smiling, so I can infer that she must be pleased about something, or wants somebody to think that she is pleased" or, "She is reaching for a glass of water. That means she might be thirsty, or she may want me to think that she is thirsty." Obviously, we do not talk to ourselves or others like that, except in certain specific situations, when for some reason we need to make more explicit our implicit perceptions, for example, "Well, I assumed that she was thirsty because she was glancing at that empty pitcher all the time, which is why I got up to fill it up again. Who knew that she was looking at it because she was thinking of hurling it at me?!"
In fact, it might be difficult for us to appreciate at this point just how much mind-reading takes place on the level inaccessible to our consciousness. For it seems that while our perceptual systems "eagerly" register the information about people's bodies and their facial expressions, they do not necessarily make all of that information available to us for our conscious interpretation. Think of the intriguing functioning of the so-called "mirror neurons." Studies of imitation in monkeys and humans have discovered a "neural mirror system that demonstrates an internal correlation between the representations of perceptual and motor functionalities." What this means is that "an action is understood when its observation causes the motor system of the observer to ēresonate.'" So when you observe someone else grasping a cup, the "same population of neurons that control the execution of grasping movements becomes active in [your own] motor areas." At least on some level, your brain does not seem to distinguish between you doing something and a person that you observe doing it.
The system of mirror neurons, which is responsible for activating the parts of your brain that would have been activated had you performed the action yourself, may "underlie cognitive functions that are as wide-ranging as language understanding and theory of mind." After all, theory of mind does not simply kick in fully formed by the magical age of four: it develops gradually. Thus it is possible that newborn infants' ability to imitate facial expressions of their caregivers—an ability apparently underwritten by mirror neurons—constitutes an early stage of the maturation of theory of mind.
In other words, our neural circuits are powerfully attuned to the presence, behavior, and emotional display of other members of our species. This attunement begins early (since some form of it is already present in newborn infants) and takes numerous nuanced forms as we grow into our environment. We are intensely aware of the body language and facial expressions of other people, even if the full extent and significance of such awareness escape us.
As a complement to research on mirror neurons, consider, too, the work on "interactional synchrony" by William Condon and his colleagues, who filmed a variety of communicative interactions and then analyzed frame by frame the body motions and speech sounds of participants. As William Benzon reports in Beethoven's Anvil: Music in Mind and Culture (2001), Condon discovered not only what he calls a "self synchrony," that is, "the relationship between a person's speech patterns and their body movements: head, shoulders, arm and hand gestures, and so on," but also an "interactional synchrony," that is, the "relationship between the listener's body and the speaker's voice."
Whereas the existence of self synchrony is not particularly surprising ("after all, the same nervous system is doing both the speaking and the gesturing, and the cortical structures for speech and manipulation are close to one another"), the emergent synchrony between "the listener's gestures [and] the speaker's vocal patterns" is a rather striking phenomenon. It turns out that the "listener's body movements lag behind the speech patterns by 42 milliseconds or less (roughly one frame of film at 24 frames per second)," and that "infants exhibit near-adult competence at interactional synchrony within 20 minutes of birth." All this shows once more that our minds and bodies are attuned to the minds and bodies of people around us to a much higher degree than we are consciously aware of.
. . .
As I sit in the library and work on my introduction to a volume on eighteenth-century British philanthropy, struggling with words and thus intensely focused on my own thought processes, a strange girl passes by. Something in me registers the direction of her walk ("She must want to go to the second floor"), her relaxed posture ("She is not in a hurry"), an umbrella under her arm ("She must have thought it would be raining today"), and the cut of her shirt ("She must have wanted to show off those well-toned arms"). And this is only what I am consciously aware of—God knows what else in her posture and facial expression my neural circuits are picking up on even though I do not care about this girl, most likely will never see her again, and won't recognize her if I do see her. We read minds nonstop, however biased, unintentionally wrong, intentionally muted, and just plain useless our attributions of states of mind to ourselves and to others might be.
You see now that when I refer to Ttheory of Mind and mind reading in this book, I touch just the tip of an iceberg. That is, I speak about the aspect of mind-reading that we can register and articulate—if "kerpa" smiles sadly, there must be a certain mental state behind that facial expression, which in turn means (and here our essentialist biases kick in promptly) that an entity known as a "kerpa" must be a sentient being. Just how we intuit the presence of a state of mind behind those distended facial muscles and what complex neural circuitry underlies our awareness of the kerpa's body language is a large issue, a research-in-progress for years to come. What is important for the purpose of my argument is that attributing mental states to other people—as one expression of our profound neural-social awareness of them—is not something that we have conscious control of and can turn on and off as our fancy strikes us.
4. Theory of Mind and Categorization: Preliminary Implications
Some preliminary implications of the research on Theory of Mind for our study of categorization become clear right away as we review the differences in our cognitive processing of living kinds (including persons, animals, and plants), artifacts, and substances. Let us say that we take a certain entity X to be a living being, a person. This categorization opens up possibilities for "more specific processing over the folkpsychological domain." That is, we can now engage in mind-reading and scrutinize X's behavior as indicative of certain mental states. We may decide, for example, that a strange smile playing on her face conveys a feeling other than straightforward satisfaction; or we may interpret her body language, when she throws herself on the sofa the minute she enters the house, as both indicative of her exhaustion after a long walk and of her intention to show us just how profoundly tired and thus incapable of doing any further chores she is at this point.
If we categorize a given entity as an animal (e.g., a cat), we can do some limited processing over the folk-psychological domain. We can infer, for example, that when the cat hisses at her owner and tries to bite and scratch him, she is angry at him for leaving her alone for several days. The majority of our inferences, however, will belong to the folk-biological domain, that is, we will think of this specific cat in terms of what is "natural" to her species. We may keep in mind, for example, that cats need a certain kind of nourishment for survival; that they prey upon some animals and feel endangered by others; that they have certain reproductive and recreational needs; and then adjust our behavior toward this specific cat accordingly.
If we categorize a given entity as a plant (e.g., cabbage), we will do further processing exclusively over the folk-biological domain, for cabbages, too, have features "natural" for their species. For example, if we want to cultivate a patch of cabbages under our window, we will keep in mind the conditions under which cabbages, as a species, tend to thrive and adjust our watering and fertilizing strategies accordingly.
If we categorize a given object as an artifact (e.g., a bicycle), it enables us to engage in further processing over two conceptual domains. The first domain is that of folk-psychology, but with a very limited application focusing on the intended function of the object. We assume that the bicycle was made to fulfill a certain function: that is, it was its maker's intention that it would serve as a means of transportation. The second domain is that of folk mechanics: we proceed to figure out that the bicycle cannot stand on its own unless propped up against something; that it is separate from the ground on which it lies; that it weighs more than a telephone but less than a telephone pole to which it is currently chained, and so forth. In contrast to our reasoning about persons, if we want to explain to somebody why our bicycle is lying on the ground, we will not ordinarily say that it must have lain down because it was exhausted after a long ride and wanted to show us just how exhausted it was; we may observe instead that it must have fallen down because it was not propped properly against the wall or that the object to which it was chained weighed less than the bicycle itself.
Finally, if we categorize a certain entity as a substance (e.g., quartz), we open up possibilities for further processing exclusively over the domain of folk mechanics, for example, we figure out what happens to quartz if we place it in water (will it slowly dissolve? will it sink?). As a point of contrast to our conceptualization of artifacts, thinking of quartz as found in the state of nature does not automatically activate any aspects of our folk-psychological domain, for quartz does not have a function and as such does not convey any information about the intention of somebody who "made" it. If it does, that is, if we start using this particular piece of quartz as a paperweight, we reclassify it as an artifact, thus adding to our folkmechanical thinking about quartz some elements of folkpsychological thinking about its function.
How far back into childhood can we trace this processing of perceptual data by different domains? Here is a study featuring 5-month-old infants that highlights the early age at which the differentiation between living beings (here, specifically humans) and artifacts manifests itself. Recent research by Valerie Kuhlmeier, Paul Bloom, and Karen Wynn suggests that in 5-month-olds, this differentiation may be so strong that it fosters certain erroneous perceptions about the material nature of people. Kuhlmeier and her colleagues modified the well-known earlier experiments that show that infants have expectations of continuous motion when applied to objects. As evolutionary psychologist Jesse M. Bering puts it, "like any material substance, human bodies cannot go from A ¬ C without first passing along the trajectory of B. For inanimate objects, infants are surprised (i.e., look longer) when the object disappears from behind one barrier and then seems to reemerge from behind another nonadjacent barrier. In the case of a human who violates the law of continuous motion, however, 5-month-olds are not surprised."
It appears then that "infants are mistaken about the physical constraints that apply to humans," that is, they "do not readily view people as material objects." The existence of the "human/inanimate distinction, and the differential application of principles to each, may [thus] help infants to define these areas of knowledge early in development." Moreover, "the appreciation that these construals overlap—that in certain regards, people are just objects—may be a developmental accomplishment."
Still, even after that "developmental accomplishment" has occurred and we have indeed come to understand that "in certain regards, people are just objects," this mature new understanding goes only that deep. For even then we remain fascinated by cultural representations that pointedly portray people as objects. Such representations remain interesting for us precisely because they activate inferences, from different conceptual domains, that cannot be reconciled with each other.
5. Concepts That Resist Categorization
Observe the self-enforcing circularity of our inferential processes. Imagine having just begun reading a fictional story and learning that its protagonist, whom we know only as "Andrew," forgot certain numbers. Even more specifically, we learn that "it had been a long time," but "if he had wanted to remember, he could not forget," but "he had not wanted to remember." Upon hearing this, we infer that Andrew is a person, for it is only in the nature of people to be capable of such complex mental states as wanting to forget something. Categorizing Andrew as a person immediately opens up the possibility that he has plenty of other, so far undiscussed, and most of them never to be discussed, thoughts, desires, and attitudes, for, again, it is in the nature of people—even if they are fictional characters—to have them. For example, we are thus prepared to hear that Andrew used to feel particularly close to his maternal grandmother; that he hates SUVs; that he suspects that his second wife thinks that he is somewhat emotionally unavailable; that he has a good sense of humor; and so forth. This kind of information would conform to and reinforce our earlier assumption that Andrew is a human being, and that, in its turn, would make more information of this kind conceptually welcome.
And so it will go on and on in an agreeable feedback loop—as it does in countless works of fiction—unless we happen to read a work of science fiction, in which case we may suddenly learn that the number that Andrew has forgotten is his own serial number. And since it is certainly not in the nature of people to have serial numbers—that particular feature being reserved for artifacts—our hitherto smoothly operating feedback loop comes to a screeching halt.
Let us embroil ourselves in the same conceptual conundrum starting from a different point. Imagine having just begun reading a fictional story and learning that a certain entity, known to us as "NDR," had been "manufactured" a long time ago. So far so good: the word "manufactured" immediately leads us to classify this entity with artifacts and assume that NDR is manmade, inanimate, and has a specific function. Whatever that function is, we are prepared to hear about it, for it will agree with what we have already inferred about NDR. We are riding our feedback loop and are ready to be reinforced in our perception of NDR as an artifact, which would in turn prepare us for learning about other artifactual qualities of NDR.
But something unexpected takes place. We are told that NDR had a serial number—which is still perfectly fine and acceptable—and then, that he forgot that number, in fact, "had not wanted to remember" it. This puts an insuperable block in the path of our hitherto smooth inferential process. No matter how much extra information we may be given now that confirms our inferences about our NDR as an artifact—we could be told, for example, that it was "smoothly designed and functional"—the bit about his wanting to forget something sticks out as a sore thumb because it activates a whole new series of inferences that we associate not with artifacts but with living beings, and, most immediately, with human beings.
It does not matter, in other words, what our starting point is—we may begin by assuming, based on the existing textual evidence, that a certain entity is a human being, only to discover to our dismay that this human being has a serial number; or we may begin by assuming that a certain entity is an artifact, only to be confronted soon thereafter with this artifact's capability of wanting to forget things—in either case, we arrive at the juncture at which we have to readjust our feedback loop quite drastically.
How do we do it? Can we do it at all? Not really, according to Atran and Boyer, who argue that events and entities which violate our intuitive ontological expectations are never fully assimilated by any one ontological category. As such, they retain our interest, and stay in our memory, and remain perennially open to new interpretations (as do supernatural agents and magical artifacts in religions across the world). As Atran observes,
The assignment to one of the primary ontological domains fails because further processing in accordance with intuitively innate expectations about folkmechanics, folkbiology, or folkpsychology is blocked. For example, the Arab talisman, European black magic crystal ball, and Maya sastun are naturally inert substances or artifacts with supernatural animate and sentient powers. As such, they violate intuitive expectations about folkbiology and folkpsychology, and cannot be assigned to PERSON or ANIMAL, not can they remain simply SUBSTANCE or ARTIFACT. Talkative, vengeful, or pensive animals cannot be assigned to either PERSON or ANIMAL; omniscient but bodiless gods cannot be assigned to PERSON.
By violating innate ontological commitments—for example, endowing spirits with movement and feelings but no body—processing can never be brought to factual closure, and indeterminately many interpretations can be generated to deal with indefinitely many newly arising situations. Notice that bringing processing to factual closure does not require actual verification, but only the reasonable possibility of such verification. Someone who is heard but not seen is visible in principle, whereas an invisible being is not.
Boyer proposes the term counterontological to describe the entities that resist ontological closures. His reason for introducing this neologism is that the more familiar term counterintuitive can be easily read as "strange, funny, exceptional, or extraordinary," whereas the entities in question are not even necessarily surprising. For example, if my culture has a concept of a man who did not die after he was crucified, I "cannot really register puzzlement or astonishment every single time" that man's fate is mentioned. "It becomes part of [my] familiar world" that there are persons around who do not die when killed. But this concept is still counterontological because it includes "information contradicting some information provided by ontological categories," i.e., that living beings, and people in particular, die when they are killed and do not come back.
By failing to be assimilated by the category that it has initially activated—in this particular case, a living being, a person—a counterontological entity remains a promising source of new interpretations, or, to make the same point somewhat differently, a fruitful source of new stories. Moreover, to return to the title of my book, these new stories will be profoundly structured by the ontological violation in question.
This point is crucial, so the next section will consider it in some detail. First, however, we need to address a potential misunderstanding that may arise when we juxtapose the notion of the counterontological with the assumption that ours is an essentializing species in a world without essences.
Counterontology implies ontology. That is, the notion of violation implies that there is a certain rule that can be violated. So when we think of the fictional robot as having complex mental states, we must also begin to essentialize that robot because we habitually essentialize beings who can have mental states. And in essentializing this robot, we go further than we usually do when we essentialize some artifacts in our everyday life (e.g., when we feel that something about their previous owner has "rubbed off" on them). But—and herein lies the potential for misunderstanding—how can one violate any rule by essentializing anything (in this case, the robot) if there are really no essences in the world? In other words, how can we call an entity counterontological if the particular ontology that it presumably violates is false (or nonexistent) to begin with?
You can see how, put this way, this issue may become a problem that will dog us until the end of this study. It needs to be resolved, and I propose resolving it by establishing an affinity between what Boyer calls ontologies and what Gelman calls heuristics. Remember again that Boyer deals not with entities out there existing independently from our perception, but with cognitive ontologies—that is, the ways of processing information about the world grounded in the particularities of our cognitive architecture. Similarly, Gelman observes that the belief in essentialism "can be considered an unarticulated heuristic rather than a detailed, well-worked-out theory." But underarticulated, or even plain wrong, essentialism still gets us through the day—not always in the best shape, but what else is there for us to fall back onto?
This is to say that to the extent to which essentialism is the only game in town, it can be considered a cognitive ontology. And so from now on, when I talk about this particular violation of ontological beliefs, I mean the violation of our underarticulated, hazy, plain wrong, often harmful, but nevertheless cognitively enduring and to that degree ontological beliefs.
6. . . . and the Stories They Make Possible
Each culture has infinite forking subsets of possibilities—we can call them situation-specific cultural scenarios—associated with the event of death. For example, if a healthy man in his thirties, who was alive yesterday, is found dead today, we would expect a different behavior on the part of his family and his community based on the circumstances of his death. If he was a soldier and was killed in the line of duty, it is likely that his family will grieve for him. That grief may be accompanied by anger, if his family members think that the death was preventable (as in the case of friendly fire) or that the cause of the war was unworthy; by pride, if they think that his brave service to a worthy cause has benefited the whole community; or by relief, if when he was alive and among them, he treated them cruelly. If the deceased was a civilian, and there are reasons to suspect foul play, the grief may be accompanied by a publicly expressed desire for revenge, which should eventually be carried out by designated communal institutions. If the person died as a result of political persecution, say, in a concentration camp sponsored by the ruthless party in power, the grief may be accompanied by fear for other members of the family, whereas the desire for revenge may be tempered by the realization that the community at large should not suspect that the relatives of the dead man dare to harbor such subversive feelings.
The nuances of each situation modify the emotional responses of the people involved. Because the nuances are infinite, so are the responses; but infinite in the case of emotional reactions does not mean arbitrary. All possible emotional scenarios are grounded in the never-explicitly-articulated but nevertheless unquestionable ontological assumption that for our species death is final, and dead people do not come back to life.
Think then what happens to all of the above scenarios if the man in question certainly died, was dead for three days, and then came back from the dead. Will his family members still feel grief mingled with anger, or desire for revenge, or fear? Or will they experience half grief-half joy? or mostly joy? or joy mixed with desire for revenge? or joy mixed with some fear? Will they try to conceal his return and pretend to be grieving and clamoring for revenge? And what will be the meaning of revenge? Would the murderer be sentenced to die, or would he be sentenced to die in the same strange sort of way, so that he too could come back in three days? Or would he be publicly commended for committing the act of murder and thus making it possible for his victim to come back in such a glorious fashion? And can we really call the murdered man a victim in this case? And is he really a man, given that he managed what no other human being ever did?
Violations of ontological expectations thus seem to be ripe with narrative possibilities. Turn to any realm of ordinary human experience (social, emotional, ethical), and consider it in the light of such a violation—and there is a story waiting for you. Of course, not every violation would do. As Boyer point outs, some of them are more inference-rich than others, "which makes the difference between good and bad stories." Imagine, for example, that the same day returns again and again—which would be a violation of our intuitive expectations about the "nature" of time—then flesh out that counterontological idea by trying to envision a predicament of an ordinary man stuck in that day. You may end up with an exciting movie, such as Groundhog Day. On the other hand, imagine that the same day returns again and again, and then try to envision a predicament of a particular magical being, for whom time flows backwards six days a week and sideways on Tuesdays, stuck in that day. I am not saying that it is absolutely impossible to muster enthusiasm for such a story featuring an "overdose" of ontological violations (working with cognitive approaches to fiction teaches one pretty quickly not to opine on what is narratively "impossible"), but it is rather difficult. Piled on thickly, ontological violations are subject to the law of diminishing returns; their narrative potential may slide downward quickly.
To clarify further the issue of inference-rich violations of our intuitive ontological expectations, consider the relative narrative value of the three following stories:
1. Once upon a time there was a darning needle. One day it was used to mend leather slippers, and it broke. The owner of the slippers dipped the broken needle in hot sealing wax and stuck it to her blouse as a brooch. Soon thereafter, however, it fell out of the blouse and down into the sink. It landed at the bottom of the gutter, and it still lies there.
I think you'll agree with me that this story is not terribly exciting. The possibility of your remembering it for its own lively sake and presenting it to others as an entertaining little tidbit is slight. We can envision situations in which you will repeat it to somebody to make some larger point. For example, if you are a thrifty person wishing to warn a friend who uses darning needles for wrong purposes, you may offer this story up as a cautionary tale. Or, if we imagine a community where needles are scarce and function as status symbols we can use that story to illustrate an adage along the lines of sic transit gloria mundi.
You can see what contortions I am going through to endow the chronicle of the darning needle with some narrative appeal. The truth is that every turn of this story merely confirms our intuitive expectations about artifacts; for example: artifacts have functions; at times they change their functions; if dropped, a tiny object is likely to get lost; artifacts are subjects to laws of gravity. As such, this little vignette calls for an absolute minimum of new interpretations and leaves us with no burning unanswered questions, except, perhaps, "who cares?" and "what's the point?"
But let us change that story slightly:
2. Once upon a time there was a darning needle. One day it was used to mend leather slippers, and it broke. The owner of the slippers dipped the broken needle in hot sealing wax and stuck it to her blouse as a brooch. Soon thereafter, however, it fell out of the blouse and flew up in the air. It still floats over the kitchen table.
This narrative is distinctly more interesting because it violates our intuitive expectations over the domain of folk mechanics. By defying gravity, the needle emerges as a strange little object whose behavior prompts numerous questions, for example, will the thread float in the air too? Will the slippers? Was it something about the wax that did it? How will people in the house react to this event? What kind of house is this? What is going on? The author's work is cut out for her now: she has to construct a narrative framework within which this violation of our intuitive ontologies makes sense. We shall feel distinctly dissatisfied if the story simply ends here.
Note that once this story is indeed extended to make sense of the floating needle, it is likely that we won't hear much about that particular artifact anymore. The gently hovering needle is not that fascinating by itself (Kubrick's 2001: A Space Odyssey  is not, after all, a chronicle of the hovering steel slab, however counterontological an object such a slab is). Its attraction lies in its being a part of something else, some possible world similar to and yet different from ours. It is the account of that world that we are hoping to get, or, to be more specific, it is the description of that world's inhabitants, their relationships, thoughts, and feelings that we are hankering after. Intensely social species that ours is, we always prick our ears at the possibility of hearing a story that would provide our theory of mind with relevant material to process. The image of a floating needle seems to promise such a story.
Finally, consider the opening of "The Darning Needle"—a short story by Hans Christian Andersen—the way it actually appeared in print. The plot is exactly the same as in example 1, but the ontological violations chosen by the author open that plot up in all kinds of unforeseen directions:
3. Once upon a time there was a darning needle who was so refined that she was convinced she was a sewing needle. "Be careful! Watch what you are holding!" it shouted to the fingers who had picked her up. "I am so fine and thin that if I fall on the floor you will never be able to find me again."
We still have some discourse here that agrees comfortably with our intuitive expectations about artifacts: the needle has a function; it is subject to the laws of gravity; its size makes it difficult to find it once it is lost. At the same time, our protagonist is a talking needle! A needle aspiring to a higher social status and wishing to convince others around her that she deserves to be treated as if she has already attained that exalted status! A self-deluded needle! Such a creature clearly cannot be assimilated by either the "artifact" or the "person" category. It will remain one of a kind—the needle with a distinctly human mind—and as such it immediately opens up a broad array of narrative possibilities.
What are those possibilities?
7. The Stories That Can Be Told About a Talking Needle
There were no voluble needles in the ancestral environment in which our tendency to essentialize living beings took shape. This means that once we hear that a given entity has a capacity for talking and complex mental stances, we cannot help but to start perceiving that entity in terms of what is "in the nature" of human beings. The story that opens with an image of a loudly ambitious needle can thus develop in infinitely many directions, all reflecting our intuitive ontological expectations about people.
People weave complicated webs of social relationships. They have friends, enemies, colleagues, associates, mentors, pupils, neighbors, servants, and masters. They receive inheritance, serve on jury duty, aspire to higher social status, get demoted, lie, get caught, reinvent themselves, fall in love, tell stories about themselves and others, dedicate their lives to important causes, waste their time, get married, entertain themselves and their friends, bore everybody around them to death, and so forth. Our talking needle can do all these things too.
Except that, just as in the earlier case of the man who comes back from the dead (a violation of our intuitive ontological expectations, which forces us to reconsider all familiar social scenarios concerning death, in which such a man would figure), here, the details of the social life of a protagonist who is also an artifact would be adjusted to reflect that protagonist's ambiguous ontological status. Here are some possibilities. First, the needle's sworn enemy would have the capacity to stymie its love life by hiding it behind a stack of dishes on a kitchen shelf. Second, its career advancement ripe with delightful new social opportunities could be attained through having being broken in two and dipped in hot sealing wax. Third, its admiring retinue may include a thread trailing behind it. Fourth, its service on a jury duty may prompt other members of the jury to be particularly careful so as not to sit down accidentally on their sharp tiny associate.
I am inventing some of these scenarios and lifting others from the tales of Andersen—that doyen of fictions that explore conceptual hybrids such as ambitious needles, hopeful street lamps, and mumbling cucumbers. I want to emphasize again that as immensely playful, and creative, and unpredictable as these stories are, their unpredictability does not imply arbitrariness. It is true that their range of plots and subplots is infinite—as is the range of social situations open to human beings—but the actual narrative expressions of all those plots will be structurally constrained by the nature of ontological violations present in them. This is to say that an author can write the talking needle into practically any recognizably human social situation, but then he will have to adjust some aspects of that situation to reflect the unique nature of the protagonist. The resulting narrative will thus contain a number of thought experiments exploring the implications of situating a human mind in a completely inhuman body, moreover, a body that has certain properties defined by the artifact's erstwhile function.
Stories of animated artifacts with complex and distinctly human social lives have a long pedigree. (After all, the readers of antiquity shared our cognitive biases and were as fascinated by images of conceptual hybrids as we are today.) One of my favorite examples comes from ancient Syria. On his fantastic travels, the narrator of Lucian's True Story (c. 166 A.D.) visits "Lamptown," the city that is located "in the air between the Pleiades and the Hyades, though much lower than the Zodiak." The inhabitants of Lamptown behave mostly like ordinary human beings, although they still die like . . . well, lamps:
On landing, we did not find any men at all, but a lot of lamps running about and loitering in the public square and at the harbour. Some of them were small and poor, so to speak: a few, being great and powerful, were very splendid and conspicuous. Each of them had his own house, or sconce, they have names like men, and we heard them talking. They offered us no harm, but invited us to be their guests. . . . They have a public building in the centre of the city, where their magistrate sits all night and calls each of them by name, and whoever does not answer is sentenced to death for deserting. They are executed by being put out. We were at court, saw what went on, and heard the lamps defend themselves and tell why they came late.
The social and emotional life of lamps continues to attract writers. Here is the synopsis of Pixar's short animation feature Luxo Jr., created by John Lasseter in 1986:
A baby lamp finds a ball to play with, and it's all fun and games until the ball bursts. Just when the elder Luxo thinks his kid will settle down for a bit, Luxo Jr. finds another ball—ten times larger. Luxo Jr. has a great dad in the larger lamp. Even though he is a bit unpredictable, the elder Luxo gives him room to grow and explore. And the tiny lamp has no problem with that.
Read side by side, the 166 A.D. and 1986 A. D. "lamp" stories vividly illustrate my larger point about the simultaneously cultural and cognitive construction of our fictional representations. On one hand, the image of a socially engaged lamp titillates us and yet makes sense today for the same reason that it titillated readers and yet made sense to them nineteen hundred years ago: it forces us to perceive this artifact as belonging to the domain of human beings, thus activating our inferences about what human beings do and feel. On the other hand, certain crucial details of the stories mark them as deeply grounded in their respective cultural milieus.
Some of these details are emotional. The relationship between the protagonists of Pixar animation—"the elder Luxo gives [Luxo Jr.] room to grow and explore"—are described in terms intelligible and attractive to an audience belonging to a very specific place and time. Thus, I know that fifteen years ago, as a new immigrant from Soviet Russia, I would not have felt the appeal of this sentiment as strongly as I feel it today, now that I have learned to appreciate the value that my friends and colleagues put on the view that children should be given "room to grow and explore." It's not that this sentiment was inconceivable in Russia when I was growing up, but, among the people whom I knew, it was not held at the same premium as it is (and has been for some time) among many people in this country. And whereas I have no way of telling if Lucian's first readers strongly agreed with this sentiment, I suspect that even in his time it might have been more intelligible to readers of some social classes, economic means, and personal temperaments than to others.
Yet other details are technological. Lucian's readers could understand why putting the lamp out meant killing it. Today's viewers would hardly perceive switching Luxo Jr. off as killing him; they could read it as making him take his nap so that he would wake up refreshed and ready for new exploits, however ardently he might protest against "going to bed." The omnipresence of electricity has taken some of the drama out of the extinguished light. Sure, Luxo Jr. can still "die" but it would take more than simply switching him off or unplugging him (perhaps to kill the lamp today, you have to smash it or cut the cord?).
And so forth. At every turn, the tone of the "Lamptown" passage bespeaks the sensibility that differs from the sensibility animating the story of Luxo Jr. and his father. Yet the fascination with the image of an animated artifact and the patterns of such an animation remain the same (i.e., people "loiter" and play, and so does an animated lamp; people have families, and so do animated lamps; people die, and so can an animated lamp) because they are rooted in the biases of our cognitive information processing that remained stable for those two thousand years.
A True Story is considered one of the earliest surviving examples of science fiction. Let us now turn to more familiar examples of this genre and apply to them what we have learned about our cognition of categorization. Using our new terminology, I suggest that the protagonists of these stories—robots, cyborgs, androids—are counterontological entities who straddle the respective domains of artifacts and living beings. I further argue that at least one recurrent plot turn of our science fiction stories—the plot of a rebellious robot—can be traced to our drive to restore the broken conceptual feedback loop and to fit an apparently counterontological entity within a comfortably familiar ontological category.
8. Asimov's "The Bicentennial Man"
Remember Andrew, who "had not wanted to remember" his serial number? Asimov's short story "The Bicentennial Man" (1975) begins to build on—and tease—our intuitive ontological assumptions about living beings and artifacts from page one. We meet Andrew Martin when, apparently sad and, in fact, "driven to the last resort," he visits a doctor's office. This information provides enough input for our essentializing proclivity to get in gear and enable us to categorize Andrew as a human being.
"Facing" Andrew "from behind the desk" is a "surgeon." This also gives us enough information to conceptualize this other character as a human being. The nameplate on the surgeon's desk includes a "fully identifying series of letters and numbers, which Andrew [does] not bother with. To call him Doctor would be quite enough." A slightly strange bit, this one, about the series of letters and numbers identifying a person, but we do not let it override our initial assumption that the surgeon is a human being. As a default assumption (for doctors who face us across their desks in our world are still human, even with the advance of new technologies), it needs stronger evidence to be overruled.
Moreover, we are riding a comfortable feedback loop with Andrew. His decision not to "bother" with the surgeon's identification tacitly reinforces our initial perception of him as a person, and so does his next action:
ēWhen can the operation be carried through, Doctor?' he asked.
The surgeon said softly, with that certain inalienable note of respect [yes, certainly a person] that a robot [WHAT?!] always used to a human being [right, Andrew is a human being]: ēI am not certain, sir, that I understand how or upon whom such an operation could be performed.'
The revelation that the surgeon is a robot is somewhat shocking (though now we can account for that peculiar bit about "letters and numbers" on his nameplate), and Asimov rushes on to strengthen our perception of the surgeon as an artifact:
Andrew Martin studied the robot's right hand, his cutting hand, as it lay on the desk in utter tranquility. The fingers were long and shaped into artistically metallic looping curves so graceful and appropriate that one could imagine a scalpel fitting them and becoming temporarily, one piece with them.
There would be no hesitation in his work, no stumbling, no quivering, no mistakes. That came with specialization so fiercely desired by humanity that few robots were, any longer, independently-brained. A surgeon, of course, would have to be. And this one, though brained, was so limited in his capacity that he did not recognize Andrew—had probably never heard of him.
Note the rhetoric of functionalism seeping through the description of the robot. Its fingers are "shaped" in a certain way to enable it to excel at its function, which implies a "shaper"—a "maker" behind this particular artifact. The word "brained" intensifies this impression, for we do not literally "brain"—"give brains" to human beings (we do brainwash them, but this is a metaphorical use of the functionalist language, to be discussed separately).
As the conversation continues, Asimov finds yet another way to stoke our functionalist thinking about the surgeon:
Andrew said, ēHave you ever thought you would like to be a man?'
The surgeon hesitated a moment as though the question fitted nowhere in his allotted positronic pathways. ēBut I am a robot, sir.'
ēWould it be better to be a man?'
"So as to be able to better perform his function," we infer. Like any artifact, the surgeon is what he does, and his ambitions are limited to doing that one thing well. Note here—lest we forget why this tracing of rhetoric of functionalism is important—that this rhetoric in all its different guises would not have been effective in amplifying our impression of the surgeon as an artifact, had it not been for our evolved cognitive tendency to think of artifacts in terms of their function. This rhetoric makes sense—as does a story built around that rhetoric—because it appeals to our evolved cognitive heritage.
"Obedience is my pleasure," drones the surgeon; and it seems that by the end of this introductory scene Asimov almost succeeds in making us reconceptualize as an artifact the character that we initially perceived as a living being. I say almost because as a talking and "independently-brained" machine, this robot still activates in us certain inferences associated with living beings and so remains a somewhat counterontological entity.
The issue of gradience is important here. At the end of their conversation, Andrew Martin makes a startling revelation that he, "too," is a robot. Still, it is clear to us that Andrew's ontological status is more ambiguous than that of the machine he is talking to. Andrew comes across as "more" counterontological than the surgeon: he seems to be capable of a broader range of emotions, and we have not yet heard anything about his function (though as an artifact he is bound to have one). If I were to ask you now, jumping ahead of my argument, which of the two robots is more likely to "rebel" against his masters, you would certainly point to Andrew. It is not that a rebellion is inconceivable in the case of the surgeon, but it would require some catastrophic event (e.g., a short circuit) to wake him up from his happy artifactdom.
Having first established Andrew as a human being only to then drop the bombshell of his "too" being a robot, Asimov proceeds, in the second part of the story, to build up our impression of his protagonist's ontological ambiguity. He does this by mixing rhetoric that activates inferences belonging to the domain of artifacts with rhetoric that feeds into our essentialist biases. We hear, for example, that when first "manufactured," Andrew was "smoothly designed and functional," with a "serial number NDR—," only to learn immediately after that he forgot and "had not wanted to remember" his serial number. An artifact who wants to forget that he is an artifact (which implies a fully developed theory of mind) cannot be assimilated by our cognitive systems along the same lines as a chair, a toaster, a microchip, or even a fellow machine completely satisfied with its present status (i.e., the surgeon).
The information that is supposed to signal most strongly Andrew's status as an artifact—the reference to his function—is couched in purposefully ambiguous terms. We are told that Andrew "had been intended to perform the duties of a valet, a butler, a lady's maid." The pointedly asexual ring of this job description (valets and butlers tend to be men whereas lady's maids, women) reminds us how meaningless gender differences are for a machine. But, by the same token, the triple function that forces one to move from tasks associated with one gender to tasks associated with another seems to mark Andrew as a priori more versatile than, say, a robot surgeon who is made to excel at just one task. This tentative impression of Andrew's versatility is strengthened in the next paragraph, in which he is portrayed as happily transcending his original function, for the children of his owners "would rather play with him" than use his services as a valet.
Andrew's "rebellion," which unfolds gradually and is nonviolent, consists of transcending his original design and becoming as unpredictable and multifunctional as (if not more so) any human being. First, he turns out to be a brilliant artist. His woodwork earns him a small fortune and later enables him to "buy his freedom" from his owners, even though he continues to live close to them and be treated almost as a family member. Increasingly independent as Andrew is, however, his robot status still makes him vulnerable, for any human stranger can order him around and force him to do things harmful to himself. To address this incongruence, Andrew's friends lobby for a law that would protect him, and, eventually, a series of laws are enacted "which set up conditions, upon which robot-harming orders were forbidden."
Meanwhile, Andrew undergoes a series of surgeries (many of them made possible by his innovative research in prostheology—his new area of interest), which turn him into a more "organic" creature—one who looks, eats, and breathes like a human being. His work with "prosthetized devices" takes him to the Moon, where he is "in charge of a research team of twenty human scientists," while robots treat him "with the robotic obsequiousness due to a man." Finally, Andrew insists on one more operation, this time to render himself irrevocably mortal: a suicide of sorts, but, nevertheless, the step that convinces the "World President" that Andrew should be granted his life-long ambition and be officially declared a human being. The story ends as Andrew lies dying; wishing that his last thought would be that he "was a man"; and thinking instead of the late daughter of his original master, whom he adored—a romantic failure to control his emotions, which should conclusively mark him as a person and not a robot.
Andrew's actions frighten and antagonize his makers, the powerful corporation U. S. Robots. Especially in the early days of his quest for humanity, the corporation tries to lay its hands on Andrew so as to dismantle and study him. When that fails, they revamp their entire enterprise to ensure that the new machines that they produce will never be as flexible and troublesome as this renegade robot. For, nonviolent as Andrew is, his behavior seems to vindicate the Frankenstein complex. Andrew grows smarter and richer than the human beings who "manufactured" him, and he learns to deploy a wide range of emotional and legal tactics to make them do what he wants. At one point he is aware of having contributed to the "lying," "blackmail," "badgering and humiliation" of the head of U. S. Robots; at another, he finds himself setting "stern conditions" to that man's "stunned" successor. (In both cases, Andrew's goal is to move closer to becoming human.)
I see Andrew's rebellion as "cognitively satisfying" for us, the readers, because its outcome—the erstwhile robot's official admission to the ranks of humanity—eases the cognitive tension built up by the story. Andrew is a narrative construct that activates inferences from two different conceptual domains: the domain of living beings, and the domain of artifacts. By making Andrew go through his life as a half-man and half-machine—an effect achieved by speaking in the same breath of Andrew's complex emotions and his "man-made" features—Asimov forces his readers to straddle these domains. By bringing his rebellion to fruition and showing Andrew die "as a man," he finally allows us to conceptualize him as a human being, or, if not really and truly human, then at least, as more human than before.
With this in mind, perhaps we can redefine from a cognitive perspective the psychological phenomenon known as the Frankenstein complex. When we encounter a fictional character whose ontology seems to pull us in two different directions, we intuitively grapple for the ways to restore at least one of our broken feedback loops (for we cannot restore both) and to resolve the cognitive ambiguity by conceptualizing that hybrid as either a living being or an artifact. But what kind of plot development would allow us to achieve this kind of cognitive resolution?
To become more "artifactual" in our eyes, the hybrid has to signal strongly his rigid functionalism. And so the surgeon in "The Bicentennial Man" signals it by showing his inability to think of himself outside his profession. To become more of a person, on the other hand, the hybrid has to emerge as humanly multifunctional, which means that he has to be perceived not in terms of one limited function but in terms of some ineffable, invisible, and ungraspable "essence" (as Andrew seems to be by the end of the story). To do so, however, he has to rebel against the inhumanly narrow role foisted onto him by his makers, a rebellion which could be violent or peaceful, depending on the context of the story. Our anticipation of this rebellion—which could otherwise be seen as an expression of our subconscious hope for a resolution of the conceptual problem that we have on our hands—is what we call the Frankenstein complex.
9. Cognitive Construction of "Undoubted Facts":
"The Bicentennial Man" and the Logic of Essentialism
As we follow Andrew's metamorphosis from a machine to a "man" and note the rhetoric used to negotiate the difference between a robot and a living being, we may get an uncanny feeling that either Asimov was familiar with the research of Atran, Gelman, and their colleagues, or that those scientists read a lot of Asimov prior to embarking upon their studies. As one example of the resonance between those studies and the logic of essentialism cultivated by "The Bicentennial Man," consider Gelman's report of a conversation that her colleague, Francisco Gil-White, had with a group of Kazax men in Western Mongolia:
In the midst of conversation about nature-nurture conflicts, Gil-White asked the following: ēIf I stayed here, and learned Kazax, and Kazax custheory of minds, married a Kazax girl, and became a Muslim, would I not be a Kazax?'" The respondent's reply was: "'Even if you do everything like a Kazax, and everybody says you are a Kazax, you still aren't a real Kazax because your parents are not Kazax. You are different inside.' And he pointed to his chest."
This conversation about what makes one a Kazax, with its strong underlying assumptions about the "hidden, nonobvious properties that impart identity," is both profoundly familiar and yet striking precisely because of its easy familiarity and its seeming "naturalness." We see the man's point—simply cannot help seeing it because of our own essentialist proclivities—even if we do not particularly like that point or cannot explain what it is exactly about being born to Kazax parents that makes you a Kazax (there is no "gene" for "Kazaxness," after all, no more than there is a "gene" for being a Ukrainian or Latvian).
Look now at the scene in "The Bicentennial Man" in which, after returning from the Moon, Andrew approaches the head of the legal firm that represents him and asks whether it would be possible to be "legally identified" as a human being. As he puts it,
On the Moon, I was in charge of a research team of twenty human scientists. I gave orders that no one questioned. The Lunar robots deferred to me as they would to a human being. . . . I have the shape of a human being and organs equivalent to those of a human being. My organs, in fact, are identical to some of those in a prosthetized human being. I have contributed artistically, literarily, and scientifically to human culture as much as any human being now alive . . . Why, then, am I not a human being?
The lawyer replies that, "treated as a human being by both robots and human beings," Andrew is thus a "human being de facto" and he should be satisfied with that, since any attempt to establish him as a human being de jure "would run into human prejudice and into the undoubted fact that however much [he] might be like human being, [he is] not a human being."
To me, the most interesting aspect of this conversation is the lawyer's appeal to "the undoubted fact" of Andrew's essential non-humanness. Generally, claiming that something is an "undoubted fact" may have the opposite effect of making the audience question the truth-value of the statement and suspect some interested motive in the speaker. So when we have the lawyer portraying himself as the last word on the subject (which, the story implies, he is not), we are inclined to think that he is narrow-minded and prejudiced and perhaps wrong. We have been thus inched further along toward perceiving Andrew as human.
As far as essentializing goes, this is a curious moment. What the lawyer says is grounded in our essentialist biases and as such is something that many people would actually find quite commonsensical. After all, the world in which one can look, and act, and be treated as a certain kind of person and still not be that kind of person is the same old familiar world in which "even if you do everything like a Kazax, and everybody says you are a Kazax, you still aren't a real Kazax," unless you were born a Kazax. In both cases, our evolved cognitive tendency to essentialize living kinds naturalizes a distinction that otherwise would be difficult to defend either logically or scientifically (e.g., a Kazax versus a non-Kazax; an android who is thoroughly "like" a human—in fact, almost "more" than human—versus a "real" human). So, what Asimov does here is take an everyday essentialist notion and put it in the mouth of an arrogant lawyer—a somewhat suspect authority to begin with—to the effect of undermining the validity of this notion on the grounds of suspect authority.
But undermined does not mean eradicated. The essentialist reasoning about what Andrew "really" is or is not continues to inform the story, in fact, has to, for this is the kind of story that exploits our underarticulated but enduring essentialist intuitions from its first to its last line.
Hence, soon after the conversation with the lawyer, another, more appealing authority (an older congresswoman) attempts to clarify the reason for Andrew's difference. The clarification is pointedly anti-scientific but it is made to sound plausible because of the way it feeds our essentialist biases. When Andrew persists in seeking some logical explanation of what it is that makes him so irrevocably different from human beings, he is finally told that the proof is in the brain: "Your brain is man-made, the human brain is not. Your brain is constructed, theirs developed. To any human being who is intent on keeping up the barrier between himself and a robot, those differences are a steel wall a mile high and a mile thick."
Andrew still refuses to simply bow to the wisdom of this quintessentially essentialist view (which, incidentally, recruits the rhetoric of functionalism to signal Andrew's artifactual nature) and wonders instead if one "could get at the source of [people's] antipathy" to the constructed brain. To this, his friendly interlocutor says "sadly" that it is "the robot in [him] that drives [him] in [the] direction [of] trying to reason out the human being." Again, this is a thoroughly essentialist sentiment, for it relies on our acquiescing (without even noticing it) that it is "natural" for a robotic living being to be stubbornly logical in a situation in which a "real" human being would apparently just agree that humans are unreasonable and leave it at that.
Note that I am not saying this to condemn the cruel humans who would not let "poor Andrew" in the club of "true" humanity. My point is that every turn of the story seems to make sense because of the unspoken essentialist assumptions that we bring to bear upon it. If by some impossible magic we were to remove that essentialist propping, the story would collapse as if robbed of its skeleton. (Of course, it would collapse equally fast, if not faster, were we to remove our unspoken assumption that there is a state of mind behind each action of a fictional character. Our cultural representations, including our fictional narratives, are structured by our intuitive ontological expectations on an untold variety of different and yet interlocking levels. In fact, it seems that the more we find out about the workings of our evolved cognitive architecture, the more daunting becomes the challenge of developing a conceptual framework that could grasp the overall pattern of interactions among these levels.)
Moreover, to come back to my facetious suspicion that either Asimov read too much Gelman or Gelman read too much Asimov, consider the similarity between the story's pontification about Andrew's brain as the seat of difference between him and "real" humans and our everyday reasoning about what constitutes the "essence" of a person, her "soul," her "core," as reported in The Essential Child.
It seems that to support our essentialist thinking about individuals, we tend to look for the "hidden nonobvious causal properties that impart identity" in such bodily organs as the heart or the brain. So people who undergo heart-lung transplants sometimes comment on certain personality changes, suggesting that some of their donors' "essence" has transmigrated to them. A wife of a heart donor says about meeting the recipient of her dead husband's heart: "I could feel his essence, his energy." Her interviewer (not a cognitive psychologist) chimes in: "Anyone who receives a new heart is getting a big ball of subtle energy. Ancient cultures have known about subtle energy throughout history, and have viewed it as the vital force of all creation." What Gelman finds interesting about such accounts is "not the question of whether or not it is true or even plausible, but rather how [the interviewed people] construct a personal essence: as nonvisible, internal, persisting through massive changes, and having the capacity to influence outward behaviors and preferences. It is both material (located in the heart, a flesh-and-blood bodily organ) and immaterial (an ēenergy' or ēsoul')."
As the seat of the person's "essence," the heart figures frequently in fictional stories featuring counterontological creatures, such as Tim Woodman from Frank Baum's The Wizard of Oz (1900) or Sharikov from Mikhail Bulgakov's Dog's Heart. Although in "The Bicentennial Man" Asimov opts for the brain as the locus of difference between Andrew and a "real" human being, the logic is exactly the same. We are told explicitly that the "seat of Andrew's personality is his positronic brain and it is the one part that cannot be replaced without creating a new robot"; so it is not really surprising that toward the end of the story, Andrew decides that the only thing that he can do to bring down that "steel wall a mile high and a mile thick" which separates him from humanity is to change irrevocably the nature of his brain. As he puts it,
See here, if it is the brain that is at issue, isn't the greatest difference of all the matter of immortality? Who really cares what a brain looks like or is built of or how it was formed? What matters is that brain cells die, must die. Even if every other organ in the body is maintained or replaced, the brain cells, which cannot be maintained or replaced without changing and therefore killing the personality, must eventually die.
My own positronic pathways have lasted nearly two centuries . . . and can last for centuries more. Isn't that the fundamental barrier? . . . Human beings . . . cannot tolerate an immortal human being . . . . And for that reason they won't make me a human being.
I have removed that problem. Decades ago, my positronic brain was connected to organic nerves. Now, one last operation has arranged that connection in such a way that slowly—quite slowly—the potential is being drained from my pathways.
Andrew's plan appears to succeed spectacularly. His decision to sacrifice his life to be human catches "the imagination of the world," and, shortly before his death, he is officially declared a man. Yet note how ironic both his plan and its success are from a cognitive perspective: Although we perceive both natural kinds and specific individuals in terms of their essences, any actual attempt to locate the seat of that essence is bound to remain inconclusive at best and ridiculous at worst. And so it should be, of course, given that our essentialist thinking reflects particularities of the cognitive makeup of our species rather than certain objective truths about the world.
Which means that Andrew's quest after the ever-receding essential difference between himself and human beings is, objectively speaking, meaningless even though it feels cognitively meaningful. Andrew may accumulate one human trait after another—human appearance, human creativity, prosthetic organs widely used by humans—neither any one of these traits by itself nor their combination (or "cluster," to use Kripke's term) will ever capture the essence of being human, for there is both too much and too little to capture when one hunts for the embodiment of an essence.
But if Andrew's quest is bound to fail in objective terms, it may succeed rhetorically, which in this case is even better. And so to begin substituting rhetoric for reality, Andrew takes as his starting point one essentialist assumption that has been dogging him lately—"the essence of the human being is the human brain"—and counters it with another: "the essence of the human brain lies in its mortality."
Andrew's theory of what constitutes the essence of the human brain is as plausible and indefensible as any essentialist theory. However, precisely because it is both plausible and unfalsifiable, the introduction of a strong appeal to the emotions of the people whom he is to persuade—his willingness to back up an indefensible essentialist assertion with his life—earns him the coveted name of man. This does not mean that these people have managed to re-categorize Andrew as a human being in their own minds (this is a really murky area), but they have clearly become doubtful enough about his ontological status to finally accede to his plea.
Let me use a stark example to illustrate my argument that an appeal to emotions is more effective if an issue at hand forces us to think in essentialist terms. If I show you that I am willing to die to prove my point that pigs fly, you may think that I am crazy or overly passionate about my idŽe fixe, but your opinion about pigs' levitating abilities will hardly change. But if I show you that I am willing to die to prove my point that to be a human being is "all about" being free; or loving one's family; or being mortal, you will certainly be impressed enough to think, if only for a short time, "what if she is right?" Because essentialist thinking is both a powerful mental habit and an ontological bottheory of mindless pit, we seem to be particularly vulnerable to a rhetoric that appeals to emotions to argue about essences.
(Note, incidentally, that an emotional dynamic is also at play in the earlier episode with the lawyer: because an unappealing character offers the "verdict" about Andrew's non-humanness, we are less inclined to agree with the verdict. The issue itself remains irresolvable but our emotions can be manipulated into making us think that perhaps there is a resolution and it is quite different from what this lawyer claims with such tactless confidence.)
And of course Asimov did not have to read the research of cognitive evolutionary scientists to figure out that Andrew might want to appropriate the essentialist thinking of humans in order to make his own plea for human status compelling. Writers work with perfect assurance our evolved cognitive tendencies because constructing their stories this way makes intuitive sense to them. I knew that you would say that kerpas do not smile sadly when you ignore their polite inquiries because I myself know that they cannot (that is, as far as I can "know" anything about something that does not exist). That knowledge is both innate—that is, made possible by our evolved cognitive architecture—and cultural, for had you and I been born into an alternative world in which substances did smile, it would have overridden our innate propensities. Similarly, in writing stories which rely extensively on our essentialist biases, Asimov "knew" that they would make sense for us because they made intuitive sense to him. Cognitive psychology and anthropology thus help to bring into the open implicit assumptions about the world, which make the three-way communication between the writer, the reader, and the text possible.
The fact that biological artifacts don't immediately come to mind as instances of the category of artifacts is rather puzzling. Biological artifacts are very common. After the Neolithic revolution some 13000 years ago and until the industrial age, they were the most common artifacts in the human environment. Most people had more domesticated plants and animals than tools, clothes, weapons, furniture and other inert artifacts. Why should then the notion of an artifact be psychologically based on a prototype which is not all that representative? . . . Maybe because, during the long Paleolithic era, simple inert tools were the only artifacts humans had. If there is some innate basis for our notion of an artifact, it probably evolved in an environment where stone tool were indeed prototypical, and a mere 13000 years with domesticated plants and animals around was not sufficient to displace this mental habit. (136)
Even at very early ages and in the absence of task-relevant experience (Hauser, Pearson et al., 2002; Santos, Sulkowski, Spaepen, & Hauser, 2002), non-human primates seem to parse the objects in their world into meaningful global categories—tools, foods, landmarks, and animals (L. Santos, M. D. Hauser, & E. S. Spelke, 2002). Such evidence suggests that non-human primates may have innate biases to interpret their world in domain-specific ways. In addition, non-human primates seem to reason about different domains in ways that make ontological sense; their recognition of which features matter for different domains seem to map onto those of conceptually-sophisticated human adults. For these reasons, we side with the domain-specificists and argue that both tool-using and non-tool-using primates are biased to distinguish tool-like objects from other ontological categories and that these biases facilitate experience-based learning about different kinds. (283)
developmental processes of the brain would seem to predict that . . . the statue of the Virgin brought out to the fields [as peasants pray for rain] will explain her powerful place, and every example of either the success or the failure of the crops can be attributed to her will and power. Furthermore, it would be hard to control the recursive elaboration of the statue's quasi-humanity. If she is animate in regard to rain, it is easy enough to think that maybe she is like a human woman in other ways as well, and must be provided with greater sacrifices, new garments, or more jewels. That she is embedded in a neuronal web which has, over the years, woven together the image itself with needs, hopes, actions, beliefs, past experiences, and with a set of rational inferences as well, would make her hard to dislodge from her powerful position. (Word vs Image, 47)
For further provocative discussion, see chapters 2 and 3 of Word vs Image.