Is it wrong to make fictional characters suffer?

I am sick and very fatigued this week, so the follow up to my Erik Hoel post will have to wait. The following will be messy and informal; it’s what I can muster right now.

I don’t mean to be derogatory with the term “crazy train”. Joe Carlsmith and Matt Adelstein have both used it to mean crazy-sounding implications of radical—but well-supported—philosophical assumptions. These are worth considering, because they are very “big if true,” even if they are unlikely to be true. Today I am considering one crazy-train implication of my beliefs with respect to minds.

Consider the “virtual minds” reply to Searle’s Chinese room, particularly the internalized Chinese room. Some theorists respond to Searle by saying, look, you don’t understand Chinese, but the system of you + the rulebook + the room understands Chinese. Of course you, on introspection, don’t detect any understanding! Searle replied by saying, fine, imagine there is no rulebook and no room, but I have brute-force memorized all of the rules. I still don’t understand Chinese, but there is no locus of possible understanding except me.

My favorite reply is the idea of a virtual mind, much like a virtual machine in computers. Inside of your mind you are running operations which simulate another mind, a mind which understands Chinese. These workings are not introspectively accessible to you, in the same way that if you sat down and mentally simulated a neuron-by-neuron firing of a brain in a pain state, your arm wouldn’t start throbbing; you might have no idea that the brain is in pain. And yet you can “run” something which understands what is going on.

The material functionalist position is that there’s really no difference between a sufficiently detailed virtual mind and a “real” mind. The Chinese room is only a poor approximation of a mind to the extent that it fails to do things and have capacities that a “real” mind which understands Chinese would have. I find this a pretty compelling picture for understanding, and similarly compelling (though not decisively so) for feeling. It seems to me that if it’s possible to simulate a mind that feels pain in a computer (which I think is possible, even if extremely difficult to do), then it’s possible to simulate a mind that feels pain “virtually”, in an existing mind.

In fact, it might be easier to do things this way. It’s easier for us to think of things like pain and rage and motivation as primitives, easy for us to model what others might be thinking—we have “Theory of Mind,” aka. “folk psychology”, which makes us pretty damn good at predicting how people will behave under their knowledge and motivations. In the everyday case, sitcoms and dramatic irony wouldn’t work! In evo-bio cases, this helps you convincingly deceive others, or detect deception, or cooperate.

Okay, so we can do folk psychology. One of the ways we might do that is by running a primitive internal simulation of a mind, a small virtual mind. But there is now a question for the functional materialist who also thinks pain is bad, aka. me: is it bad to create virtual minds in our own brains who suffer—aka, fictional characters?

One response to this is that thinking about fictional characters, you involve a very roughshod, poor, simplistic virtual mind. It doesn’t have a great deal of depth, and there is more “intelligent design” than real simulation. I do theater, and when I acted as Edmund the Bastard in King Lear, I felt something like rage, victory, scorn, etc., but they were really shallow, pseudo-emotions, not the “real deal”. I’d take pretending to be angry over being really angry any day.¹ However, many folks, including me, also think that small harms aggregate, that 3^^^3 people getting dust specks in their eyes is worse than a single person being tortured (and here I explain why).

Think of Theon Greyjoy from Game of Thrones: tortured, humiliated, his personality and identity stripped away until he is deeply broken inside and out. Thousands, if not millions of people saw this, empathized with it, constructed little—rudimentary! simplistic!—virtual Theons in their heads. Those Theons suffered weak, sanitized versions of the horrific torture depicted in the show, but there are very many of them. I don’t think they aggregate up to the level of harm that a single real Theon would endure, but do they add up to something bad—at the level of a gut-punch, a sharp pain, a knife wound? What about at larger scales, or with more graphic suffering? Do these count as moral harms—not because they make the observers upset, but because they consist in the simulation and construction of millions of tiny moral patients?

There are a couple responses here. One is that unlike with other aggregation complaints, where you can exaggerate the numbers and the level of harm as you like, these don’t hold here. There are far fewer human minds engaged in simulation at any given time than there are suffering animals.

And while it’s quite possible that other animals suffer as much physical agony as the smartest human, these little simulated minds are unlikely to contain such rich emotions, or comparable emotions at all. What does the capacity for significant pain get you, in evolutionary advantage? Very strong aversive signals. What does the capacity for simulating significant pain in others get you? Not a great deal. You only need to predict that others in great pain are very strongly motivated to get out of it, and that they will be incapacitated, in order to make good decisions about them. In the worst case, enough simulation might actually be harmful because it could lead to too much empathy (for evolutionary fitness). So your brain might economize by painting very primitive labels on other people, which lack the depth needed for “real” pain. I can think of a couple replies to this: maybe one from uncertainty, maybe another from the evolutionary advantage of being able to simulate your future pain, but I find this pretty persuasive.

Additionally, there is a built-in defense mechanism which probably prevents things from getting too bad. When we simulate suffering, we are not totally black-boxed out in the way Searle was from his internal Chinese room. When I acted as Edmund, I felt the pseudo-emotions. If they had gotten too distressing, I’d likely back away. So if there was a more intense “virtual Edmund” being run in my mind, his suffering would still be limited. This adds some evidence, but I find this less persuasive. Suppose that we identify the neural profile of what happens when I am in immense pain, and that when I am merely imagining someone in immense pain, almost the exact same neural profile occurs. Yet I insist I am not feeling pain, just a vague pseudo-pain. It is possible that the difference in neural profile means that no suffering is occurring. Or it is possible that suffering is occuring, but “I” do not have access to it, as in the Chinese room case. I would find my self-report very strong evidence that “I” am not suffering, but less strong evidence that there isn’t a suffering simulated mind.

In conclusion, I think there is a possibility that it is immoral to propagate characters like Theon Greyjoy in the minds of millions of viewers, but it is quite improbable, and probably the simulation is swamped by the other moral effects (empathy, desensitization, aesthetics, etc.). The reasons why, however, are based on my considerations of how likely a realistic simulation is to be the case, not because it seems crazy or science-fiction-y. If I discovered that brains worked differently, then I might change my mind. That’s what the crazy train is for!

In terms of emotional distress, that is. It does take more mental energy to pretend to be angry, since there is a lot of intentional control going on. ↩