Reasons after nonpersons

AI will make some old-school philosophical thought experiments seem much more relevant

dynomight.net/nonpersons

The year is 2029 and you’ve finally done it. You’ve written the first AI program that is so good—so damn good—that no one can deny that it thinks and has subjective experiences.

Surprisingly, the world doesn’t end. Everything seems fine. Except—all those bongcloud philosophy 101 thought experiments people have done for generations suddenly seem much less hypothetical. But also, kind of tedious?

1.

Bongcloud 101: When Vader froze Han in carbonite, did he, like, die? And did he get reincarnated later? If he was never unfrozen, then when did he die?

You 2029: If you stop your program, do you “kill” it?

2.

Bongcloud 101: A thousand years from now, you discover a mineshaft with 500 cryogenically frozen bodies. If you don’t revive them, is that murder?

You 2029: You’re walking down the street and find a thumb drive with a bunch of AI programs on it. Are you ethically obligated to start them?

3.

Bongcloud 101: I have severe epilepsy, which you treat by cutting my corpus callosum so my two cortical hemispheres can’t directly communicate. After, I say I’m fine and I seem fine, but sometimes my two arms try to do contradictory things. And if you block the light going into the left half of my eyes, I can’t tell you what’s in front of me—even though my left arm still immediately captures any available burritos. Are there two people in my head?

You 2029: You take your AI and split it into two modules with limited communication. Is there now a second consciousness?

4.

Bongcloud 101: You and me, we love each other. So much that mere marriage won’t do. To honor our bond, what we need is high-bandwidth interconnect between all four of our cortical hemispheres. So we design a, like, interpersonal love-callosum. After installation, we’re confused at first, but a few weeks later, we act as one and report that we are a single sentience. Did one of us die?

You 2029: You take your AI and my AI and link them up. Did you “kill” one?

5.

Bongcloud 101: Bored one night, you sneak into my bedroom and recode my neurons so I have the memories and personality of Napoleon. When I wake up, I’m less interested in philosophical rambling on the internet, and much more interested in taking over the world. Am I still me?

You 2029: You take your computer and switch out the AI program for a new one. Is it still the “same” AI?

(If these AI questions seem tedious, please remember that’s the point.)

6.

Bongcloud 101: Once you were 8. Now you are 48. In all those years, you got mostly new memories, you gained a new appreciation for quiet restaurants and empty lawns, and most of your atoms turned over several times. Are you still the same person?

You 2029: You take your AI and slowly change it. Eventually, you’ve changed it a lot. Is it still the “same” AI?

7.

Bongcloud 101: Your identical twin is in an accident and their cortex is destroyed. Always generous, you volunteer to have your left hemisphere removed and installed in their head. Both bodies survive and—while not the brightest—both say they are you. Which is right?

You 2029: You take half of your AI and move it to a new computer. Which computer is the old AI? Neither? Both?

8.

Bongcloud 101: You’re having one of your edgelord what does it all mean life is finite why is there no permanence-type crises. But then you have a thought: Maybe you aren’t your atoms, but the pattern, so as long as the universe exists there’s a chance your pattern could be recreated. You bring this up with your math support group and get into a debate about how the probability of being recreated might evolve as the universe approaches thermodynamic equilibrium. Does immortality hinge on the solution to some infinite series?

You 2029: Someone blows up your computer. (Something about “mesa-optimizers” and the “orthogonality thesis”.) You rewrite your AI program and run it on a new computer. Is it still the “same”?

9.

Bongcloud 101 (the archetypal bongcloud): You scan your body, destroy it, and then have a machine recreate it on Mars. Did you kill yourself, or just transport yourself? If you accidentally failed to destroy the old you, should you happily jump into a vat of acid, content that there’s another copy out there?

You 2029: When you move your AI to new hardware, you accidentally fail to turn off the old one at the same time. If you turn it off later, does that mean you killed it?

Thoughts

Here’s what I notice: When you do these experiments with people, they’re thought-provoking, if perhaps “embarrassingly bongcloud”. But when you translate them to AI, they seem pedantic and silly—it’s “obvious” that they don’t have meaningful answers.

I think this shows that these questions don’t have answers for people either. They are based on abstractions that seem fundamental, but that’s only because of contingencies in how evolution made us.

Here’s how life and consciousness seem to work:

1. Consciousness comes bundled in discrete units called “animals”. Each consciousness is tied to a specific meat-computer.

2. You can count how many conscious things are in a given room.

3. Life has a beginning and end.

4. You are the same person you were yesterday.

None of these seem to be true for AIs. Once we start saving and restoring and copying and splitting and merging them, I bet we’ll notice:

• AIs aren’t really “born”, nor do they “die”. They are just systems that evolve over time.

• The AI is independent of the hardware it’s running on.

• It’s hard to say “how many” AIs exist. There will just be modules everywhere with different degrees of interconnect. Module 42,872,232 will have a huge fast pipe to module 42,872,231 on the same chip but a tiny slow pipe to module 52,030 on a satellite circling Neptune.

• It’s often meaningless to ask if an AI is “the same” as some other AI.

But if 1-4 aren’t true for AIs, then they aren’t really true for people either, right? If people could connect and merge and remix their brains at will, we’d look at personhood very differently. They are illusions that seem to be true because evolution happened to design us in a way where they are almost true, aside from the occasional glitch in the matrix like split-brain patients or twilight anesthesia.

Of course, philosophers have been questioning these things for a long time. But their thought experiments seem very hypothetical. The illusions are very convincing, and I don’t think you can (or should?) act like they aren’t true.

But when we can actually talk to AIs, where these things seem obviously false, the vibe will be very different. The illusion will be harder to maintain, and I think we’ll shift towards a real-world bongcloud vision where “you” are “one corner of the consciousness field” (or whatever) experiencing an illusion of selfhood and continuity because evolution programmed that into you.