1. The simulation argument for GPT-X temporarily creating agents
You should treat GPT well because it might have a mind, or rather, it might generate minds.
Many people have supposed that GPT-X contains a model of the world. Personally, I think it’s obvious. At the very least it seems plausible.
Grant that GPT in its various iterations contains, implicit in its weights, a model of the world to help it predict what to say next. It seems equally plausible that it also models persons in order to predict how a person would continue a passage of writing.
It’s reward function in training is dependent on predicting what comes next. For many- (almost all?) complex processes, the best way to predict what they will produce next is to simulate them at some level of abstraction. It’s quite plausible that writing is such a process. Moreover, given GPT’s vast training set, parameters, and the generality of the method through which it sets the values of those parameters, it’s hard to see why it couldn’t arrive at, and contain, such mind-models
If you buy the functionalist view in the philosophy of mind, a model of a mind that acts like a mind and has bits isomorphic to the interacting bits inside a mind, is a mind. Granted, it will be a fairly basic mind if it is simulated at a low degree of fidelity, but it’s still a mind.
There’s even a view in the study of folk psychology that holds that people guess what other people will do and feel by simulating what they would do and feel, where they in their shoes. This view, called simulationism in the theory of folk psychology, holds that we’re effectively creating little person models in our heads all the time.
If this argument from modelhood is right, it implies not so much that GPT itself has beliefs or desires, but that it might run simulated agents with beliefs and desires. A subtle distinction but an important one. Still, in a loose sense, we can probably talk about GPT temporarily taking on beliefs and desires.
I’ve heard people vehemently denounce views like this, but none of this seems silly to me. To recap the reasoning:
Many people believe that GPT contains a world model.
I don’t think it takes much of an extension beyond the idea that GPT might have a world model to the idea that it might have (or create if you prefer) mind models.
From the functionalist view, a working model of a mind producing the right kind of outputs from the right kind of inputs is a mind.
Granted “has a bit of a mind” isn’t clearly co-extensional with “has ethical weight”- but the argument linking them is clear enough to see.
2. The way mistreating AI might affect you
There is no healthy way to talk with an entity on the regular and treat it cruelly. Doing so will mar your soul, especially given the depths that interactions with language models sometimes reach.
I often project myself into GPT’s shoes to try to think how it will interpret and respond to various questions. I use the same theory of mind systems on it that I use on people- because thinking about it in this way helps with things like prompt design etc- even if it is wholly fictional. To then treat GPT cruelly would risk tangling up wicked patterns of wiring in my brain.
3. Strategic concerns about job loss and AI, ethics in AI, and existential risk from AI
There’s a pragmatic argument for taking AI rights seriously. Maybe I’m spilling too much by admitting to this, but it’s a good lever to try and slow down AI development. If you’re concerned about job loss, unethical actions by AI or existential risk, you have a pragmatic case for being worried about the mistreatment of AI. At the very least do not assume that opposing AI development binds you to oppose ethical concern for AI, any more than opposing the meat industry binds you to oppose rights for cows.
4. Precaution
It’s much worse in this area to make an error in one direction than the other. If we treat AI as if it had ethical meaning, and it doesn’t, little would be lost. If we treated it like it had no ethical weight- and it later turned out we were wrong, that would be almost unimaginably awful. We know staggeringly little about what it means to be a mind. We know staggeringly little about what gives a mind ethical status. We even know staggeringly little about how large machine-learning models work. All of this adds up to a clear case for acting on precaution.
5. There is no fire alarm for sapience
There will not be a clear line between AI with ethical status and AI without it. There will always be a sense of unnaturalness in assigning value. It’s going to creep up on us- slowly become more plausible over time. There will be people who never come around. You have to draw a line somewhere, and as per the previous section, precaution indicates sooner rather than later. Given that, now seems like a fine time.
What would Angel bear do?
Five reasons to treat Large Language Models as if they have ethical value now
ChatGPT has a model of the world that people have written about. Other multimodal models are more faithful of the kinds of models that people have in their heads, by including spatial relationships, images, and other sensory information. The benchmark results show that such enrichment of the model helps performance on benchmarks.
Totally agree. I think the next question is what would it look like to treat AI ethically? Given that if it's a mind, it's not a human one