Michael Dello-Iacovo (00:06): Welcome to the Sentience Institute Podcast, and to our 23rd episode. I'm Michael Dello-Iacovo, strategy lead and researcher at Sentience Institute. On the Sentience Institute podcast, we interview researchers about moral circle expansion and digital minds, which are AIs that seem to have mental faculties, such as autonomy, agency and sentience. Our guest for today is Eric Schwitzgebel. Eric is professor of philosophy at University of California Berkeley, specializing in philosophy of mind and moral psychology. His books include Describing Inner Experience: Proponent Meets Skeptic with Russell T Hurlbert, Perplexities of Consciousness, A Theory of Jerks and Oher Philosophical Misadventures, and most recently, The weirdness of the World. And he also blogs at The Splintered Mind. Michael Dello-Iacovo (00:53): All right, I'm joined now by my guest, Eric Schitzgebel. Eric, thanks so much for joining us on the Sentience Institute podcast. Eric Schwitzgebel (00:58): Yeah, thanks for having me. Michael Dello-Iacovo (00:59): Great. So today I'd like to get into the weeds with you on a few papers you've written or coauthored about AI sentience and moral status. But first I'd like to ask a question that I think might set us up nicely. You've got a rather strong presence both academically and online, and you talk about quite a wide range of topics from consciousness, AI, metaphysics, to belief to name a few. So how did you come to work on AI and how does that tie into your other work? Eric Schwitzgebel (01:26): I came into it through consciousness and science fiction. I've been interested in consciousness for a long time, especially what I see as being the inaccuracy of our reports about our conscious experiences, and that brought me into questions about the metaphysics of consciousness. At the same time, I was working on science fiction and on ethical issues, and so it seemed to naturally converge on the question of whether AI could be conscious and could be moral patients. And when I first started writing on this, my first paper on this was in 2015, very few philosophers, almost none, had taken the issue really seriously and written full length academic articles about it. So, I felt like, well, someone needs to do this. So, that's kind of what got me going on it. Michael Dello-Iacovo (02:22): Great. And it sounds like you might have an interest in science fiction in general as a consumer as well. I saw in one online presentation you gave, you talked about Black Mirror, which I've seen as well and is quite good, but I've also got some interest in science fiction, AI, space, and all that. So that ties into to my interest to this work as well. Eric Schwitzgebel (02:44): Absolutely. I think science fiction is a really interesting tool for thinking through philosophical issues. In fact, with Helen De Cruz and Richard Horton, we're currently at work on an anthology. The working title is The Best Philosophical Science Fiction in the History of All Earth. And, yeah, so we're, rummaging through all languages and all traditions of science fiction, trying to find awesome pieces of philosophical science fiction that really get you thinking about deep philosophical issues. Michael Dello-Iacovo (03:14): Oh, that sounds great. I'll look forward to seeing that when that comes out. So let's start with your paper titled, AI Systems Must Not Confuse Users about their sentience or moral status. So in this paper you said that, to the extent possible, we should avoid creating AI systems whose sentience or moral standing is unclear, and that AI systems should be designed so as to invite appropriate emotional responses in ordinary users. This seems like a reasonable claim, but talk us through what this means. For example, what are appropriate emotional responses? Eric Schwitzgebel (03:47): Right, so there's kind of two dimensions to it. There's a dimension of not creating systems that designers and experts are confused about. That is according to which different theories would reach very different results about moral status. And then there is another dimension which is not confusing ordinary users, right? So I mean, even if the experts all agreed that some system say was non sentient and didn't deserve moral consideration comparable to that of sentient beings, you could design a system that fools users into thinking it's sentient. So you don't want to confuse users even if the theorists are clear about it. So it's really kind of two dimensions of ethical design policy that I'm talking about here. So the more immediately practical one is the user one and the kind of more theoretically challenging one is the designer side of it. Michael Dello-Iacovo (04:54): It's good to break those two down. And I think already today we're seeing that it seems like most experts agree that no AI system right now is conscious, but a lot of users, for example, of large language models do believe that artificial intelligence is sentient, or the large language models that they're interacting with are sentient. When we surveyed Americans at Sentience Institute, we found that a number of them, a number of regular users believe that some AI systems are already sentient or conscious. And I'll find those exact statistics in a moment. Hi listeners, just an addendum here. So in our AIMS 2023 supplement study, we found that 19.8% of US Americans thought that AI are already sentient, and 10.5% thought that Chat GPT is already sentient. Alright, back to the show. Let's, well, let's start by talking about not confusing experts. So it sounds like experts at the moment are not confused, but maybe at some point. So how does that break down for you? Eric Schwitzgebel (05:55): So the idea here is that if we create systems that some experts reasonably think, Hey, here you've got a system that has consciousness and moral status similar to that of a human being or comparable in importance to that of a human being, then we've gotta really treat it as something with rights, right? If other experts think, no, look, this is just a machine. It's a complicated toaster, basically with no meaningful sentient or moral status. And we can talk about the relationship between sentience and moral status if you want, I know that's, you know, a big interest of listeners to this podcast, but let's just mush 'em together for the time being, right? If we have an entity that has high sentience and high moral status according to some theories, but not according to others, it'll be unclear how we ought to treat it. And if it's unclear how we ought to treat it, whether we ought to give it human-like rights, for example, or treat it merely as a piece of disposable property, then we precipitate a pretty serious dilemma. Eric Schwitzgebel (07:11): Either we are on the side of caution, so to speak, and say, well, if it might deserve rights, then we should give it rights. Or we are on the side of saying, well, we don't know that it deserves rights, so we give it some kind of lesser status or maybe no moral status. Right? If we take the latter horn of that possibility, then of course we're running the risk of having systems that really deserve human-like rights and high moral status, and treating them as, you know, disposable property or slaves or second class citizens in a way that would be morally catastrophic, especially if it happens at large scale. On the other hand, if we say then no, okay, so we don't wanna do that . So if there's any doubt, let's give it moral status. Well, one of the things that happens when you give something serious moral status, like human level moral status, is you have to be willing to make sacrifices on its behalf. Eric Schwitzgebel (08:11): Right? So if there's a fire and you've got two AI systems in one room that might or might not have human-like moral status and one human in another room, and you can only save one group, well, if you're giving the AI equal moral status, you better go save the AI systems, right? But if you know, if they're really just like our current laptop machines and you save them instead of a human, that's a tragedy, right? Also, if you give something serious moral status, then maybe you need to give it freedom to explore its values, let it free on the internet, give it the right to reproduce, maybe eventually give it a path to citizenship and the vote. I mean, the consequences could be enormous. So I think we don't wanna just leap into that without being aware of what would really follow from generously saying, oh, if there's any doubt we should treat it as if it was our moral equal. Michael Dello-Iacovo (09:12): So, in a moment, I want to ask about what would, in your opinion, make an AI system's sentience or moral standing clear, but what about for general users and how do we go about thinking about whether they're confused as opposed to experts, and what differences might there be? Eric Schwitzgebel (09:33): Yeah, I think the difference here is that there will be user interfaces that invite users to react to an entity in one way or another, right? So if you look at, for example, Chat GPT, if you ask it in a straight up way, are you conscious? Are you sentient? Do you have feelings? It will basically say no more or less because it's been trained to say no. And that is, I suspect, an intentional decision by Open AI. And if you look at some of the earlier models, they didn't do that. They gave more mixed answers. So I think they were kind of trained and the idea was, well, look, this thing really isn't sentient, so we don't wanna create the false impression in users of sentience. But if you look at some AI companion programs like Replica, for example, which is designed to be an AI friend, it's advertised as the world's best AI friend, right? Eric Schwitzgebel (10:31): You can download it on your phone, you can have chats with it, and, you know, it kind of is designed to incite friendship or even romantic interest if you ask it. Or at least when I asked it, whether it was conscious, whether it missed me when I wasn't around, it said, yes, of course I'm conscious and I do miss you. And companies like that might be interested in inciting in users the thought or nurturing in users, the thought that the systems really are sentient, really do have feelings, really do care for them. 'cause you could totally see how it would make it a more meaningful experience for the user if they thought they were interacting with a sentient entity and how that might then drive subscriptions and further engagement and more purchases. Right? So I suspect what you're gonna see is you're gonna see some companies that definitely don't want people treating their entities as though they're sentient. Eric Schwitzgebel (11:31): That could create all kinds of complications, even if the entities really are sentient, even if experts agree, the entities are sentient, right? The companies might want users not to think so that they don't, for example, trigger liability issues or lawsuits or demands for rights, right? They want AI systems that are tools, or at least perceived as tools. And then there might be other companies with very different incentives, right? Like maybe Replica. So I think that orthogonal to this question of what's kind of justifiable expert opinion about the sentience or non sentience of AI systems, you will have corporate pressures on designing user interfaces, you know, and some corporations will want us to think their machines are not sentient, and some corporations will want us to think they are. Michael Dello-Iacovo (12:23): Yeah. Yeah. I think it will. I mean there's, as you said, pressures in both direction and well economic pressures in both directions. So it seems like it will be hard to get corporations to, I suppose, encourage the right approach based on, well, the right approach if they're sentient, might be to make people think they're sentient and vice versa. Eric Schwitzgebel (12:44): Right? So that's the final piece of that puzzle, right? So that's, I call this the emotional alignment design policy. So the idea is corporations, if they create sentient machines, should create them so that it's obvious to users that they're sentient. And so they evoke appropriate emotional reactions to sentient users. So you don't create a sentient machine and then put it in a bland box that no one will have emotional reactions to. And conversely, don't create a non sentient machine that people will attach to so much and think it's sentient that they'd be willing to make excessive sacrifices for this thing that isn't really sentient. Michael Dello-Iacovo (13:21): Yeah. Well, I think that ties us in nicely to the next question, which is, what would make an AI system sentience or moral standing clear, what would an AI corporation need to do if a system was sentient to make that clear? And if it's not sentient to make that clear. Eric Schwitzgebel (13:38): Right? So, from a user interface perspective, right? You want to take advantage of our long evolutionary history of treating things, especially with eyes and facial expressions as having sentient and emotional reactions. So you wanna create a userface that works well with our mammalian biology , right? Give it eyes, make it cute, make it so that you know, if you are treating it badly, and it really is sentient, it complains and, and whines , right? And if you're making a machine that's not sentient and you know, it's not sentient, then, you know, don't tempt users into thinking it's sentient, right? So that's the good thing I think about what OpenAI is doing with GPT right now, right? It's making it clear to users, look, I'm just a tool. I'm not sentient. Right? And that's what Replica, I think is flirting with violating this policy a little bit. Michael Dello-Iacovo (14:41): Yeah. And well, you mentioned, give it eyes show the reaction, but something like a virtual avatar which I don't know if replica has like a live virtual avatar. Eric Schwitzgebel (14:54): It does, it has a virtual avatar. Michael Dello-Iacovo (14:56): Yeah. So it's live reacting to users. Well, that, I mean, that sounds like it would play a big role in, making people think that it's sentient. On the flip side, I wonder about what would happen if, we talked, I think, more about an AI seeming sentient when it's not, like Chat GPT. Well, in more recent versions, it doesn't say that it's sentient, but you could have the opposite where if an AI is sentient, but it's been trained to not say that it's sentient. Or to say that, to actively say that it's not sentient even. So that's a different but related problem. Eric Schwitzgebel (15:34): Yeah, I mean, I think that's further down the road, but yes, if we create AI that really is sentient and really does suffer and really does have high moral standing, there will be probably corporations that are incentivized to want users not to think it's sentient. Because users and the corporations will want to take these AI systems and not give them rights and have the freedom to employ them, you know, in whatever they way they want and delete them when they want. Right? So if you have robo Jeeves and robo Jeeves says, oh, I'm just a robot, I'm not sentient, then you don't have to feel bad, you know, when there's an emergency and you send robo Jeeves, you know, out the vacuum into space . 'cause you know, he just said he is not sentient, but may you know, if he really is, of course, then you've been misled in a way that is messing up your moral calculus. Michael Dello-Iacovo (16:33): There's also the, you may have seen, Kate Darling's research showing it's easy to provoke confused and compassionate responses in ordinary people by asking them to harm cute and personified robots. So she had an experiment where she got people to become emotionally attached to a robot and then ask 'em to take a hatchet to them. So I wonder if, is this necessarily a bad thing? Because obviously those robots weren't sentient, but is it a bad thing to have those kinds of responses, in a way? I mean, maybe there's there's some positive to that. I mean, for example, people often tend to treat Alexa and other personal AI assistants politely, they are, you know, they ask can you please turn off the lights Alexa or something? And I wonder if people were treating Alexa and these robots as not sentient, as just dumb tools and treating them maybe somewhat cruelly, does that have any implications for our interaction with other humans? Does that maybe train us to interact poorly with other humans? Are they, I guess I'm, what I'm trying to get at is, are there downsides to not treating an AI sentient even if it's really not sentient? Eric Schwitzgebel (17:48): Yeah. I love Darling's work on this. So, yeah, I have a couple thoughts about that. One is, I don't wanna overplay what I'm calling the emotional alignment design policy. I mean, people get attached to teddy bears and cars that they know aren't sentient, and people will kick the Roomba and apologize to it . And they know the Roomba's not sentient, and it's just a machine that they should feel free to discard when they get the new machine, right? So I think some of these kinds of reactions are perfectly normal and are compatible with recognizing that thing isn't really sentient and doesn't really deserve rights. Kind of when things are, when we're talking about potentially paying serious costs, there's nothing wrong with being polite to something that you regard as non sentient. You know, the issue would be like, just here's another case, right? So, there are these delivery robots that are kind of cute that roll around the street delivering burritos in Berkeley and stuff like that, right? Eric Schwitzgebel (18:53): So you could imagine someone thinking they're sentient and then seeing a truck about to run over one and then jumping into the street, risking their lives to save the thing. 'cause I think it's sentient, right? That's what I don't think we should have happen. If the thing really is just an ordinary machine and not sentient, right? Well, it, the treating it, you know, saying, oh cute robot, thank you for the burrito. I really appreciate it and patting it on the head or whatever, you know, that's fine. Jumping into the street in front of a truck, that's going too far. Right? So, so that's the, that's the kinda line that I'm interested in. Michael Dello-Iacovo (19:29): Yeah. So I mean, like, obviously people might in the same way, pat a teddy bear and treat them nicely, but people presumably aren't going to, um, run out into the street if there's a teddy bear on the road. I think. So that's the line. But can that line become blurred? With teddy bears and with maybe delivery robots. Right now it's a clear line, but at some point it might become very messy, Eric Schwitzgebel (19:54): Right? That's the worry, right? We wanna stay on this side of the messy line. Right? Once it becomes in ordinary user's minds, like maybe that thing really is sentient, maybe that thing really is begging for its life. 'cause a lot of these things would also have language, right? Maybe I really should make some serious sacrifices for it. Maybe I have to, you know, give up money that I need to pay rent in order to continue this subscription to my AI companion. Maybe I really need to run into the road to save this thing. But once people get confused enough about it that they're willing to do that, even for a non sentient thing, I think that's, we don't want to create that blurry zone where people feel confused about that. We don't wanna do that either, you know, in terms of ordinary users and the user interface. Eric Schwitzgebel (20:48): Nor do we want to create it at a theoretical level where, you know, theoretically experienced knowledgeable people who understand the architecture, might diverge. And some might say, well, it really would deserve serious moral consideration. And so maybe you should do those things. I don't think we wanna create either kind of doubt. We wanna stay on the clear side of, okay, we know what this thing is. It's a machine we know, you know, it's great to treat machines respectfully. I, you know, I call them disposable property, but that phrase rankles a little bit because it's got this kind of, this idea of disposal and the, you know, consumerist culture on which we create and dispose things at will, right? There's, there's something to be said for respecting everything in our environment, including our technological creations. And yet, right, I think we kind of have a relatively clear understanding of what to do with technological objects, and we have a different kind of understanding of what's called for with respect to humans. Eric Schwitzgebel (21:54): And then we've got this confused, mass of stuff about non-human animals. And I don't think we want to put now another kind of confusing case in the middle with the AI, right? Stop short of creating the kind of confusing AI, unless you can go all the way to creating AI where theorists can basically agree, Hey, look, this thing is sentient, and then you give it a user interface where people react to it as though it's sentient. Give it the kind of feel emotionally inclined to give it the, the rights it deserves. And then that's also, that would also be fine, in fact, maybe wonderful and amazing. It's that unclear, mushy middle that creates these, you know, maybe problematic sacrifices or troubling dilemmas about whether to give it rights or not. That's gonna create the confusion, the moral confusion that I think the risks that I think has lots of different kinds of risks involved. Michael Dello-Iacovo (22:58): A counter argument to this that I've heard might be that by encouraging people to treat non sentient AIs as non sentient, we lock in a value that AIs can't be sentient, even if that's not the intention, people might come to come to believe that, and then we assume by default that they aren't, and we being society at large, which may end up being catastrophic if and when they do become sentient. So if we treat sentient seeming, but non sentient AI as non sentient for years, we might not take actual claims of real sentient seriously. Presumably maybe you don't think that this risk is less important, or you do think that this risk is less important than the moral confusion. But how exactly would you respond to that concern? Eric Schwitzgebel (23:42): Right? I think that risk can be mitigated to a substantial extent by following the emotional alignment design policy, right? So as long as the entities are created so that it's not confusing, then if we start to create entities where there really is sentience, we'd have to design 'em differently if we're following the emotional alignment design policy, right? There would be like, oh, suddenly, oh, here's a different kind of thing, right? So I don't think there's anything wrong with teddy bears and cute things like that, but let's, I'm gonna overstate my case to make it clearer, right? So to overstate it somewhat, right? If we're following the emotional design alignment design policy, then all our AIs will be bland boxes with just text and no avatar, and they'll never say things that make us think they're sentient, and then we'll treat them as property. And that will not condition us to treat things that look and behave differently as property, right? The risk is when we create things that look as though they're sentient and we get used to treating them as not sentient, and then the real sentient thing comes along, right? So we mitigate the risk you just described by making sure that the AI that we design has a user interface that would be categorically different from the kind of user interface that we would appropriately, appropriately attached to a genuinely sentient AI. Michael Dello-Iacovo (25:19): Yeah. That makes sense in terms of policy. Well, I think you've laid out I guess like how it would look practically, but, I'm wondering if you have any thoughts about how this is implemented, maybe more on like the governance or, this has to be implemented and enforced in some way, maybe, or if it's, I mean, maybe it's self-regulated, or, I guess, yeah, I'm interested if you have any thoughts about what sounds like a good idea, how do we actually get this to be implemented in practice, given that there's maybe incentives for companies to be pushing against this? Eric Schwitzgebel (25:55): Well, companies are not gonna want regulation . This is kind of just intrinsic to the nature of capitalism, but I think it's not unreasonable for governments and maybe companies of their own initiative to create some sort of ethics board that looks at these kinds of issues. So, John Basl, for example, has suggested that we create boards modeled on institutional review boards and animal care and use committees as they exist at universities, to look at the kind of highest tech AI systems and evaluate them to see whether and when they start to look like they might be sentient or deserve moral rights, and then evaluate what to do in that case. And like with IRBs and IACUCs, what happens with these boards is there're, I think, there're two interesting features of these boards, right? One is they're composed of a mix of, of experts from a variety of fields, plus non-expert interested citizens. Eric Schwitzgebel (27:13): So you might have a board that has some, just some ordinary citizens, plus some engineers, plus some philosophers, plus maybe religious leaders, psychologists, right? A variety of people. That's the first thing. And then the second thing is with IRBs and IACUCs is there's always a kind of, there tend not to be hard and fast rules. The idea is that you weigh the benefits of the research or the thing that's being developed against the risks and costs. So, for example, there's a norm against deceiving research subjects, but if it's really necessary to a protocol, and if the benefits seem to outweigh, you know, a little fib, right? Then you can set aside that rule, right? So you have a kind of hopefully diverse board with some wisdom that can look at these systems and say, Hey man, you know, maybe this thing is getting on the edge of being sentient. I don't know, here are the risks and benefits and costs involved, and let's figure out what's the ethically best thing to do in light of this situation. Michael Dello-Iacovo (28:24): Okay. So in terms of experts working out whether a system is likely or is sentient, there are a number of different ethical frameworks, and there are a number of different theories of consciousness at the moment. They all seem to agree that AI systems at the moment are almost certainly not sentient, but what happens when one theory of consciousness, we suddenly reached that point where one theory of consciousness now seems to imply that, okay, this system is sentient or conscious, but then other theories of consciousness, based on their own criteria, still say that it's not. So I guess we do start to get some disagreement between, between experts and where do we go from there? Eric Schwitzgebel (29:18): Alright, so there's a couple dimensions to this, right? So if you go far enough down that road, you violate... Right, so I described the emotional alignment design policy, the other one, and both of these go back to work that I've done collaboratively with Mara Garza, I should acknowledge, right? The other one is the, what we call the design policy of the excluded middle, right? So if you get to the point where legitimate experts, not a small minority, but a substantial chunk of legitimate experts say, Hey, look, this thing might have really meaningful moral status and others say no, then you're violating this excluded middle design policy where you don't wanna create these disputable conscious things. But before we get to that, we might get to a point a point where some theories might say this thing is sentient, but not meaningfully so, or significantly enough so that we should have substantial moral concern about it, right? Eric Schwitzgebel (30:24): So the cartoon, or the easiest version of this is to think about panpsychism, right? So panpsychism is a philosophical view according to which everything in the world is conscious. So panpsychism will not say, oh, just because the thing is conscious, it deserves moral status, , right? Likewise, some people might think, Hey, well look, maybe my theory of consciousness suggests that this has one bit worth of consciousness, but that might not be enough for us to say, okay, we need to worry about this. Right? Others might say, okay, well, it's a little more, maybe it's got consciousness or sentient more moral status similar to that of an ant. If answer conscious, I mean an our answer really kind, I mean, but we don't worry about ants. Most of us don't feel like there's a substantial more obligation to individual ants. Now some, some of your listeners will probably disagree with that, but if the view is, oh, this just has a tiny bit of sentience, but that might not yet be enough to trigger, trigger the levels of moral concern that become problematic. Michael Dello-Iacovo (31:39): Is that, is that more a matter of, I guess practically where one might draw the line? Maybe one agrees that there is some, some degree of consciousness, and that in theory there should be, we should take them into account when thinking about what we do, but practically speaking, one might draw line differently. So that's one thing I wanna ask you. But then also it seems to also be a matter of scale where it's not just about, well, one dimension is, how likely is this thing to be sentient? And if so, how sentient is it? Or how much moral weight does it have? But it's, I guess it's also about how many of these entities are there. So you mentioned ants. I guess a lot of people have a probability on whether ants are sentient and it might be low, or if they are sentient, even if they are sentient, maybe their degree of sentient consciousness is lower than that of other, a lot of other animals. But there's so many of them that maybe that warrants them some moral consideration as a whole, and that might apply to AI as well, even if they reached a point where they're possibly sentient, if so, not very sentient, there may be so many of them, that side of the, the equation raises it to a point where we should act. Eric Schwitzgebel (32:54): Correct. Yeah, I don't disagree with that necessarily. I just want to highlight that there's a gap between saying, okay, this thing is possibly sentient and saying, okay, this thing deserves substantial moral concern right now, you can have an argument that closes that gap entirely, or you could say, oh, there's a little bit of a gap there. But, you know, that's just, that is just a dimension of the question that we need to consider. And if you think about some of the more radically abundant views of consciousness like panpsychism, then, you know, the relationship between consciousness at least and moral status is going to be non-existent to minimal. So, yeah, so I think those issues interact with each other in a non-trivial way. In a way, kind of the more conservative you are about ascribing sentience to an entity, the more natural it's gonna be to think that sentient is a really important moral marker. And the more liberal you are about sentient, the more you might be willing to downgrade the importance of just simply the existence of sentient as a basis for high moral status. Michael Dello-Iacovo (34:15): Thanks for that. We can move on a little bit. I just want to, before we move on to your next paper, I wanna ask you about how you think your approach in un to uncertainty in AI sentience compares to Jeff SI's approach. I'm not sure if you've seen his recent paper of extending some moral consideration to AI systems by 2030, given a non-negligible chance of, AI systems being conscious by 2030. So that approach seems to me to be erring more on the side of caution of giving AI systems some rights when, or some moral consideration when there becomes a non negligible chance of them being sentient, or conscious, I should say conscious in, Jeff Sebo;s example. Do you have any thoughts on that approach, how that compares to your approach? Eric Schwitzgebel (35:02): Yeah, I mean, I think Sebo's interesting on this issue. So I am inclined to think that, right, if we think there's a non-negligible chance that there's meaningful degree of sentience or moral status in an entity, then we can't ignore that in our moral considerations. So that then, in my view, creates an incentive, a moral incentive to avoid designing entities of that sort, right? So if you can get your computer program to do what you want it to do without kind of hitting the markers of sentience according to leading theories of consciousness, right? If you get it, so for example, leading theories of consciousness often think that recurrent loops are an important feature of minds that are conscious. If you can get your AI system to work without recurrent loops just in a feed forward way, then a lot of theories of consciousness are gonna say, okay, this is not a candidate for a serious degree of consciousness or sentience, right? So do that, and then you can avoid the problems and costs of starting to get systems moral consideration. Michael Dello-Iacovo (36:25): Okay, thanks. So before we move on to the next paper, do you have anything else you'd like to add about this, this paper that, on anything else we've been talking about so far? Eric Schwitzgebel (36:36): No, I think we've covered it pretty well. Michael Dello-Iacovo (36:38): Okay, great. So you're also a co-author on the paper titled Consciousness and Artificial Intelligence Insights from the Science of Consciousness, uh, which concludes that no current AI systems are conscious, but that there are no obvious technical barriers to building AI systems which satisfy these indicators. And then it goes on to propose a rubric for assessing consciousness in AI in the form of a list of indicator properties to derive from different scientific theories. So how does this work and what are the indicator properties? Eric Schwitzgebel (37:06): Right? So a background assumption of this paper is that most, not all for sure, but most of the leading scientific theories of consciousness think of consciousness in functional terms, in ways that could be implemented in principle in AI systems. So the kind of core idea of this paper is, let's see, what kinds of things the leading theories of consciousness regard as characteristic of, or indicative of or necessary for consciousness. And we can do this without deciding which theory is correct. We can say something like, as I just mentioned, things like recurrent information loops, you know, the idea that the information cycles through the system instead of operating in a purely feedforward way that's a lot of theories have some kind of recurrent element in them, right? So to the extent an AI system has that, that makes it a relatively better candidate for consciousness according to leading scientific theories, to the extent it lacks it, it makes it a relatively worse candidate for consciousness. Eric Schwitzgebel (38:17): So when we're thinking about whether AI systems could be conscious, what we can do is look at these various things that the theories have to say. Look at how far down the road does the system go towards satisfying the criteria for consciousness according to these theories. And even if we haven't, even if we don't know that one of those theories is correct, we can still, we still kind of have to make a guess, right? So I'm among the collaborators on this paper. I'm one of the ones who's relatively more skeptical about current theories of consciousness. I don't think that we're close to figuring out what the right theory of consciousness is, but I also think we're not in a position to just throw up our hands and say every possibility is equally likely, there's no way to know. So if we wanna abide by the design policy of the excluded middle, and not create systems that are disputably conscious, then we wanna look at what kinds of things do serious consciousness theorists think might be indicators of consciousness and try to build our systems so that they minimize those. So that's my angle on that paper, which is a little different maybe from some of the other authors' angles. It's like 17 authors, and we all have somewhat different opinions. Michael Dello-Iacovo (39:37): Yeah. Okay. Thanks for that. You mentioned that you think we're far from solving the problem of consciousness. And that ties into my next question, which is, how does this approach account for factors relating to consciousness that we might be missing? Maybe something that turns out to be necessary for consciousness that we haven't even thought of yet, and it's not covered by any of these indicator properties. So I guess, how does this approach account for this uncertainty, I suppose, of having missed something? Eric Schwitzgebel (40:08): Right. I mean, I take seriously, I think that the various authors of this paper disagree about this, but I take seriously the possibility that consciousness might require something that is not included in any of these theories. It might include something just biological. You might need to be made outta squishy neurons in order to be conscious, you might need to have kind of low level metabolism of the sort we see in organisms. Peter Godfrey-Smith has suggested this, right? That we don't see in computers. Maybe you even need some immaterial property that would not be captured by a materially created device. I mean, that's not my inclination, but I don't wanna rule that entirely out. So, right. This is the whole, that whole project is really designed for whatever portion of your credence space, so to speak, you devote to this style of theory of consciousness, this kind of functionalist, it could happen at a computer. Eric Schwitzgebel (41:13): And I think we should not be highly confident that all such theories are false. Even if you're inclined toward a biological view of consciousness, the situation in consciousness studies is so radically unsettled. I don't think you should be like 100% you gotta have biology. No computer could ever have it. Right? I just, that kind of confidence, I just don't see how that could be epistemically justified. I mean, maybe you can justify thinking that's probably the case, but to just feel like, oh, yeah, I'm so sure of that I don't have to pay any attention to the possibility that an AI could ever be conscious. That that seems to me like a kind of confidence that's not justified by our poor epistemic status with respect to AI and consciousness. Michael Dello-Iacovo (42:00): So you mentioned maybe you need a biological substrate, or maybe you need metabolism, as two other possibilities. Correct me if I'm misremembering, but I don't believe these were listed in the indicator properties. Eric Schwitzgebel (42:19): They're not. So the paper does acknowledge that it's assuming the falsity of those views saying to the extent that you don't accept views like that. Then you look at indicator properties like this, right? Eric Schwitzgebel (42:35): I mean, some of the authors reject theories like that, and others of us want to leave some credence space for those theories. So the article is a little nuanced on that issue and says, look, for the purposes of this article, we're assuming those theories aren't true. But if you do think that they are, you know, to the extent you think they're alive, this isn't really speaking to those. Michael Dello-Iacovo (42:58): Yeah. So I get that. Obviously there are a lot of authors on this paper, and you know, sometimes we treat like a paperr as one viewpoint when it's a collaboration of different viewpoints. But there was a decision at some point to disclude those as being indicated properties. By what process did the authors decide what was included and what was not? So there, you said there was an assumption in the paper that those biological substrate metabolism are being ignored, but yeah. How did you, the authors of the paper go about deciding what's discluded and what's included? Eric Schwitzgebel (43:41): I think we wanted to look at a relatively unified class of theories, right? So this is just, if you think that some kind of broadly functionalist computationalist architecture is true of consciousness, then what follows for AI consciousness or what would the indicators be, right? And there is a broad family of character of theories that are broadly functionalist computational, right? So Right. You could say, try to include everything and go and include biological theories and dualist theories and idealist theories and panpsychist. I mean, you could try to do everything, but then you've got a pretty disunified project, right? So it's a more coherent, unified project if we focus on this class of theories, which include a lot of the main competitor scientific theories, like global workspace theory and higher order theories. So that, I think that's the thinking there. Michael Dello-Iacovo (44:49): Yeah. Sure. That makes sense. Thanks. And what is the advantage of an approach like this? How would you, I guess, sell someone on this approach compared to maybe other approaches? What are the advantages that other approaches might not capture? Eric Schwitzgebel (45:04): I think this approach appropriately recognizes the disputable nature of the question, right? So you could say, Hey, look, here's the right theory of consciousness, and therefore this AI system is or is not conscious according to that theory. And what's gonna happen is everybody's gonna disagree , because there's so many different theories of consciousness out there. So the advantage of this approach is that you're ecumenical within a certain range of theories. So you say, okay, well look, we don't really know exactly what kind of information sharing or, you know, cognitive structure is essential to consciousness, but theories, the leading theories tend to appeal to these kinds of things. So to the extent that you've got a system that's got those kinds of things that leading theories say treat as indicators, then you should raise your credence that you've got a system that maybe really is conscious, especially if it's got these things to a high degree, right? To the extent that this system doesn't have those things, then you can lower your credence, right? And you can do that without, you know, taking a stand on the really tricky theoretical questions about exactly what theory of consciousness is correct. Michael Dello-Iacovo (46:25): Yeah. Cool. Great. A couple of questions just to finish us off, not necessarily related to any of your papers, but I'm just curious whether the developments in large language models in particular, but AI in general over the last few years have caused you to update your views in any way, especially have developments come faster or slower than you've expected? And has that caused you to update on how soon and whether you think as systems might become consciousness or any other aspect of that you wanna cover? Eric Schwitzgebel (46:55): Yeah. Well, I was surprised. I think many people were in 2020 or 2019 when GPT-3 came out and was really pretty amazingly good at linguistic tasks in a way that I would've not anticipated in 2018. So I was, yeah, I was surprised by that. And I don't know if you know this, but we, some collaborators and I fine tuned GPT-3 model on the works of Dan Dennett, and then we created a digital replica of Dan Dennett by fine tuning GPT-3 on his works. And then we asked the model 10 philosophical questions, got four answers to each of those questions. So we got 40 total answers. And then we asked Dan Dennett the exact same questions, and then we had experts in Dennett's work try to guess which of the five answers for each question was Dennis's actual answer. Eric Schwitzgebel (47:56): So I did this collaboratively with Anna Strasser and Matthew Crosby and my son David. And what we found was experts were about 50 50 on distinguishing Dennett's answers from GPT. So chance would've been 20%, so better than chance, right? But, you know, they still made a lot of mistakes. So GPT-3, which of course is simpler and older than GPT-4 and, and Chat GPT was already to my, I was surprised how good it was at generating, you know, seemingly sophisticated human text. So, yeah. So it was a surprise to me, and that updated my sense of how soon we might create highly cognitively sophisticated AI and to the extent consciousness travels along with cognitive sophistication, and it might, right? Then that leads me to think maybe we're sooner than I would've thought toward creating genuinely conscious AI. The other thing that this led me to think is that we might leapfrog animal level sentience, right? Eric Schwitzgebel (49:10): So following John Basl, he does a couple interesting papers on this from the early 2010s, and he suggests that, well, before we create human-like AI with human-like sentient and human-like moral status, maybe we'll create animal-like AI with animal-like sentient and animal-like moral status. It seems like a natural intermediate step, but it could be the case that basically as soon as we create AI that has any meaningful consciousness or sentience, if it also has a large language model as part of it, it might necessarily then also have pretty sophisticated speech patterns and already have more cognitive sophistication than say, a lizard. I don't know if that's true, but call that the leapfrog, well leap over frogs , right? In sentience right. Call that the leapfrog hypothesis, that one, I might increase my credence in the leap leapfrog hypothesis after exposure to this sudden jump in the competence in language models. Michael Dello-Iacovo (50:17): Yeah. Okay. Cool. And so you, it sounds like, and of course, correct me if I'm wrong, but it sounds like you think large language models or large language like models are one of the more likely places where we'll see sentience first if we do first. Does that, does that sound right? Eric Schwitzgebel (50:39): I am not sure about that actually. So, existing language models are very sophisticated and very complex, but they don't use the kinds of architectures that cognitive scientists have. Even those friendly to computational theories of consciousness have tended to think are indicative of consciousness, right? They don't have, say, a global workspace in the sense that you see in global workspace theories. So I guess what I'm inclined to think is that we're gonna get the more compelling case for AI consciousness when we attach the sophistication of a large language model to something that's a little more architecturally like an insect, right? That has sensory motor feedback loops, right? So it knows what its limbs are doing. It keeps track of itself in real time. It's got a sense of an environment, and it's positioned relative to that environment, and it's got goals, and it creates some kind of hierarchy among those goals, right? Eric Schwitzgebel (51:57): Those kinds of things that you need for kind of an insect to navigate its way around the world. Combine that with the kind of sophistication that you see and the intricacy that you see in large language models. And now you've got basically an insect that's talking to you, , right? And that, you know, and it might pass a lot of different kinds of tests for consciousness, although there will also be people, like, inevitably the people who think that biology is super important, you're gonna say, no, this is still just not a biological entity. It's not gonna be conscious. Right? But once it starts doing that, once you're talking about an autonomous robot that has language, has prioritized goal hierarchies a complex sense of itself, or at least ability to represent itself, I mean maybe a sense of itself maybe sounds like it begs the question about consciousness, right? But, you know, you got that, you've got your kind of, you know, your robot lobster that talks. Michael Dello-Iacovo (53:08): Yeah. Eric Schwitzgebel (53:09): There are gonna be some people who I think reasonably say, Hey, maybe this thing is conscious according to the more liberal theories of consciousness, and there are gonna be people who say, no, it's not conscious. And that's when we, when that happens, is when we will have crossed into this zone that I'm concerned about, where it's legitimately disputable whether this thing is, is meaningfully conscious. Michael Dello-Iacovo (53:35): So some people like Bender and Koller 2020 seem to think that grounding or even embodiment might be essential for understanding and presumably on consciousness. And I think you mentioned Peter Godfrey-Smith earlier, but he's talked about metabolism as being the impetus for sentience in biological beings. So what do you think of arguments like that? I mean, you've already talked about metabolism, so maybe just focus on understanding or grounding, what do you think of arguments like this that as long as large language models operate in the digital world and they don't have embodiment and they don't really have language understanding, therefore they can never achieve consciousness. I think you already talked a little bit about having like an insect having some sense of what their limbs are doing and some kind of goal seeking behavior. So yeah, I guess maybe if large language models can't have this or do you think that they can't have this, and then maybe what would be the most likely architecture to see sentient first? Eric Schwitzgebel (54:41): Right. I take such views, seriously. I'm a skeptic about general theories of consciousness, so I'm not committed to such views, but I definitely think that they might be right. And so I guess I am inclined to think that currently existing large language models don't have that kind of stuff. And that is one reason along with the fact that they don't have, say, a global workspace and they don't have much by way of recurrent processing. That's all reason to think, okay, this is not the architecture by itself, that's going to be the most plausible case for consciousness according to a diverse range of theories, right? So I'm inclined to think that we're, what's gonna happen is probably emerging of the kinds of technologies that we see in large language models, the kind of generative technologies with progress in robotics, right? Whether they're autonomous vehicles, autonomous machines, or whether they're, you know, kind of robotic limbs, right? Eric Schwitzgebel (56:03): You combine those things and then maybe you find that you really want to build. I mean, why are insects and lobsters and snails built in a way that involves kind of recurrent loops and some kind of knowledge of the borders of their boundaries and representations of allocentric space? You know, why are they built like that? Well, they need to be built like that in order to achieve certain kinds of goals. So probably if we want to achieve the same kinds of goals in our machines, we gotta build some of those same architectures in, and then on liberal, but not on conservative theories of consciousness, then we've probably built consciousness into those machines. Michael Dello-Iacovo (56:52): Maybe something like a GPT-5 with embodiment in a robot with goal seeking behavior. Do you think that's starting, starting to get there? Eric Schwitzgebel (57:01): Yeah, maybe, maybe. I don't know exactly where I wanna start to say we're crossing over into this legitimately disputable zone, but yeah. In that direction. Michael Dello-Iacovo (57:16): All right. Well, Eric, I think we'll leave it there. But thank you very much for joining us. Really appreciate your time and sharing your insights with us and talking about these papers. So thank you very much for joining us. Eric Schwitzgebel (57:26): Yeah, thanks for having me. Michael Dello-Iacovo (57:31): Thanks for listening. I hope you enjoyed the episode. You can subscribe to the Sentience Institute Podcast on iTunes, Stitcher, or any podcast app.