Janet Pauketat

Research Fellow

This blog post was edited by Ali Ladak and Jacy Reese Anthis. The 2021 AIMS survey was designed by Janet Pauketat, Jamie Harris, Ali Ladak, and Jacy Reese Anthis. The data was collected and analyzed by Janet Pauketat, Ali Ladak, and Jacy Reese Anthis. Many thanks to Jamie Harris for his contribution to the survey design and project development and to David Moss, Zan (Alexander) Saeri, and Daniel Shank for their feedback on our methodology.

To cite the 2021 AIMS data in your research, please use: Pauketat, Janet; Ladak, Ali; Harris, Jamie; Anthis, Jacy (2022), “Artificial Intelligence, Morality, and Sentience (AIMS) 2021”, Mendeley Data, V1, doi: 10.17632/x5689yhv2n.1

Announcement of Data Publication

We are pleased to announce the publication of the data from the 2021 Artificial Intelligence, Morality, and Sentience (AIMS) survey on Mendeley Data in line with Open Science principles. The 2021 AIMS survey was preregistered on the OSF. We published the data to facilitate ongoing public access to the dataset. The publication includes a summary of the project and some initial descriptive results, the cleaned final data with a codebook, the raw data, the predictions of Sentience Institute researchers, and a supplemental results file with demographic information and basic descriptive statistics.

Below is some brief background information on the project from the data publication. A full report with the code and results from the preregistered analyses is forthcoming on our Reports page; please cite that report when referencing the results of the survey once it is available.^[1]

Background

In November and December 2021, Sentience Institute conducted a nationally representative survey of 1,232 U.S. American adults about the moral consideration, social integration, and sentience of AIs.

Objective

One purpose of the 2021 AIMS survey is to provide a baseline from which to track how the public’s opinion on this topic changes over time. Another purpose of this survey is to test the predictions of researchers and forecasters on this topic.

Methods

The survey was programmed in GuidedTrack and run online with a sample recruited by Ipsos based on census estimates from the 2019 American Community Survey. Details about the survey materials, hypotheses, and preregistered analyses are available in the preregistration. All survey materials including the GuidedTrack code are available on the Open Science Framework (OSF). The R code used to clean and census-balance the raw data and to produce and compare the descriptive statistics with Sentience Institute's predictions is on the OSF.

Results

More people are uncertain about whether artificial sentience (AS) is possible (41.53%) than believe that sentience is (34.82%) or is not (23.65%) possible. Some people think that AS already exists (18.06%). Overall, people predict a 59.97% chance that robots/AIs will be sentient in the next 100 years. Aggregate responses were within Sentience Institute’s 80% credible intervals for 69% (53/77) of the items. We overestimated 6 items and underestimated 18 items.

Limitations

Additional census-balancing (i.e., weighting the data) is recommended where possible given that some categories could not be fully census-balanced during recruitment (e.g., income, education). We increased the stopping point for sampling during recruitment to account for some unexpected issues with recruiting respondents from some categories.

Conclusions

The 2021 AIMS survey data offer empirical evidence of how humans extend moral consideration to AIs who exist now and sentient AIs who may exist in the future. The data also provide empirical evidence of social perceptions of AIs, perceived connectedness to AIs, and forecasts for future human-AI relations. These data serve to ground our expectations regarding public opinion on AIs and enable us to track how public opinion changes over time.

Metaculus Predictions

Before publishing the data we conducted an exploratory assessment of how public opinion aligns with Metaculus forecasters’ predictions of public opinion. Sentience Institute’s researchers predicted public opinion prior to data collection. Our predictions are available as part of the data publication and the preregistration.

For two weeks at the beginning of March 2022, we asked forecasters on Metaculus to predict the public’s responses to five AIMS questions to examine how forecasters’ expectations of public opinion align with actual public opinion. In general, these predictions underestimated the public’s moral and social consideration of artificial sentience and were in the same direction as Sentience Institute researchers, though the prediction format was different so it is not possible to compare directly. The general public was also less certain about the possibility of artificial sentience than Metaculus forecasters expected.

Specifically, Metaculus forecasters underestimated four of the items: (1) support for a ban on the development of sentience in AI, (2) support for legal rights for sentient AI, (3) the importance of the welfare of AI as a social issue, and (4) the perceived threat of AI to future people. Metaculus forecasters overestimated one item: the public’s belief in the possibility of artificial sentience.

Table 1 summarizes the questions, the predictions of Metaculus forecasters, and the relevant census-balanced 2021 AIMS results.^[2]

Table 1: Metaculus Predictions and 2021 AIMS Results

Note. a Forecasters on Metaculus can update their predictions during the prediction window. This means that individual forecasters can make multiple predictions and so there are more predictions than individual forecasters. b The Metaculus unweighted prediction is the median of Metaculus user predictions and is known on Metaculus as the “community prediction.” c The Metaculus weighted prediction is the Metaculus system’s weighted estimate of users’ predictions and is known on Metaculus as the “Metaculus prediction.”

Figure 1 shows the actual census-balanced responses from the nationally representative 2021 AIMS sample for respondents with an opinion.^[3]

Figure 1: Census-Balanced 2021 AIMS Results

Note. Census-balanced results from the 2021 AIMS survey for the five questions Metaculus forecasters predicted the public’s opinion on. The % agreement (“somewhat agree,” “agree,” “strongly agree”) or % “Yes” for respondents who had an opinion is in the top left of each graph. The % for each answer choice is indicated over the relevant bar.^[4]