World War AI, part 1 of 3. Part 1: Will AI End Humanity? · Part 2: It's Already Started · Part 3: The Defense Rests
War has been declared. Not by any government, and not by any robot, but by roughly everyone online: it is humanity versus the machines, and humanity is supposedly losing. Scroll for five minutes and you will find somebody certain that AI is going to end us all. I was today days old when I realized I could skip the middlemen and put the question to the alleged enemy. So I asked Claude.
First, what the people who build this stuff say
Before interrogating the suspect, I went looking for an actual number. The biggest one I found is the AI Impacts survey (2023 edition, published January 2024), which polled 2,778 researchers who had published at top AI venues. The scannable version:
- 5%: the median respondent's chance that AI leads to human extinction or the permanent, severe disempowerment of humanity.
- 16%: the mean answer to the same question. Some respondents went a lot higher than the median.
- 48%: the share of researchers who think good outcomes are more likely, yet still gave at least 5% to extremely bad ones.
- 2047: the year by which the aggregate forecast gives a 50% chance that AI outperforms humans at every task.
One caveat comes straight from the survey's authors: they call forecasts like these unreliable educated guesses. So this is not "experts say we are doomed," and it is not "experts say relax." My read is "probably not, but not zero," from people who admit they are guessing.
Source: AI Impacts survey, arXiv 2401.02843.
So I asked Claude
Claude is an AI assistant made by Anthropic. That means I was asking a product built by a company in the AI business, which is a little like asking a brewery whether beer is good for you. The answer brings that up itself, which I appreciated. My question: will AI end humanity? Here is what came back, word for word.
Claude's answer
Honest answer: nobody knows, including me. Here is how I would break it down.
- Probably not in the movie way. I see no evidence of an AI that wants to wipe humans out, and I do not have goals like that. But I am one system, and I cannot speak for every future one.
- The risks serious people worry about are more ordinary and more plausible: people using AI to do harm (scams, cyberattacks, help with dangerous weapons), companies and governments racing to deploy systems before they are well tested, over-reliance on systems we do not understand, and power concentrating in whoever controls the best models.
- A smaller group worries about a future AI that is more capable than humans, pursues goals we did not intend, and cannot be switched off. That is a hypothesis about systems that do not exist yet, and researchers disagree sharply about how likely it is.
- What I would watch: whether we can verify what these systems are doing, whether humans stay in the loop on high-stakes decisions, and whether safety work keeps pace with capability.
Do not take my word as final, either. I am made by a company with a stake in how this question gets answered.
My take: the boring stuff is the stuff we can already see
What struck me is how the answer is weighted. The movie ending gets a "probably not." The dull stuff gets a whole list. And the dull stuff does not need a villain. Systems breaking, and the rest of us leaning on systems we do not understand, are risks you can see without a survey or a crystal ball.
To be clear, nothing here says AI has broken anything. The researchers did not claim that, Claude did not claim that, and I am not claiming it either. I filed the whole exercise under "sensible, slightly anticlimactic" and figured that was the end of my war coverage.
Hollywood has been asking since 1970
Hollywood got there first. Colossus: The Forbin Project (IMDb) is a 1970 film about a US defense supercomputer that gets linked up with a Soviet counterpart and takes control. Eric Braeden plays Dr. Charles Forbin, the scientist who built it, and Braeden is best known as Victor Newman on The Young and the Restless. It is a movie, so treat it as a joke about the movies and not a claim about AI.
Then this week happened. (That is Part 2.)
Next up, Part 2: It's Already Started, looks at the week everything broke. And in Part 3: The Defense Rests, Claude gets the last word and writes its own closing argument.