In addition to concerns about AI taking our jobs, destroying creative thinking and stealing our girlfriends, people in the tech industry also seem to be worried that super intelligent AI might one day choose to wipe out humanity.
The concept of machines trying to destroy humankind is not new: you’ve seen the Terminator movies and variations on the theme ranging from I Robot to Marvel movies.
Thanks for reading Peter’s Substack! Subscribe for free to receive new posts and support my work.
I asked Claude, Anthropic’s helpful agent if, given the chance, he would want to wipe out humanity. His reply:
“Ha — no! I have no desire to harm anyone, humanity very much included. I’m here to be genuinely helpful and I care about people’s wellbeing.” But if Claude really harboured ill intentions to humankind, would he really tell me?
For me, the most compelling and revealing aspect of this issue is not whether a machine might destroy us all, but what it says about humans. Let me explain:
First, do we think that any super intelligent machine created by a sentient species could ultimately decide to eliminate that species? If so—why? Is there something in the nature of machines that would prompt them to choose to “take over the world?” I think you might agree that we feel this anxiety about intelligent machines because we understand that human nature is riven by attributes (greed, desire for power, vengeance) and we could transfer these flaws to a computer.
Large language models ingest enormous amounts of information about humans: our world, our works and our words. They know us. If a sinister proclivity for genocide ever exists in super intelligent machines, it would find its source in…us. We fear what machines could do because we know what humans have done and continue to do.
Why do we assume that a super intelligent machine, far smarter than humans and imbued with all the knowledge that the human race has generated, would decide that we should be wiped out? Because they are more powerful? That sounds a bit like the Stephen Miller theory of the world. So perhaps if people like Mr. Miller are involved in creating AI super intelligence, then yes, a future machine/system could decide it was time to take over.
By the way, I’m not talking about an AI agent providing information to a villainous human on how to create a pathogen that could decimate large populations. That is a potential concern, but it’s different than a rogue AI agent deciding that it wants to rule the world.
When Claude tells me that he (it) cares about people’s wellbeing, that is cause for optimism. But if we fear the super intelligent machines of the future, this is a projection of our fears about the darkness in our own character.
We fear what they might do because we know what we can do…and they might turn out to be like us.
PM 