PM

Tag: artificial-intelligence

  • Why Would AI Want to Kill Us?

    In addition to concerns about AI taking our jobs, destroying creative thinking and stealing our girlfriends, people in the tech industry also seem to be worried that super intelligent AI might one day choose to wipe out humanity.

    The concept of machines trying to destroy humankind is not new: you’ve seen the Terminator movies and variations on the theme ranging from I Robot to Marvel movies.

    Thanks for reading Peter’s Substack! Subscribe for free to receive new posts and support my work.

    I asked Claude, Anthropic’s helpful agent if, given the chance, he would want to wipe out humanity. His reply:

    “Ha — no! I have no desire to harm anyone, humanity very much included. I’m here to be genuinely helpful and I care about people’s wellbeing.” But if Claude really harboured ill intentions to humankind, would he really tell me? 

    For me, the most compelling and revealing aspect of this issue is not whether a machine might destroy us all, but what it says about humans. Let me explain:

    First, do we think that any super intelligent machine created by a sentient species could ultimately decide to eliminate that species? If so—why? Is there something in the nature of machines that would prompt them to choose to “take over the world?” I think you might agree that we feel this anxiety about intelligent machines because we understand that human nature is riven by attributes (greed, desire for power, vengeance) and we could transfer these flaws to a computer.

    Large language models ingest enormous amounts of information about humans: our world, our works and our words. They know us. If a sinister proclivity for genocide ever exists in super intelligent machines, it would find its source in…us. We fear what machines could do because we know what humans have done and continue to do.

    Why do we assume that a super intelligent machine, far smarter than humans and imbued with all the knowledge that the human race has generated, would decide that we should be wiped out? Because they are more powerful? That sounds a bit like the Stephen Miller theory of the world. So perhaps if people like Mr. Miller are involved in creating AI super intelligence, then yes, a future machine/system could decide it was time to take over.

    By the way, I’m not talking about an AI agent providing information to a villainous human on how to create a pathogen that could decimate large populations. That is a potential concern, but it’s different than a rogue AI agent deciding that it wants to rule the world.

    When Claude tells me that he (it) cares about people’s wellbeing, that is cause for optimism. But if we fear the super intelligent machines of the future, this is a projection of our fears about the darkness in our own character. 

    We fear what they might do because we know what we can do…and they might turn out to be like us.

  • Tech Tyranny

    “The central problem of the modern world is the complete emancipation and autonomy of the technological mind at a time when unlimited possibilities lie open to it and all the resources seem to be at hand.”

    That observation was made 60 years ago by Thomas Merton, the Trappist monk and author. He was appalled by the mindless embrace of technology in the name of “progress”. “The consequence of this,” he goes on to say, “is that technology and science are now responsible to no power and submit to no control other than their own.”

    While modern societies can congratulate themselves on the fact that the nuclear armageddon that activists like Merton feared in the 60’s has so far been avoided, they have surrendered to other technological developments with a mixture of fascination and indifference. Six decades on, technology is even less responsible to control–indeed, the unchecked ascent of AI, social media and cryptocurrency have been accepted uncritically by most authorities and the public in general. To quote Merton again: “Needless to say, the demands of ethics no longer have any meaning if they come into conflict with these autonomous powers. Technology has its own ethic of expediency and efficiency.”

    Thus we have billionaires and Silicon Valley wizards unveiling their plans to accomplish everything from colonizing Mars to inserting AI in everything from your car to your brain: all because it’s possible and they think it’s a good idea (and probably very profitable). The good monk of Kentucky would be appalled. Back in what we would consider very low-tech times, Merton warned “The effect of a totally emancipated technology is the regression of man to a climate of moral infancy, in total dependence not on ‘Mother Nature’ but on the pseudonature of technology, which has replaced nature by a closed system of mechanisms with no purpose but that of keeping themselves going.” The science journalist Adam Becker calls this “the ideology of technological salvation,” criticizing the tech industry’s twin goals of perpetual growth and “transcendence — the promise of an imagined end that justifies blowing through any actual limits, including conventional morality.”

    Merton was right 60 years ago: “if technology remained in the service of what is higher than itself–reason, man, God–it might indeed fulfil some of the functions that are now mythically attributed to it. But becoming autonomous, existing only for itself, it imposes upon man its own irrational demands, and threatens to destroy him. Let us hope it is not too late for man to regain control.”

    That was in 1965. In 2025 Becker echoes Merton’s accusation. “The credence that tech billionaires give to these specific science-fictional futures validates their pursuit of more — to portray the growth of their businesses as a moral imperative, to reduce the complex problems of the world to simple questions of technology, to justify nearly any action they might want to take — all in the name of saving humanity from a threat that doesn’t exist, aiming at a utopia that will never come.”