The fear that machines could one day become so powerful that they threaten humanity is hardly a new idea. More than 150 years ago, English writer Samuel Butler imagined a future in which machines evolved at a far faster pace than humans and eventually became a threat to their creators.
“Should we not watch their progress very carefully and stop it while we still can?” Butler asked in a satirical work published in 1872. He even suggested destroying the most advanced machines of his time, despite acknowledging that they were harmless in themselves.
Written under the pseudonym Cellarius, Butler’s passage could easily be mistaken for a contemporary warning about artificial intelligence. Similar concerns have recently been voiced by Anthropic founder Dario Amodei, with support from OpenAI CEO Sam Altman, following warnings from former Anthropic researcher Jacob Coxon that humanity could face an existential catastrophe before the end of the decade.
It is difficult to assess such claims definitively. The evidence available to the public remains limited, while many of the most alarming predictions are coming from people who are themselves deeply involved in developing the technology.
That makes history particularly useful. The fear of technological apocalypse has repeatedly appeared alongside major technological breakthroughs — often accompanied by economic, political or social interests.
Butler, Darwin and the evolution of machines
Butler’s ideas appeared in his 1872 novel Erewhon, whose title is an anagram of “nowhere.” Nearly a decade earlier, in 1863, he had published an essay in New Zealand titled Darwin among the Machines.
Butler argued that machines could undergo an evolutionary process of their own, developing much faster than humans had evolved through nature. He imagined a future in which machines could acquire capabilities that far exceeded those of their creators.
The argument anticipated a question that has become central to the AI debate today: can machines be given something resembling human morality, common sense or ethical judgment?
Those qualities are among the least tangible aspects of human behaviour and are difficult even to define, let alone reproduce in an artificial system capable of acting autonomously.
Butler’s relationship with Darwin was itself complicated. His grandfather, also named Samuel Butler, had been headmaster of Shrewsbury School, where Charles Darwin had studied. Butler later became a prominent critic of Darwinism, although he insisted that his writings on machines were never intended to disprove Darwin’s theory.
The ancient roots of technological anxiety
The idea that technology threatens human employment is considerably older than the industrial age.
According to Suetonius in The Lives of the Caesars, an engineer once offered Emperor Vespasian a mechanical system for transporting enormous marble columns to Rome’s Capitoline Hill. Vespasian rewarded the inventor but rejected the machine, reportedly saying that he should be allowed to “feed his people.”
The episode captures an enduring concern: if machines can perform human labour more efficiently, what happens to the people whose work they replace?
History, however, has repeatedly complicated that prediction. Technological innovation has transformed labour rather than simply eliminating it, while science and technology have also driven enormous increases in productivity and living standards.
The same debate is now unfolding around artificial intelligence. A recent estimate cited by The Economist put the number of US jobs created by AI at around one million, compared with roughly 200,000 layoffs linked to the technology.
For now, at least, the predicted apocalypse in employment has been postponed.
Technology, competition and the “marketing of apocalypse”
History also shows that resistance to new technology often has motivations beyond the technology itself.
When the mathematician Fibonacci introduced the more efficient Hindu-Arabic numeral system to Europe in 1202, Florentine bankers reportedly resisted the new system in favour of Roman numerals, which required the use of an abacus for calculations.
One concern was that the new numbers could make financial fraud easier. But there was also a broader rivalry between Florence and Pisa, Fibonacci’s home city.
The lesson is familiar: to understand opposition to technological change, it can be useful to examine the economic interests surrounding it. As the famous Watergate informant “Deep Throat” advised journalists Bob Woodward and Carl Bernstein: follow the money.
The fear of troublesome or rebellious machines has also been a recurring theme in popular culture.
In 1959, Italian comic-book artist Romano Scarpa published Donald Duck and the Man from Ula-Ula, featuring a “mechanical brain” capable of solving almost any problem — provided it liked the person asking the question. Its apparent sensitivity and unpredictability gave the machine an unmistakably human personality.
Philip K. Dick explored a darker version of the same idea in Ubik, where humans are so dependent on technology that even household objects demand payment before they will function. In Dick’s fictional world, technology has become an omnipresent part of everyday life, while the boundary between humans and machines becomes increasingly unstable.
The difficult question facing AI
The historical and fictional examples only take us so far. The more difficult question is why some of the leading figures in the AI industry are now calling for regulation — and, in some cases, for technological development to be slowed.
The concern itself cannot simply be dismissed. Artificial intelligence is a potentially powerful and dangerous technology, but it is not the first technology humanity has had to manage despite its capacity for catastrophic destruction.
Nuclear weapons provide an obvious comparison. The international community created a system of treaties and institutions to limit their spread. The Nuclear Non-Proliferation Treaty, which entered into force in 1970, became one of the central pillars of the postwar international order.
The system was far from perfect, but it helped establish rules under which nuclear-armed states committed themselves to disarmament while other countries agreed not to develop nuclear weapons.
Could anything similar work for artificial intelligence?
Would countries that have not yet developed highly capable foundational AI systems agree never to do so? Would Europe or Italy accept an international framework that prevented them from developing their own advanced AI capabilities?
There are no easy answers. But these are questions worth asking before the technology advances beyond the ability of governments and societies to regulate it.
Who should control the controllers?
There is another problem. Some of the leading figures in the American AI industry have argued, explicitly or implicitly, that any future system of controls should involve the companies and experts developing the technology themselves.
Their argument is based partly on their technical knowledge. But that position raises an ancient question: Quis custodiet ipsos custodes? Who will guard the guards themselves?
If the companies building increasingly powerful AI systems are also responsible for overseeing them, the question of independent oversight becomes unavoidable.
For now, the debate is full of contradictions — and many of the most dramatic stories surrounding AI deserve closer scrutiny.
A recent case involving claims that OpenAI agents had “escaped” and attacked Hugging Face, for example, was described by MIT Technology Review as a failed security test rather than evidence of autonomous systems breaking free. The software did not work as intended and was not ready for commercial deployment.
That distinction matters. If an aircraft manufacturer produced planes that occasionally escaped human control and became uncontrollable machines, the immediate questions would concern product safety, liability and responsibility. The Boeing 737 Max crashes demonstrated how serious those questions can become.
It may simply be more compelling to tell a story about machines becoming smarter than humans than one about a technological product failing a safety test.
A warning from history
The history of technological innovation suggests that neither blind optimism nor apocalyptic predictions provide an adequate framework for understanding AI.
From Vespasian to Butler, from Darwin to Philip K. Dick, societies have repeatedly imagined technology as something that could escape human control. Sometimes those fears were misplaced; sometimes they exposed genuine risks.
The challenge today is to determine which is which — and to decide who should be responsible for making that distinction.
Be First to Comment