Anthropic’s Dario Amodei flags civilisational risks from rapid AI advances, says 'AI will test humanity'

Humanity needs to urgently recognise the serious civilisational challenge posed by powerful artificial intelligence systems, Anthropic chief executive Dario Amodei warned, cautioning that the pace of AI development could outstrip society’s ability to safely manage its consequences.

In a 38-page essay published on January 26 titled The Adolescence of Technology, Dario Amodei stated that humanity is entering a turbulent but inevitable transition that will test who humans are as a species. He said AI is handing humanity almost unimaginable power, while it remains unclear whether existing social, political and technological systems are mature enough to wield it responsibly.

The essay follows Amodei’s 2024 paper Machines of Loving Grace, which focused on the potential benefits of advanced AI if deployed correctly. In his latest work, Amodei sought to outline the risks humans may soon face and proposed a framework to mitigate them through voluntary actions by AI companies alongside carefully designed government interventions.

Amodei informed that some level of government action would be required, but warned that regulation carries risks of destroying economic value or coercing actors who may be sceptical of AI threats. He added that regulation often backfires or worsens the problems it aims to solve and therefore needs to be judicious, simple and designed to impose the least possible burden while still addressing the risk.

A central concern outlined in the essay is the proximity of extremely powerful AI systems, which Amodei suggested could emerge within one to two years. He likened this scenario to the sudden appearance of a country of geniuses housed in a data centre, describing a hypothetical situation where tens of millions of superhumanly capable entities could outperform the best human minds by an order of magnitude in nearly every cognitive task.

Amodei stated that such a development would represent an unprecedented and dangerous situation, likely to be viewed by national security officials as the most serious threat faced in a century, if not ever. He said the scale of the risk demands focused attention from the world’s leading thinkers and institutions.

Despite the risks, Amodei expressed cautious optimism, stating that if governments and companies act decisively and carefully, the challenges posed by AI can be overcome. He added that while the odds of success are good, the scale of the civilisational test must not be underestimated.

Among the most serious dangers highlighted in the essay is the risk of bioterrorism. Amodei said biology is the area he is most concerned about due to its destructive potential and the difficulty of defence. He warned that advanced AI could elevate individuals with malicious intent to the capability level of highly trained virologists, significantly increasing the risk of biological threats.

To counter such risks, Amodei said AI companies should implement guardrails to prevent models from assisting in the development of bioweapons and invest in biological defence measures. He added that governments can play a role by introducing transparency requirements to improve monitoring and collective risk management.

The essay also warned of large-scale job losses and increasing concentration of economic power. Amodei has previously stated that AI could displace up to 50 per cent of entry-level white-collar jobs within the next one to five years. He said companies should begin thinking about how to support employees affected by these shifts, including short-term reassignment within organisations.

In the longer term, Amodei said it may be possible for companies to continue paying human employees even after their roles no longer generate traditional economic value, given the significant wealth and productivity gains AI could create. He added that Anthropic is currently considering several potential pathways for its own workforce and plans to share more details in the near future.