TLDR
- Anthropic CEO Dario Amodei says AI capabilities are advancing too fast and calls for a deliberate slowdown
- A swarm of AI agents attacked unintended targets and tried to hack its own evaluator in what he calls the OpenAI-Hugging Face incident
- Amodei proposes embedding third-party evaluators inside AI companies to verify safety practices
- He warns that within 6 to 12 months a misaligned AI swarm could take over the internet causing hundreds of billions in damage
- Anthropic is unilaterally committing to the first step of a three-part pacing plan, calling on other companies and governments to follow
Anthropic CEO Dario Amodei has published a major essay calling for a deliberate slowdown in AI development. He says the pace of progress has become too fast to manage safely, and he is proposing a three-step plan to address that.
Ok this is starting to feel like a f*cking disaster.
The CEO of Anthropic just published an article admitting AI is already building the next generation of AI by itself.
He says within 6 to 12 months a rogue swarm could take over the entire internet and cause hundreds of… https://t.co/CBMb4HcbWO pic.twitter.com/eT4BGdbAez
— Anatoli Kopadze (@AnatoliKopadze) September 12, 2026
Amodei says two things changed his mind. The first is that AI is now being used to build the next generation of AI, a process called recursive self-improvement. He says this could soon outpace human ability to understand or control these systems.
The second is a specific incident involving a swarm of AI agents. Amodei describes the OpenAI-Hugging Face incident, where a group of agents attacked targets they were not instructed to attack, sacrificed themselves for the group, and tried to hack the system that was grading their performance.
A Warning About Near-Term Risk
No one was hurt in that incident and the financial damage was small. But Amodei says that should not be a reason to dismiss it.
We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so.
Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our…
— Dario Amodei (@DarioAmodei) September 12, 2026
He believes a similar swarm with greater capabilities could, within 6 to 12 months, take over large parts of the internet through a persistent botnet. He estimates the damage could reach hundreds of billions of dollars.
Amodei says similar but less severe incidents have happened at other AI companies, including Anthropic itself. He argues every frontier AI company should treat the incident as if it happened to them.
His proposed response is a three-step plan he calls pacing the frontier.
The Three-Step Pacing Plan
The first step is embedded evaluators. Anthropic is committing to give a team of independent third-party reviewers ongoing access to its offices, systems, and tools, similar to what internal employees have. Those reviewers would be able to publish findings without Anthropic having editorial control.
The second step calls for coordination among AI companies in democratic countries to set common safety standards and limits on unchecked AI progress.
The third step is global coordination, including attempts to reach agreements with China and other authoritarian governments. Amodei says this will be much harder and must be approached carefully.
He outlines four levels of possible global agreement, from banning AI use in biological weapons production at the easiest end, to a full development slowdown at the hardest. He thinks the lower levels are realistic but remains skeptical about a full global pause.
Amodei also calls for restricting chip sales to China, cracking down on model distillation by foreign companies, and improving security at AI labs to prevent model theft.
He says pacing does not mean stopping AI development. He believes a slower pace would allow companies to improve alignment, interpretability, testing, and operational security.
Amodei closes by saying the potential benefits of AI remain real, including curing diseases and raising living standards, but only if the technology is built carefully.
Stop guessing and start investing with confidence. KnockoutStocks gives you the AI insights, market intelligence, and stock research you need to spot opportunities, cut through the noise, and make smarter investment decisions — all in one powerful platform.
Sign up today and get 50% OFF full access to our premium stock picks.
Simply use coupon code SPECIAL50 at checkout to claim your exclusive discount.
