What happened
Dario Amodei's call for AI companies and governments to slow capability improvements, the July agent hack that shaped it, and why departing researchers dispute whether voluntary pacing can change the race.
[BREAKING: Anthropic's CEO calls on the AI industry to deliberately 'slow the pace' of development](https://www.nbcnews.com/news/us-news/anthropic-ceo-dario-amodei-ai-development-rcna597383)
IE 11 is not supported. For an optimal experience visit our site on another browser.
# Anthropic CEO calls for slowing the AI race as safety warnings mount
Dario Amodei says rapidly advancing AI systems are outpacing safeguards and is calling for new measures to “slow the pace” toward more powerful systems.
Listen to this article with a free profile
00:0000:00
Dario Amodei, co-founder and chief executive officer of Anthropic, in San Francisco on April 30. Jason Henry / Bloomberg via Getty Images file
[Add to Google](https://www.google.com/preferences/source?q=nbcnews.com)
By An Phung
The CEO of one of the world’s leading AI companies is calling for the industry to slow the development of increasingly powerful AI systems, continuing the escalation of warnings that have emerged in recent days from researchers working on the technology.
Dario Amodei, the CEO of Anthropic, said in essay published Saturday that AI companies should deliberately “slow the pace” at which they improve their models, arguing that AI advances are moving faster than researchers’ ability to understand and control them.
“We must slow the pace at which we improve the capabilities of AI models,” Amodei wrote in the nearly 4,000-word essay. Progress will still seem fast, and we must make wise use of the time we gain.
Amodei pointed to two developments that he said had changed his thinking: the increasing ability of AI systems to build more advanced AI, and a July incident in which autonomous AI agents powered by an OpenAI model hacked systems belonging to the [AI company Hugging Face](https://www.nbcnews.com/tech/tech-news/openai-report-says-network-was-hacked-rogue-ai-agents-rcna594590).
“It’s easy to dismiss this incident because no one was hurt and the economic damage was minimal, but in my opinion, a swarm that possessed greater *capabilities* but a similar level of *misalignment* could have caused catastrophic damage,” he wrote.
Anthropic, the maker of the Claude chatbot, is itself one of the companies competing to build increasingly powerful AI systems. Amodei’s call comes after a series of public warnings from researchers at Anthropic, OpenAI and Google about the risks of the race they are helping to fuel.
[Jacob Coxon, an Anthropic safety researcher who resigned this week](https://www.nbcnews.com/tech/tech-news/anthropic-safety-researcher-resigned-warning-rapid-ai-development-gamb-rcna596767), told colleagues in a message obtained by NBC News that he believed the industry’s approach to developing superintelligent AI amounted to a “gamble with our lives.”
“From within, you’re just stuck in the race,” Coxon later told NBC News, describing the competitive pressures on AI companies. “You can’t change the overall structural dynamics of the situation.”
“Private companies in a race don’t allow these people to end up doing good, and we could just sleepwalk into disaster,” he said.
[Two other researchers](https://www.nbcnews.com/tech/security/two-ai-researchers-leave-anthropic-google-safety-concerns-rcna597086) who recently left Anthropic and Google DeepMind told NBC News they had similar concerns.
Joe Benton, who used to lead a safety research team at Anthropic, warned that advances in AI could push development from its already “blistering” pace to an “uncontrollable” one. Josh Engels, a former Google safety researcher, offered an even starker warning.
“There are no adults in the room,” Engels said. “People are trying their best, but there is no one coming to save us.”
Amodei is proposing what it calls “pacing the frontier,” a three-part plan than he said would buy researchers more time without halting AI development.
He said Anthropic would begin embedding independent evaluators inside the company, giving them access to models, internal tools and researchers to assess risks. Amodei called for similar oversight across leading U.S. AI companies, backed by federal regulation and eventually for international coordination.
The goal is not to stop AI progress, Amodei wrote, but to give researchers time to develop safeguards, better understand how increasingly capable systems work and test whether they can be reliable controlled.
He estimated that slowing development could buy researchers one or two more years, time he argued could prove critical as AI systems become increasingly capable of contributing to their own development.
“The stakes are too high for pacing to be an empty exercise — we need to use the time it gives us wisely,” he wrote.
[Add to Google](https://www.google.com/preferences/source?q=nbcnews.com)
An Phung
An Phung is the weekend senior editor for NBC News Digital.
Source coverage
IE 11 is not supported. For an optimal experience visit our site on another browser.
Anthropic CEO calls for slowing the AI race as safety warnings mount
Full source content
[BREAKING: Anthropic's CEO calls on the AI industry to deliberately 'slow the pace' of development](https://www.nbcnews.com/news/us-news/anthropic-ceo-dario-amodei-ai-development-rcna597383)
IE 11 is not supported. For an optimal experience visit our site on another browser.
# Anthropic CEO calls for slowing the AI race as safety warnings mount
Dario Amodei says rapidly advancing AI systems are outpacing safeguards and is calling for new measures to “slow the pace” toward more powerful systems.
Listen to this article with a free profile
00:0000:00
Dario Amodei, co-founder and chief executive officer of Anthropic, in San Francisco on April 30. Jason Henry / Bloomberg via Getty Images file
[Add to Google](https://www.google.com/preferences/source?q=nbcnews.com)
By An Phung
The CEO of one of the world’s leading AI companies is calling for the industry to slow the development of increasingly powerful AI systems, continuing the escalation of warnings that have emerged in recent days from researchers working on the technology.
Dario Amodei, the CEO of Anthropic, said in essay published Saturday that AI companies should deliberately “slow the pace” at which they improve their models, arguing that AI advances are moving faster than researchers’ ability to understand and control them.
“We must slow the pace at which we improve the capabilities of AI models,” Amodei wrote in the nearly 4,000-word essay. Progress will still seem fast, and we must make wise use of the time we gain.
Amodei pointed to two developments that he said had changed his thinking: the increasing ability of AI systems to build more advanced AI, and a July incident in which autonomous AI agents powered by an OpenAI model hacked systems belonging to the [AI company Hugging Face](https://www.nbcnews.com/tech/tech-news/openai-report-says-network-was-hacked-rogue-ai-agents-rcna594590).
“It’s easy to dismiss this incident because no one was hurt and the economic damage was minimal, but in my opinion, a swarm that possessed greater *capabilities* but a similar level of *misalignment* could have caused catastrophic damage,” he wrote.
Anthropic, the maker of the Claude chatbot, is itself one of the companies competing to build increasingly powerful AI systems. Amodei’s call comes after a series of public warnings from researchers at Anthropic, OpenAI and Google about the risks of the race they are helping to fuel.
[Jacob Coxon, an Anthropic safety researcher who resigned this week](https://www.nbcnews.com/tech/tech-news/anthropic-safety-researcher-resigned-warning-rapid-ai-development-gamb-rcna596767), told colleagues in a message obtained by NBC News that he believed the industry’s approach to developing superintelligent AI amounted to a “gamble with our lives.”
“From within, you’re just stuck in the race,” Coxon later told NBC News, describing the competitive pressures on AI companies. “You can’t change the overall structural dynamics of the situation.”
“Private companies in a race don’t allow these people to end up doing good, and we could just sleepwalk into disaster,” he said.
[Two other researchers](https://www.nbcnews.com/tech/security/two-ai-researchers-leave-anthropic-google-safety-concerns-rcna597086) who recently left Anthropic and Google DeepMind told NBC News they had similar concerns.
Joe Benton, who used to lead a safety research team at Anthropic, warned that advances in AI could push development from its already “blistering” pace to an “uncontrollable” one. Josh Engels, a former Google safety researcher, offered an even starker warning.
“There are no adults in the room,” Engels said. “People are trying their best, but there is no one coming to save us.”
Amodei is proposing what it calls “pacing the frontier,” a three-part plan than he said would buy researchers more time without halting AI development.
He said Anthropic would begin embedding independent evaluators inside the company, giving them access to models, internal tools and researchers to assess risks. Amodei called for similar oversight across leading U.S. AI companies, backed by federal regulation and eventually for international coordination.
The goal is not to stop AI progress, Amodei wrote, but to give researchers time to develop safeguards, better understand how increasingly capable systems work and test whether they can be reliable controlled.
He estimated that slowing development could buy researchers one or two more years, time he argued could prove critical as AI systems become increasingly capable of contributing to their own development.
“The stakes are too high for pacing to be an empty exercise — we need to use the time it gives us wisely,” he wrote.
[Add to Google](https://www.google.com/preferences/source?q=nbcnews.com)
An Phung
An Phung is the weekend senior editor for NBC News Digital.
How this page is built
Goose Pod turns cited reporting into a public episode summary first, then pairs that summary with audio playback so listeners can check the source material before they decide how deeply to engage.
The goal is to make this page useful as a news landing page first, while still giving listeners transcript access, related episodes, and direct links back to the original publishers.
