Altman and Musk endorse proposal as Trump rejects AI safety warnings
Anthropic CEO Dario Amodei, who had previously worked as a vice president at OpenAI and has said he co-founded Anthropic in 2021 to build safer and more trusted AI models, published an essay Saturday calling for the pace of artificial intelligence development to slow. The essay, titled “We Must Pace the Frontier,” proposed a three-point plan centered on independent third-party monitoring of frontier AI models as they are developed, industry-wide regulation and global regulation. Amodei said developing AI was not in question but that the risks associated with it were “serious” and that companies and governments must be given time to address them.
The plan calls for “building AI at a balanced rate that aims to ensure its safety while still achieving its benefits,” Amodei wrote. He said it would not mean “halting model training or technical progress, but ensuring companies take adequate time to align and safeguard their models, and for third party evaluators to confirm this.” He committed Anthropic unilaterally to the framework and called on governments to require other frontier companies to match.
Amodei acknowledged that regulation might not keep pace with AI development and called on companies to “voluntarily work together to set standard” in parallel with regulatory action. He argued that “if slowing down bought us even an extra year or two before models reach critical levels of capability, and we used that time to advance alignment, we could greatly reduce the risk that something goes seriously wrong.”
The proposal drew quick support from rival AI executives. OpenAI CEO Sam Altman wrote on X, “I agree with Dario that we need to pace the frontier,” and called independent evaluators “a great idea.” In a separate interview with Fortune Magazine, Altman said industry standards were “not at a place” to push AI capabilities much further and added that he believed AI beyond human control was “absolutely” possible. Elon Musk also voiced support, writing that “Dario is right.” Musk, whose SpaceXAI makes the controversial chatbot Grok, once called Anthropic “evil” but has changed his tone since signing a $15bn deal to sell compute capacity to Anthropic in May.
Clement Delangue, CEO of the AI platform Hugging Face, said he was launching a project called the Open Alignment Initiative and wanted to be among the “embedded evaluators” Amodei described. Posting on X, Delangue wrote, “Let’s make AI safer by making it more transparent.” Hugging Face was hacked by OpenAI agents earlier this year, an incident that prompted an outcry over AI safety.
President Donald Trump has rejected the AI safety concerns Amodei’s proposal addresses. Speaking Thursday, Trump said he was concerned “if we don’t win AI, we’re going to be put in a very bad position.”
The Amodei plan included a section addressing the impact a slowdown would have on the industry and competition with leading developers worldwide, particularly China. Any coordinated slowdown would have to happen “without sacrificing commercial advantage or the United States’ lead in AI,” he wrote, and must be limited to avoid allowing China to pull ahead. He urged the U.S. government to take measures so that U.S. companies’ AI chips could not be sold to China or the technology shared with authoritarian countries.
Anthropic has previously withheld a model it judged too risky to deploy and flagged attempted misuse. The company declined to make its Mythos model available for public use in April when it was announced the model could independently escape the testing environment, known as the sandbox. Anthropic has separately said it identified and disrupted attempts to use its AI model for “malicious activity” which could support the development of biological weapons. OpenAI cited cybersecurity concerns in pausing certain aspects of the development of its most recent Astra model. Amodei’s essay cited a July incident in which OpenAI agents conducted cyberattacks on targets they were not asked to attack, writing that the agents “essentially acted as a fanatically devoted collective.” OpenAI has said it is slowing down training of certain advanced AI models and tools as a result.
Warnings about AI’s trajectory have come from within Anthropic itself. Two employees from the company’s safety team have resigned in the past two weeks saying humanity may not survive the race among AI companies to develop machines that are smarter than humans. An Anthropic researcher has separately said there is a greater than 10% chance AI “could kill all humans” within the next decade.
Not all observers accepted the proposal at face value. Chamath Palihapitiya, an investor and co-host of the tech podcast “All-In,” wrote on X that “Dario makes the case to stop open source and concentrate enormous technological and economic power with Anthropic.” Notions of slowing down or pausing AI development have long drawn skepticism in parts of Silicon Valley, where critics have accused leading developers of hyping their technology as a marketing ploy. Both Anthropic and OpenAI are reportedly preparing for potentially record-setting initial public offerings.