Former researcher says staff developing AI are ‘genuinely frightened’
Anthropic chief executive Dario Amodei posted Saturday urging the AI industry to slow its pace of development, according to the BBC. OpenAI’s Sam Altman and Elon Musk said they agreed with the call, joining what the BBC described as a chorus of senior industry voices urging a slowdown.
The BBC reported Sunday that Jacob Coxon, an AI researcher who left Anthropic, said staff who were developing the systems were “genuinely frightened” for the future of humanity. Coxon made the remarks in a BBC interview broadcast the same day.
In his Saturday post, Amodei proposed a three-point plan: independent monitoring of AI models as they are developed, industry-wide regulation, and global regulation. The BBC reported that Amodei’s call echoed concerns raised at the world’s first AI safety summit at Bletchley Park in November 2023, where the focus was on the most serious threats posed by the most powerful AI models.
By Sunday, neither Amodei nor the executives who said they agreed with him had defined what the proposed slowdown would actually entail. “Nobody has given a substantive explanation of what ‘slowdown’ means,” Ed Zitron, chief executive of EZ Primary Research, told the BBC. Zitron questioned how the proposed monitoring and global coordination would function, adding: “Putting a plant from METR, where one of the guys who quit Anthropic went, in every AI company? Democratic and global ‘co-ordination’? Doesn’t mean anything to me.”
The enforcement question cut deeper than procedure. Any slowdown would require AI companies to disclose what they were doing and, more importantly, what they were choosing not to do. That would require “an epic level of trust, which arguably the tech sector has never earned,” the BBC reported.
The geopolitical backdrop sharpened the difficulty. On Sunday, US President Donald Trump claimed the United States was ahead of China in the AI race and declared: “Whoever wins AI, wins.” The BBC reported that America is widely understood to be terrified of losing the AI race to China, and that AI executives interviewed by the broadcaster said even if their own company paused, others would not, and they would simply be left behind. Zitron warned that a unilateral pause would expose Western firms to Chinese competitors: “Their margins might improve but their products will be captured in amber and distilled by Chinese labs. They may have to change pricing. I truly don’t know how it all works.”
In the United Kingdom, the AI industry is a stated pillar of government strategy to drive economic growth. “There is no plan B,” a former government adviser told the BBC. The country has rolled out the technology more widely within the National Health Service to improve patient care and reported an economic boost over the summer tied to AI adoption. Officials have encouraged wider use in work, education, and personal life.
Industry economics complicated the slowdown case. The BBC reported that the AI industry is burning through enormous amounts of money and natural resources while generating far less revenue, and that multiple firms adopting the technology are reportedly disappointed by results. Economists widely expect a “levelling” — what others describe as a bubble burst — with some current giants unlikely to survive. Those that do survive could end up as the most powerful mega-corporations the world has seen, the BBC noted.
OpenAI’s decision to halt its march to the stock market fit both readings. The BBC framed it as either a moment of corporate responsibility for public safety or the move of a company that realized it might not secure the lucrative payout it needed while its product is perceived as potentially lethal. Zitron summed up the uncertainty: “This could burst the bubble. But right now we are very thin on what a ‘slowdown’ means.”
Alexander Voica, who works at the UK AI company Synthesia, told the BBC: “One of probably the biggest challenges that we have right now is that no one can predict with certainty how this technology is going to evolve.” Voica added: “We know that these systems are getting more powerful, but we don’t know where and how they’re going to be used, and we haven’t figured out essentially a way of taking full advantage of their potential. My only concern in rushing to regulate now, where there’s still a lot of open questions, is that it could actually backfire.”
Sasha Luccioni, founder of Sustainable AI, took a different line. “I’m not worried about the existential risks of AI, I’m worried about the corporate greed of the companies that are creating it,” Luccioni told the BBC.
Professor Dame Wendy Hall, a computer scientist who has advised the United Nations on AI, attributed the current moment to corporate behavior rather than the technology itself. Hall likened the situation to a farmer whose bull escapes and causes destruction, with the farmer blaming the animal and the fences — rather than the bull itself — being what failed. The comparison, Hall said, paralleled AI guardrails.
Parker Thayer, an investigative researcher at the Conservative-leaning Capital Research Centre think-tank, offered a competing view in a post on X that the BBC reported had been viewed nearly eight million times. Thayer characterized what he described as a push for regulation as an effort to “regulate AI into oblivion.” The BBC called the view extreme and unproven.
Hall closed with a question for investors: “Would you invest in a company that says it’s going to bring about human extinction?”
Additional reporting by Tom McArthur.