Anthropic co-founder says mandatory AI 'kill switch' may be needed

Anthropic scientist puts AI extinction possibility above 10 percent

Anthropic co-founder Jack Clark told the BBC that a mandatory, third-party-checked “kill switch” for dangerous artificial intelligence may eventually need to be required of the industry, as warnings from AI executives, researchers and lawmakers about risks from the technology intensified this week. Clark’s comments followed a weekend call from Anthropic chief executive Dario Amodei for the pace of AI development to slow and be more closely monitored.

Speaking to the BBC earlier on Monday, Clark — one of Anthropic’s seven founders — said a way of shutting off AI software completely if it becomes too dangerous was something society “might want to eventually pass rules around.” “Most labs have different ways of being able to pull the plug,” Clark said, including Anthropic, but he added that lawmakers may need to enforce having one. His comments add to the growing number of tech-industry and political figures calling for better safeguards.

Other industry leaders have backed Amodei’s call. OpenAI chief executive Sam Altman and Elon Musk of xAI have said they agree with Anthropic’s proposal for industry-wide deceleration and regulation, as well as independent monitoring of AI model development. Amodei also said any action to rein in AI should be done “without sacrificing commercial advantage,” though some have questioned the motivations behind the slowdown call.

Concerns about AI’s potential risks escalated in recent days after a post by Jacob Coxon, a researcher who left Anthropic over concerns AI could wipe out humanity, went viral. In response, Hubinger said he thought the possibility of AI causing human extinction “within the next decade” was more than 10 percent.

On Monday, Microsoft AI became the latest major company to discuss limiting the models it is developing, as it published an outline document on what it called “humanist AI.” The firm’s chief executive, Mustafa Suleyman, told CNBC that the firm had been working on the new guidance for months and had decided to publish it now due to the current debate around the technology.

The push for new safeguards runs alongside resistance from the White House. In a series of social-media posts on Monday, Trump compared warnings about AI to the “Global Warming Scam,” which he said was “being perpetrated by the Radical Left Dumocrats.” Trump called himself “the Hoax Buster” and likened concerns about the safety of the technology to “the RUSSIA, RUSSIA, RUSSIA HOAX.” In an earlier post, he wrote: “There is a SICK conspiracy going on against AI and Data Centers, and the only one that is happy about it is China. WHOEVER WINS AI, WINS!”

Those concerns triggered a selloff in shares of some tech firms on Monday as investors weighed the impact of a potential slowdown in the development of AI. On Monday, Chinese state media said experts had warned that “Washington’s growing anxiety over China’s rapid AI development appears to be increasingly distorting its policy priorities.” “They know that China has become a strong competitor in AI, and worry that if the US slows down development or tightens regulation, China could catch up even faster,” Xin Qiang, a professor of American Studies at Fudan University, told the Global Times.

The political spotlight on AI policy is set to sharpen Tuesday, when former Trump chief strategist Bannon and veteran progressive Senator Sanders — figures on opposite sides of US politics — are due to speak at the “Pro-Human Assembly” in Washington. According to its website, the gathering “aims to define a shared vision and articulate a better, more human-centric path for AI development.” Sanders told BBC’s Newsnight last week that “we have got to do something immediately to stop the uncontrolled growth of AI,” warning of “the existential threat of the possibility of humanity being wiped out.”