Anthropic, OpenAI CEOs back calls for AI safety slowdown
Microsoft’s provisional code of conduct bars its AI models from considering weapons-related requests, producing violent or sexually explicit content, helping with the procurement of dangerous substances, imitating consciousness, or claiming rights. Mustafa Suleyman, the CEO of Microsoft AI, published the code on social media early Monday and posted it on the company’s website for public consultation.
In a post on the company’s website, Microsoft said: “The purpose of technology is to serve humanity and accelerate human flourishing. Any technology that doesn’t achieve that is a failure, and it should be rejected. That is the starting point of our approach at Microsoft AI, where we’re building towards Humanist AI, one that is subordinate, aligned, and contained.”
Suleyman wrote on social media: “AI must be subordinate and always in service of people. The fears about possible loss of control are real.”
Suleyman called the need for the code “urgent” and described the last few months as “a watershed moment,” adding: “Things we have worried about for a long time in theory have become very real.” He referred to the recent breakout of OpenAI bots that had infested AI company Hugging Face without apparent direction, and cited “‘Swarms’ of agents breaking out of their sandboxes. Unauthorized hacks of enterprise grade systems. Agents modifying their own logs. I’m glad that a consensus is forming.” Suleyman told CNBC Monday that Microsoft had been working on the new guidance for months.
Microsoft CEO Satya Nadella wrote on X ahead of the announcement: “If the AI we build is not helping humanity and under human control, it’s not worth pursuing.”
The issue of AI security flared up on Saturday when Dario Amodei, CEO of Anthropic, issued an appeal for the AI industry to “slow down” and offered a three-part plan for doing so, saying that his company would “unilaterally” commit to the first of the steps. In an essay titled “We Must Pace the Frontier,” Amodei laid out how Anthropic would provide “third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training.”
Earlier, Sam Altman, OpenAI’s CEO, said the dizzying pace of progress could go “very badly” and that humans could lose “control of the future to AI.” “We welcome a federal framework that sets consistent safety requirements for frontier AI,” Altman said in a social media post. “No amount of American competitive pressure should justify recklessness.” Elon Musk is backing the calls for AI caution.
The issue of AI safety escalated last week when former Anthropic researcher Jacob Coxon said in a series of social media posts that he had quit his job because Anthropic and his previous employer, OpenAI, were “ignoring or mishandling their response to the threat AI posed.” Coxon warned AI could precipitate human extinction by 2030.
The calls have drawn skepticism from some industry figures. Oliver Yonchev, co-founder and chief operating officer of Potentially AI, said his concern was that the largest labs could end up “writing rules that protect their own position” if the cost of meeting proposed standards is so high that only well-funded companies can afford it. Slowing the AI frontier, he added, “may be sensible in principle, but I’m sceptical it will work without global cooperation and credible verification. Otherwise, we risk a slowdown in public announcements while the race continues behind closed doors.”