AI researchers ask US to support international limits on development pace

The letter warns capabilities could outpace human control

An open letter signed by 1,367 researchers and engineers at frontier artificial-intelligence labs asks the U.S. government to support “an international effort to develop the technical and governance tools needed to deliberately pace the frontier of automated AI development,” Russell wrote.

The letter states, “There is a real risk that capability development rapidly accelerates beyond our ability to understand or control the resulting systems.” The signers work mainly at OpenAI, Anthropic and Google DeepMind, according to Russell.

Russell interpreted “pacing the frontier” as developing and releasing more advanced systems only when they can be shown to be safe. He wrote that the approach would require a licensing process with strong safety verification, registration of AI systems, the ability to monitor and terminate them when necessary, and enforceable international agreements.

Russell said those tools would take time to establish and would be difficult to put in place after an emergency had begun. He presented the letter as a response to concerns that AI capabilities could accelerate rapidly, including through systems that contribute to their own improvement.

The signers’ personal statements described three concerns, Russell said. They called for coordination to avoid an arms race, described a lack of credible plans for controlling systems more capable than humans, and expressed fear about the consequences if development continues without a negotiated slowdown.

One signer described the world as being “locked in a deadly race towards an intelligence explosion” and said coordination was needed “to slow down the race.” Another wrote that the AI industry has “the explicit goal of building things smarter than humans” while researchers do not yet know how to keep such systems under human control.

Other signers described being afraid of futures that did not include “a near-future negotiated slowdown” and wrote, “I want a world where my kids can have a full life not a world where they are wiped out,” according to Russell.

Russell said almost 100 signers provided personal statements and that similar concerns arose in private conversations, closed-door meetings and off-the-record chat groups. He said the signers include CEOs, co-founders, chief scientists and hundreds of AI researchers.