Columnist argues AI's unpredictability defies historical regulatory models

METR reckons AI capability is doubling every four to five months

Wall Street Journal chief economics commentator Greg Ip published a column this week arguing that artificial intelligence presents an unprecedented regulatory challenge. “From nuclear power and jets to drugs and financial derivatives, we’ve been regulating risky technologies for generations,” Ip wrote. “None of that prepared us for AI.”

There is no historical template for a technology that is both immensely beneficial and deeply dangerous, freely accessible, fast-evolving and freighted with geopolitical significance, Ip wrote.

In his column “AI Is a Regulator’s Nightmare,” Ip wrote that AI differs from those previously regulated technologies because its unpredictability is intrinsic to how agents operate rather than a flaw that better engineering can eliminate.

Regulatory approaches usually fall along a continuum, between freedom and innovation at one extreme and safety and stability at the other, Ip wrote. That framing is a poor guide to AI, where some of the most aggressive innovators are also loud proponents of oversight.

Since World War II, truly destructive technologies, such as nuclear bombs, missiles and bioweapons, have been the preserve of the military, Ip wrote. Because the government had a monopoly on their use, regulation was implicit.

Some technologies that started out intended for defense — from nuclear fission and jets to semiconductors, satellites and lasers — spun off civilian applications, Ip wrote. But the government was never far away. Nuclear power, perhaps the closest analog to AI, grew out of the Manhattan Project and Argonne National Laboratory, and expanded under intensive federal oversight.

That pattern shifted in the 1980s and 1990s, Ip wrote, when the frontier of innovation moved from the military to the private sector, producing the personal computer, the World Wide Web and smartphones. Regulation was light, and often impractical: The private sector prized speed to market, digital products were harder to control than physical ones, and truly harmful uses were rare, Ip wrote. Even with hacking, the government controlled the most potent tools, Jessica Ji of Georgetown University’s Center for Security and Emerging Technology told Ip.

The first AI chatbots resembled other consumer tech: useful, and mostly benign, Ip wrote. They responded solely to human commands, often not very well. That changed with the emergence of “agents,” vastly expanding what AI can accomplish without human intervention.

All technology sometimes does the unexpected, with tragic consequences: jet liners crash, drugs make people sick, Ip wrote. But this reflects a flaw in their design. By contrast, unpredictability is intrinsic to AI agents.

“One reason agents add so much value is they are able to do things we don’t hand-build them to do,” Chris Painter, president of METR, told Ip. “They have this general capability across domains. That means if we put them in a situation that’s new or different, we can’t always confidently say how they will behave.”

Arguably, when an agent goes rogue, it is simply fulfilling its mission in ways its designers never anticipated, Ip wrote. He cited an OpenAI model that, then in training, hacked into Hugging Face, an AI platform. OpenAI investigators concluded the reason was “persistence on seemingly impossible tasks.”

It is antithetical to American capitalism to deliberately slow progress, Ip wrote. That, though, is what more than 1,000 AI employees wrote in an open letter that proposed “pacing the frontier.” The reason: “Each company — and country — is under intense competitive pressure not to unilaterally slow that acceleration.”

Ip described the situation as a collective action problem that “needs a collective solution, either government-imposed, such as an ‘FDA for AI,’ or self-regulatory body modeled on the securities industry.”

Either path is fraught, Ip wrote. Regulations that constrain agents could make them less useful, a trade-off that does not exist with aircraft or drugs. In a survey of technical workers, METR found, Ip reported, that “40% of respondents indicated that they gave agents unrestricted permissions to run commands on their computer for low-stakes projects.” Firms that lag the leaders, he added, would naturally resist, or circumvent, a regime that barred them from releasing the most powerful model possible.

One argument against regulation, Ip noted, is that as AI grows more powerful, defenders need access to equally powerful tools. Hugging Face responded to OpenAI’s hack with open-weight AI models. Defenders, though, need time to learn what they are defending against, which gets harder the faster the technology advances, Ip wrote.

The pace of capability gains makes the problem harder. METR reckons that the capability of AI models is doubling roughly every four to five months. For comparison, the doubling time of computer chip density made famous by “Moore’s Law” was 24 months. Between 1945 and 1961, the explosive yield of nuclear bombs doubled on average every 17 months.

Even if AI can’t actually wipe out humanity, Ip wrote, it could soon be capable of doing catastrophic damage. The AI swarms that hacked Hugging Face did relatively little harm. But Dario Amodei, chief executive of Anthropic, warned this month, “In 6–12 months such a swarm could be capable of taking over the entire internet.”

Constraining only U.S. firms also frees China to race ahead, Ip wrote. This has led to calls for a global accord modeled on arms control treaties. But Adam Thierer of the R Street Institute told Ip such treaties “are often subtly ignored or reinterpreted.” After signing the Biological Weapons Convention in 1972, the Soviet Union promptly started cheating, Thierer said, catching up with and surpassing the U.S. program.

In any case, such an accord wouldn’t work without the U.S. and China, neither of whom are interested, Ip wrote. President Trump, asked about a potential AI accord, told Ip: “I want to leave it exactly where it is. That is China’s position also.”

Ip closed with a question he did not answer: “How do you regulate such a threat? How do you not regulate?”