SAN FRANCISCO — Nicholas Carlini, the well-respected hacker who in March warned a standing-room-only crowd of cybersecurity experts that Anthropic’s Mythos AI model had changed the balance of power between attackers and defenders, has spent recent days in Washington helping the company explain safeguards to the very government officials who then banned foreign use of the technology.
The lanky 35-year-old researcher at Anthropic had long been considered the industry’s “professional skeptic” of AI cybersecurity claims, Dan Guido, chief executive of cybersecurity firm Trail of Bits, said. But after getting his hands on Mythos, Carlini changed his mind. In a presentation at a beaux-arts building that once housed San Francisco’s Hibernia Bank, he showed how he had used the AI to find and exploit a critical bug in the web-publishing software Ghost, then another in the Linux operating system.
“The balance that existed between attackers and defenders over the past two decades seems like it’s probably coming to an end,” Carlini told the crowd. “It’s pretty clear to me that these current models are better vulnerability researchers than I am.”
Two days after the talk, Carlini sent a note to his colleagues at Anthropic. “I don’t think we should release Mythos yet,” he wrote.
Carlini had never before found a bug in Linux or in Ghost. In a few days, Mythos searched through Linux’s code several thousand times and found 479 bugs. “A competent security researcher could go their whole life without finding a Linux kernel vulnerability,” Carlini said.
The bug in Ghost was one of 500 bugs Mythos uncovered in a two-week period. Ghost’s developers patched it Feb. 16, weeks before Carlini’s talk. But not everyone who used Ghost updated their software, and hackers later launched widespread attacks on websites without the update, according to cybersecurity firm Xlab, which reported that more than 700 sites were hacked within a month.
Last week, Anthropic released updates called Mythos 5 and a product called Fable 5, a version with safety measures. On Friday, the Trump administration banned foreign governments, companies and individuals from using both. Anthropic shut off access to everyone to comply.
The restrictions were spurred by an Amazon report finding that Fable could be coaxed into finding bugs, people familiar with the matter said. Amazon Chief Executive Andy Jassy called officials including Treasury Secretary Scott Bessent to share the findings, and administration officials grew more alarmed after conversations with government security experts.
After the calls, officials including National Cyber Director Sean Cairncross gave CEO Dario Amodei and other Anthropic leaders an ultimatum: work with the government and take down the models that day or face a ban on foreign users, a source close to Anthropic said. The company was told it had 90 minutes and was not given details about the security risk.
President Trump asked Commerce Secretary Howard Lutnick to help handle the situation and approved shutting off all foreign use of the models, some people familiar with the matter said. Lutnick sent Amodei a letter notifying him the rules had been implemented shortly after 5 p.m. ET.
When Lutnick and Amodei spoke that evening, Amodei said, “This means we can’t have the model out,” according to people familiar with the call. “That’s the point,” Lutnick responded.
The episode escalates a monthslong dispute between the government and Anthropic, which is valued at nearly $1 trillion. Defense Secretary Pete Hegseth and Amodei clashed earlier this year over the company’s efforts to control military use of its products, with the Pentagon pushing back and triggering multiple lawsuits. The two sides have also differed over AI policy, chip exports to China and Anthropic’s ties to nonprofits that are big donors to liberal causes.
In recent days, administration officials and Anthropic executives, including Carlini, have held hours of meetings. Some administration officials have said a resolution should include an acknowledgment from Anthropic that its rollout and communication could have been improved, people familiar with the talks said.
Michael Horowitz, a senior fellow for technology and innovation at the Council on Foreign Relations and a former Defense Department official, said the government and Anthropic “clearly have an inability to communicate effectively with each other” and that “more technical exchanges should be helpful.”
Mythos has already found more than 10,000 bugs, according to the company. In a March memo, Carlini wrote that Mythos is “the first model that can find and exploit vulnerabilities at scale.” The model is capable of creating “exploit” code — software that leverages bugs to cause harm.
Carlini believes it is only a matter of months before other models catch up with Mythos. “I don’t think we should release Mythos yet” — his earlier warning — has been overtaken by events: Mythos has been released, and the government has restricted its access. Now the skeptic who sounded the alarm is the expert sent to calm the policy makers who took him at his word.