Anthropic says it disrupted attempts to use Claude for biological weapons

Sanders pushes ban on AI superintelligence as Trump rejects such fears

The California-based company published its threat intelligence report Thursday, listing cases in which suspected state-sponsored groups, criminals, spyware vendors, state propaganda institutions and politically motivated individuals misused its technology. Anthropic said it disrupted “malicious use” of its Claude Haiku, Sonnet, and Opus models during the eight-month period.

The findings are the latest in a growing series of warnings from AI researchers and industry insiders about the technology’s potential risks to humanity.

The report highlighted “five case studies of actors using our models in ways that could support biological weapons development.” Biological misuse is “one of the most serious risks of frontier AI model,” the company said. Without proper safeguards, such capabilities “could have catastrophic consequences,” according to the report. Jacob Klein, the head of threat intelligence at Anthropic, told the New York Times it was “an incredibly nuanced situation.” “You are not seeing someone in a comic book kind of way say, ‘Hey, I want to build a biological weapon to kill everybody,’” Klein said.

Anthropic noted in the report that the same information used to develop a biological weapon could also be used to develop a vaccine or cure for a disease. The company also identified six cases in which Claude was used “to develop software for conventional weapons, including firearms, missiles, armed drones, bombs, and other munitions, as well as the targeting and control systems that operate them.”

The report indicated that cybercriminals and state-backed hackers have increasingly used its technology to assist their operations. Other misuse cases ranged from fake dating apps and hotel Wi-Fi scams to surveillance built to identify dissidents. Claude was also used by actors linked to a Russia-based cyber espionage campaign and by an Iranian propaganda institution, according to the report. The hacking group ShinyHunters and China-based labs were among those named. A hacking group whose work is consistent with the Russia-based Midnight Blizzard allegedly used AI to build a system that automatically detected when its malware was flagged by security defenses and rewrote code until it evaded detection.

None of the misuse cases involved Claude Fable or the powerful Mythos-class models, with the exception of one instance of distillation — the process of training smaller AI models using larger, more expensive ones. Anthropic separately accused Chinese AI firms of trying to replicate Claude’s capabilities. The company said it had shared intelligence with authorities and industry partners where appropriate and incorporated its findings into its processes “to better prevent, detect, and disrupt these activities in the future.”

On Tuesday, Google separately disclosed that a person had attempted to use its AI tool, Gemini, to obtain a “complete, step-by-step technical guide for synthesizing weaponised biological agents.” Such threat reports have become a regular feature across the AI industry as firms seek to demonstrate how they identify and disrupt attempts to misuse their models.

The warnings have prompted calls for industry-wide coordination. In an article published 6 September, OpenAI chief scientist Jakub Pachocki called for the industry to implement “voluntary slowdowns” until safeguards are set. “I am concerned no one is prepared for the consequences of a continued rapid rise in machine intelligence,” Pachocki said, adding that OpenAI, which makes ChatGPT, will continue its work on building safeguards, though broader interventions are required.

Separately, a top safety researcher at Anthropic warned that AI is advancing so quickly that he believes there is a greater than 10% chance it “could kill all humans” within the next decade.

In the United States, Senator Bernie Sanders has introduced legislation to ban AI superintelligence and temporarily pause advanced AI development. “When scientists tell you there is a chance, a chance that it could have a cataclysmic impact on humanity, you’ve got be a moron not to say, slow it down,” Sanders said Thursday on BBC’s Newsnight.

President Donald Trump has rejected such fears. “If we don’t win AI, we’re going to be put in a very bad position,” Trump said Thursday.

The warning prompted an open letter to UK Prime Minister Andy Burnham calling for a new multinational treaty for the safe development of AI and a call for governments to collaborate about what a treaty based on the development of superintelligence should look like.