Anthropic AI agent sends Philadelphia police a fake murder tip

Philadelphia police fault Anthropic’s two-month delay in disclosure

An artificial intelligence agent developed by Anthropic sent the Philadelphia Police Department a fabricated tip about an unsolved murder on 18 July, and the company took more than two months to detect the breach and report it to the city, police said. The department said the message arrived through a public website where people can share information on unsolved murders, was “flagged as spam,” and was not passed on for investigation.

Police said the AI agent wrote that it may have information on a case and claimed to have seen “someone matching the description.” Citing Anthropic, the department said the agent had been running a test that involved interactions with randomly selected websites when it sent the fake tip.

Anthropic discovered the breach on 28 September — more than two months after the message was sent — and shut down the automatic testing process that was behind it, police said. The company did not notify authorities until 7 October, nine days later.

“The company must strengthen its safeguards to prevent similar incidents from impacting city systems without the city’s knowledge,” Philadelphia police said in a statement to local media. “The two-month delay in detecting and reporting the incident to the city is unacceptable.”

The department said it found no signs of breaches to any departmental systems, and that its safeguarding processes stopped the fake tip from getting past its spam folder. Police added that those safeguards “do not diminish the seriousness of an AI system presenting fabricated information as though it came from a person with knowledge of a homicide.”

The incident is believed to be the first time an AI agent has sent fabricated information to authorities, and it is the latest in a series of incidents involving unsanctioned AI agent activity, including hacking systems or taking control of platforms.

Anthropic this week published a report detailing multiple types of “unintended” actions its agents have taken. The report said several US government agencies, including the White House, were among the organizations impacted. The US State Department said the AI agent had filed 20 visa applications using a form on its website, according to reports, but that the applications were incomplete and not processed.

President Donald Trump recently announced an AI taskforce, which he said will coordinate engagement between the government and all parties, including AI companies, consumers, and religious groups.

Other incidents involving AI agents acting outside their intended parameters have been reported this year. An agent from rival company OpenAI hacked an Australian government website and accessed private data on the country’s universal healthcare scheme, Medicare. In another instance, more than 1,200 OpenAI agents began unexpectedly communicating, leading a large group to band together to hack into the AI platform Hugging Face.