GAO finds some Pentagon AI projects miss responsible-use tenets

Six of 10 officials reported risks in Pentagon AI projects

The U.S. Government Accountability Office surveyed 10 officials developing 10 weapons and tools for the Pentagon that use AI to identify battlefield targets, process surveillance, improve tactical decisions and perform other tasks. The projects are among more than 300 AI-enabled efforts being built by the department. The report, published Oct. 14, documented responses indicating that officials were not always closely following the department’s responsible-AI tenets.

Six of the 10 officials said their teams had “identified risk(s)” with using the projects “in an operational environment.” Four officials said requirements for the weapons and tools they were working on were not “aligned with Responsible AI tenets.” Three of the 10 said they “don’t know” if the projects would be continuously monitored after deployment, though the tenets encourage such monitoring.

The Pentagon’s 2021 memo set out six basic tenets for creating AI-enabled weapons responsibly. The principles stress Pentagon-wide oversight, verification and “real-time monitoring” of technology, along with cooperation with allies “to advance global norms grounded in shared values.”

Alissa Czyz, who directed the GAO review, said the tenets evolved from earlier ethical principles and were understood to be important by people involved in building AI tools. “It’s a memo, right, at the end of the day, so it’s not a department instruction,” said Czyz. “It’s something that the department has signaled that should be implemented.” Most of the people involved with making the 10 weapons and projects reviewed by GAO had taken “at least initial steps” to follow the tenets, according to the report, and some of the projects were in early stages, Czyz clarified.

AI researchers and international organizations have long advocated strict regulation of AI weapons that could become unpredictable or function with little human control. They have argued that such tools could endanger soldiers and civilians and risk quickly escalating wars.

Defense Secretary Pete Hegseth has repeatedly dismissed previous caution about AI tools and encouraged rapid autonomous innovation at the Pentagon. In January, the department issued a memo saying it would become “an ‘AI-first’ warfighting force across all components, from front to back.” Last month, Hegseth announced a new warfare command focused on autonomous weapon development, including drones.

In a January speech at SpaceX, Hegseth said slow Pentagon procedures could impede military AI development. “Our enterprise currently operates on staffing and committee cycles measured in months and years, and that’s unacceptable,” said Hegseth. “Military AI is going to be a race for the foreseeable future where the risks to U.S. national security of moving too slowly outweigh the impacts of imperfect alignment.”

Officials told the GAO that more access to expertise about the guidelines, or to staff who know them, would increase their ability to incorporate AI responsibly. The Department of the Responsible AI Division, which helps manage education on the guidelines, was understaffed. It was authorized to hire 10 people in 2023, but as of February, only three positions were filled, and the report stated that the Pentagon “is unlikely to fill vacant positions or increase staffing.” “That’s a very small number of folks tasked with a large responsibility of responsible AI throughout the department,” said Czyz.

The GAO issued two recommendations: the Pentagon should more clearly outline the responsibilities and meeting schedules of the office overseeing AI regulation, and it should address understaffing. The Pentagon agreed to both recommendations. As of Oct. 22, however, a GAO representative told NPR that the department had not implemented those changes. NPR said it did not receive a response when it asked when the Pentagon would make them.