China rebuffs US calls to slow AI development ahead of Trump-Xi summit

China’s third safety framework covers agents and recursive self-improvement

Over the last few weeks the mood music from Silicon Valley has grown increasingly sombre, as insider after insider has warned of the potential catastrophes that could result from “loss of control” of the most advanced AIs. Last Saturday, Anthropic chief executive Dario Amodei published an essay pitching a “worldwide pacing of the frontier” in AI development — something he said would “require cooperation with China.” The dispute comes days before President Donald Trump is set to meet counterpart Xi Jinping in Washington, with AI governance expected on the agenda.

The pace and quality of AI development in China is second only to the United States, with the two superpowers locked in an intensifying rivalry as the stakes rise with each new model’s capabilities. Analysts generally agree that frontier, or cutting-edge, US models are still some way ahead of their Chinese counterparts.

Amodei made his framing explicit, mentioning China specifically when arguing it was key “to keep democracies’ AI lead over autocracies as large as possible.” He advocated for measures including continued export controls to China on advanced chips and machinery — the existence of which some argue already amounts to a forced unilateral “pacing.”

“China is definitely not receptive” to a slowdown on US firms’ terms, said Lian Jye Su of technology research group Omdia. “It sees these calls as a bid to lock in US advantage,” Su added. For the Chinese government, AI is a “core sovereignty capability,” and Beijing will not want to jeopardise the progress it has made in closing the gap with US frontier models, according to Su.

China’s foreign ministry responded to Amodei’s piece by saying “fearmongering, confrontation and vicious competition will only disrupt the process of global AI governance.” However, several analysts told the Guardian it would be a mistake to assume that response indicated a rejection of the broader concerns Amodei raised. “In many ways, I think there is more common ground than people might assume between what Amodei is saying and what the Chinese government is saying,” said Gabriel Wagner, a researcher at Beijing-based AI safety consultancy Concordia AI.

This week China released the third iteration of its AI safety governance framework — a document that functions as “a mapping of the AI risk landscape and roughly what one should do about it,” according to Wagner. The updates cover the risks posed by agents, cybersecurity threats from frontier models, and the issue of recursive self-improvement — an as-yet hypothetical scenario in which AI becomes advanced enough to improve itself.

China’s regulatory posture contrasts with that of the United States. Trump this week called fears AI could wipe out humanity a “hoax.” Analysts are sceptical a major breakthrough on AI will emerge from the Trump-Xi meeting next week.

The ruling Communist Party, attuned to risks to its political control, has been proactive in ensuring the new technology has developed within boundaries set by the state. China’s centralized authority over critical infrastructure and commercial activity leaves it generally better placed than the United States “to address any AI disaster,” according to Su. Chinese companies, however, do less voluntary monitoring and evaluation work than their US counterparts, said Leia Wang of the Carnegie Endowment for International Peace. Neither government seems prepared for a loss-of-control incident like those outlined over the last few weeks, she said.

China’s regulatory approach is “iterative,” Wang added — “they are quick to adapt and release new measures.” China has maintained an algorithm registry since 2022, and warnings on agents released this year suggest a switch in focus “from controlling what AI says to controlling what it does,” per Concordia AI. Beijing also issued national regulations this year aimed at curbing emotional dependency on companion AIs.

At the same time, AI is being actively encouraged across the economy to promote growth, a goal enshrined in China’s latest Five Year Plan. Local governments are offering funding and low rents to attract companies.

About two years ago, education entrepreneur Boris But had what he calls a “penny-drop moment” — that, like countless others in China, his company needed to embrace AI before it was made obsolete by it. His clients, well-off parents looking to send their children to study overseas, had begun arriving in his office reeling off information and opinions gleaned not from rival firms, but from chatbots. In response, he started developing his own AI study tool, concluding: “If you can’t beat them, join them.” But was recently invited to pitch his company, Hampshire Education, to a small city three hours from Shanghai that is working to establish a reputation as a tech hub. Factories across China’s manufacturing base are being encouraged to convert their processes for the AI era.

“Safety and controllability are certainly major concerns, and they are actively discussed,” said a spokesperson for Black Lake Technologies, an industrial agent platform that says it has worked with 40,000 factories globally. “But in manufacturing, these concerns tend to be much more concrete and operational … what companies care most about is whether the AI is reliable.”

Daily life in China is already highly digitised, and the country’s traditionally limited public discourse around data privacy has smoothed adoption. Digital health service A-fu, which features AI chatbot versions of prominent doctors trained on real clinical studies, recently hit 150 million users. “Most people treat AI as an already-useful everyday tool, not an abstract threat, so the practical attitude is to live with it,” said Ni Tao, founding editor of tech website yangtzeer.com. “There is real anxiety about jobs and misinformation, but it coexists with a strong belief that China must master AI rather than slow down.”