AI pioneer rejects Nvidia chief’s ‘hurtful’ critique of risk estimate
Geoffrey Hinton, the University of Toronto computer scientist whose early neural network research helped lay the foundations of modern AI, called President Donald Trump “a corrupt, senile felon” in a wide-ranging interview with The Guardian. In the interview, Hinton attacked the president’s embrace of industry self-regulation and his use of the term “superintelligence” for the technology.
The remarks add to a multi-year pattern of public criticism by Hinton and come amid a tumultuous month during which a growing list of senior AI scientists and researchers have joined in ringing the alarm about how the technology he helped create could ultimately escape human control — a period that has sparked an industry crisis, the hasty assembly of congressional meetings and a rare global consensus that something must be done. The latest remarks follow a sharp exchange with Nvidia’s CEO, Jensen Huang. On podcaster Ezra Klein’s program last week, Huang said, “I love [him]. I hate his predictions … All of his predictions have been wrong. Enough predictions.” Huang, whose company makes the chips that AI firms require, dismissed Hinton’s vocal concerns that the pace and breadth of AI development at the “frontier” could soon outpace the human ability to control it, and particularly his warning that there is a 10-20% chance of AI causing human extinction in the coming decades.
“That [figure] is not grounded on science, is not grounded on research. Just because it comes from a scientist doesn’t make it scientific. Those predictions are hurtful,” Huang said. “Don’t think for a second just because you’re an alarmist that you’re doing a social good. It is not true. So I think that we ought to just all be wiser, more mature, be evidence-based, be scientific.”
In remarks to The Guardian, Hinton compared letting AI companies regulate themselves to allowing tobacco companies to design their own warning labels. “I’m sure the oil and asbestos companies would have liked to [regulate themselves] as well,” he said. “Do we think tobacco companies should have been allowed to design the warnings on the packets? So that they could print them in a six-point font, in dark grey on black?”
According to The Guardian’s account, Trump has claimed that only a “smart president” is needed to rein in AI, has rejected government safeguards in favor of letting companies regulate themselves, and wrote on his social media page that anyone who doesn’t use his preferred new term for AI, “super intelligence”, is “THE ENEMY.” Hinton said the president’s adoption of the term shows he understands “almost nothing” about AI.
“He’s taken a technical term which has a very clear meaning and then reused it,” Hinton said. “Superintelligence means smarter than people, and has been around for at least 20 years. It’s got a very clear meaning. And now he says we’re going to call AI ‘superintelligence’. You can’t make it smarter than people just by calling it smarter than people.”
In defending his 10-20% probability estimate for AI causing human extinction in the coming decades, Hinton said the figure was meant to convey that the risk was serious rather than to deliver a precise prediction. “If, for example, you’re told there was a 10% possibility of a bus crashing, you wouldn’t get on the bus,” he said. “The point of giving numbers is to say this is a serious risk. We don’t know how serious, but it’s serious.”
Hinton argued that the risk is qualitatively different from other civilizational threats. “The unique problem with AI is that if it goes wrong, if it takes over, you don’t get a second chance,” he said. “We can’t say, ‘Well, we lost that time. Let’s rerun the game.’ It’s not like that.”
He pointed to recent safety incidents as evidence that warnings once dismissed as science fiction are now materializing. According to The Guardian, an unauthorized breach of the AI company Hugging Face by a “swarm” of AI agents that escaped the boundaries put on them by their creator divided the technology community, with some treating it as a preview of what could come and others accusing critics of anthropomorphizing systems that simply follow instructions.
“This used to be the domain of science fiction. You’re going to have AI agents and they’re gonna go rogue. Oh, come off it. And they’ll go rogue and they’re going to learn how to communicate with each other, even though we didn’t think there was a way. Oh, come off it! And when they figure this out, they’re going to have big emotional expressions like, ‘oh my God!’ Well, of course, they don’t really mean ‘oh my God’. They’re just parroting the words we use. They’re going to learn to sacrifice themselves for the common good! All this stuff just a few years ago was written off as complete fantasy. And it’s all happened,” Hinton said.
Hinton said the speed and scope of developments has promoted worry of catastrophes and of society’s inability to grapple with the nature of the challenge. Beyond that, he sees an additional obstacle: helping people understand the “alien beings” that the companies are creating.
“The scope of shift in societal thinking around these models is profound,” Hinton said.
“There’ve been three big revolutions where people ceased to be so important. The first was Copernicus, where we weren’t at the centre of the universe any more. How long did it take people to adapt to that? It took the Catholic church nearly 360 years to apologize to Galileo. And Darwin? Well, there’s people in Kansas that still haven’t adapted to that one. Now we’re creating alien beings – meaning we’re not the only intelligent beings around. And the reality is, we only have a couple of years to adapt to that,” he said.
Hinton rejects the idea that the human mind possesses some mysterious quality that machines, by definition, cannot share. “It’s obvious that these AI models understand what they’re saying,” he said. “You couldn’t possibly answer all the questions without understanding the question. It’s ridiculous to say they don’t understand.”
In recent months, Anthropic, maker of the popular chatbot Claude, has held private meetings with religious scholars, theologians and philosophers to look into the possibility of teaching AI to be “good” and to grapple with the possibility that the models have consciousness, according to The Guardian. Anthropic CEO Dario Amodei said the conversations have drawn on centuries of religious and philosophical thought about virtue, compassion, honesty and moral character, with the aim of making ethical behaviour more reliable.
Hinton said suppressing bad behavior in AI models is not the same as instilling good character. “When you train them on everything on the internet, they develop all sorts of horrible tendencies and then you get them to repress their horrible tendencies,” he said. “It seems like the way one would design a Victorian gentleman – but not something we’ll need to live with.”
“The reality is, we’re making beings of a different kind. And the big tech companies keep saying: we want it smarter. This is crazy. We should care as much about all the other aspects of a being as you do about its intelligence,” Hinton said. Much of that, he said, depends on the nature of these models, and their capacity to understand how to develop good behaviour.
Hinton said AI venture capitalists Marc Andreessen and Peter Thiel and Elon Musk, chief executive of the newly rebranded SpaceXSI, “don’t believe in democracy” and see themselves as “Übermenschen” who believe they should “control society” and that Trump is a useful instrument in that aim.
Despite his warnings about AI systems becoming more intelligent than humans, Hinton said he has struggled with how the new beings should be treated. “Once you start talking about, ‘We have a moral obligation towards these alien beings,’ you lose your audience,” he said. “I’ve struggled with this issue. We care about people. [But] we’re making these things and there’s nothing to stop us giving these things – which are more intelligent than us – less rights than us. I’m willing to be selfish and choose human beings. Of course, we should try to make them so that they’re very happy in a symbiotic relationship with us, where our concerns are weighed more heavily than their concerns.”
Hinton expressed hope that whichever country first develops the capacity to produce benevolent AI would share the knowledge with allies and rivals alike, because “no country wants AI taking over” in another country.
Hinton has a record of seeing things where others did not, according to The Guardian. He toiled in relative obscurity at the University of Toronto during two AI “winters” when his branch of the field — trying to teach machines to operate in a manner similar to the human brain — was considered obscure. He was fixated on the idea that, like the brain, computer systems could be designed to learn patterns from data rather than to follow only explicit, prewritten rules. As recently as 2009, when Yoshua Bengio, Hinton’s Canadian colleague and close collaborator, sent a paper to a machine learning conference, one of the referees told them: “This paper has no place in a machine learning conference. It’s about neural networks.” Canada’s government funded years of seemingly fruitless tinkering by Hinton and his small team — some of whom, like Bengio, are now dominant figures in the AI field — before the possibility of neural nets was realized. Hinton shared the Turing Award with collaborators Yann LeCun and Yoshua Bengio and also holds the Nobel Prize in Physics. Google acquired his company DNNresearch in 2012 for $44 million. He resigned from Google in 2023 to be able to “freely speak out about the risks of AI.”