From King Charles to U.S. President Donald Trump, Prime Minister Mark Carney and top AI executives, the debate over artificial intelligence safety risks and whether to slow down leading-edge research is moving fast on the heels of recent warnings from within the tech industry.
At the same time, prominent AI advocates, like the CEO of Toronto-based Cohere, say “fearmongering” is currently overshadowing the technology’s upsides, such as potential breakthroughs in medicine and clean energy.
WATCH | The ‘godfather of AI’ says regulation is needed:
‘We’ve begun to realize just how dangerous’ AI is, says Nobel laureate | Hanomansing Tonight
Geoffrey Hinton, the ‘godfather of artificial intelligence’ and a Nobel Prize laureate, warned U.S. Congress this week that there may only have a year left to regulate AI before it gets out of control. He answers viewer concerns about artificial intelligence development.
Why so much fear about AI?
Several fear factors are at play. Powerful AI models can execute self-directed cyberattacks. Such was the case in July, when OpenAI agents hacked into a rival firm. This led to recent pleas for regulation from some within the tech industry.
Others, like former Google CEO Eric Schmidt, have long warned that AI doesn’t need to go rogue to be dangerous. This cohort worries modern AI tools will lower barriers for engineering novel bioweapons without traditional laboratories or hasten development of autonomous weapons systems.
Meanwhile, staunch proponents of AI technology, like Trump and Meta CEO Mark Zuckerberg, argue that slowing down research on the most potent models would effectively cede technological leadership to China. Others on this side of the debate, like venture capitalist Marc Andreessen, believe a pause would cost lives via preventable diseases and medical errors, as well as delayed scientific discoveries.
Calls to place limits on AI are nothing new.
In March 2023, four months after OpenAI released ChatGPT, an open letter calling for a pause on training certain AI systems received high-profile endorsements from Elon Musk, Apple co-founder Steve Wozniak and Canadian AI pioneer Yoshua Bengio.
Who is for an AI slowdown?
OpenAI & Sam Altman
“We do not believe that the AI industry has solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer,” the San Francisco-based company wrote in a company blog post on Wednesday.
“I think the world is right to be afraid of this,” CEO Sam Altman added in a recent interview.
Background: On Wednesday, OpenAI disclosed six reports of “unexpected or concerning” behaviour in AI models that it observed in recent months. The company develops cutting-edge generative models — most notably ChatGPT.
Anthropic CEO Dario Amodei
“We must slow the pace at which we improve the capabilities of AI models. Progress will still seem fast, and we must make wise use of the time we gain,” Amodei wrote in an online essay published last Saturday.
“My own father died of a disease that was cured just a few years after his death, and I myself survived an early-stage cancer that would not have been treatable even fifty years ago. Carefully wielded, AI can be the latest in a long line of technological miracles that have uplifted and ennobled humanity.”
Background: Amodei has clarified that he does not support a total or permanent freeze on AI research. He advocates for a structured “pacing” strategy to ensure risk mitigation and alignment techniques keep up with raw capability.
King Charles
King Charles walks with Nvidia CEO Jensen Huang to a meeting for AI tech leaders at Dumfries House in Cumnock, Scotland, on Thursday. (Jonathan Brady/pool/Reuters)
While King Charles has not explicitly called for a slowdown, he says he wants to see tighter controls.
“There seems urgency in adequately considering the existential dangers of such technologies falling into the wrong hands, and being used in potentially catastrophic ways,” King Charles said at an AI event in Scotland on Thursday attended by industry representatives from Nvidia, Google, OpenAI and Anthropic.
“Surely we need sufficient means of control before it is all too late?”
Background: This isn’t his first warning on the potential dangers of artificial intelligence. In 2023, the British monarch called AI a discovery on par with electricity and splitting the atom at an event held at Bletchley Park, a site made famous by codebreakers during the Second World War. However, he warned the world must brace for “unintended consequences.”
Who is against an AI slowdown?
Nvidia CEO Jensen Huang
“You should go as fast as you can, but no faster than that,” Nvidia founder and CEO Jensen Huang said on Thursday, at an AI conference in Scotland. “If it’s not ready, just hold it back.”
Background: Nvidia makes chips for training and running complex AI models. The company has seen its value explode in recent years amid a wave of investor enthusiasm for artificial intelligence and the hardware needed to power it.
U.S. President Donald Trump
When it comes to AI, Trump says slowing down American innovation or imposing rigid guardrails would hand a strategic advantage to foreign rivals. This dynamic will be on full display during next week’s planned meeting between Trump and Chinese President Xi Jinping in Washington.
“We’re leading China in AI, we’re the most sophisticated country in the world, and frankly, I want to keep it that way because whoever wins AI, wins,” Trump told reporters on Sunday. “We can put guardrails, we can do this and that. But I think you have a lot of negative forces that are bringing it up that shouldn’t be bringing it up.”
Trump did not specify the “negative forces” he referred to.
China’s state security minister says advanced U.S. models such as Anthropic’s Mythos and OpenAI’s GPT-5.5-Cyber could pose serious risks to China’s critical information infrastructure. That’s according to a report from Reuters on Monday.
Cohere CEO Aidan Gomez
WATCH | Cohere CEO says Canada has a rare ‘ticket to the AI race on a global stage’:
‘Canada deserves a champion’: CEO on the global AI race
Critics of the pace at which AI development is taking place say more safeguards are needed to protect users, should AI models evolve beyond human control. It’s something Aidan Gomez, CEO of the Toronto-based AI firm Cohere, is also calling for, as well as for Canada to assert its sovereignty in the global AI race.
Cohere CEO Aidan Gomez criticized the push by some U.S. tech executives for an industry-led slowdown and regulation of AI in a recent blog post, arguing it’s a veiled attempt to “bend competition rules” and dictate terms to other companies.
Gomez told CBC News in an interview Tuesday that he sees room for governments to regulate AI safety, but blasted comments from a researcher at Anthropic who pegged the odds of AI killing “all humans” within the next decade at about 10 per cent.
“I think the extinction risk-type talk is fearmongering,” he said. “It’s born from sci-fi stories and Terminator-like scenarios that can cause a lot of fear, and a lot of misunderstanding and damage.”
Background: Unlike companies like OpenAI, which are focused on consumer apps like ChatGPT, Toronto-based Cohere builds tools for businesses to handle complex tasks and big collections of documents. On Wednesday, the company finalized a merger deal with Germany’s Aleph Alpha, which does similar work.
Meta CEO Mark Zuckerberg
“Every lab has the responsibility and incentive to move at the pace required to train its models safely, and the ability to take its own actions to ensure that happens,” Zuckerberg wrote on the social media platform X on Tuesday.
Background: Meta, owner of Facebook, is heavily focused on leadership in AI, committing tens of billions of dollars toward custom AI infrastructure.
What does Ottawa say?
Prime Minister Mark Carney, shown at the European Parliament in Strasbourg, France, on Thursday, has pitched the idea of a ‘technology stability’ organization to better co-ordinate internationally on advances in AI. (Justin Tang/The Canadian Press)
While Prime Minister Mark Carney hasn’t explicitly weighed in on the prospect of an AI slowdown, he has pitched the idea of creating a “technology stability” organization.
“There’s a need for co-ordination,” Carney said in an interview with Bloomberg News on Monday. “Ultimately, a technology stability board, in our view, along the lines of the Financial Stability Board, would make sense.”
Based in Basel, Switzerland, the Financial Stability Board monitors and makes recommendations about the global financial system. It has 71 members representing institutions around the world, including Canada.
Asked about the idea of an AI research slowdown, Evan Solomon, Canada’s federal minister for AI and digital innovation, would only say that safety is the government’s top priority.
“Safety has been our number one concern, the safety of Canadians to make sure that the risks of any technology, including AI, are mitigated,” Solomon told CBC News in an interview on Tuesday.
“We’re setting up, for the first time in Canadian history, a regulator that will have teeth to hold big companies, tech companies accountable.”
On Wednesday, the federal government announced a $150 million investment in LawZero, a Montreal-based non-profit launched by Bengio to research AI safety.