TORONTO – If humanity wants to avoid a doomsday scenario, just slowing down AI development won’t cut it.
That’s the warning from Montreal-based AI researcher David Krueger, who was blunt about what he thinks has to happen instead.
“What we need is an immediate, indefinite international moratorium on frontier AI development,” said Krueger, an assistant professor of AI at the University of Montreal and a core academic member at Mila, Quebec’s artificial intelligence institute. His biography describes his work as “reducing the risk of human extinction from artificial intelligence.”
“We need to stop building more powerful AI now.”
His comments come after Anthropic CEO Dario Amodei published an essay Saturday calling on AI companies to slow the pace of development. The proposal drew support from OpenAI’s Sam Altman and SpaceX’s Elon Musk, a rare display of unity among some of the industry’s biggest names.
“Calls to slow down AI are very, very welcome,” Krueger said, while taking issue with what’s been proposed. “The particular details of these plans, I disagree with in various ways. I think they’re not going far enough, fast enough.”
Amodei has called for regulation and international co-operation, along with third-party evaluators who would have “employee-level” access to AI companies to verify safety practices and report incidents.
Krueger, who also founded the non-profit Evitable to educate the public about the severe risks of AI, worries that what Amodei has asked for is inadequate and that AI testing methods are no longer reliable.
“The tests basically don’t work, and they’re going to get less and less reliable as the AI gets smarter, because the AI can basically pretend,” he said. “It can play nice, it can play dumb, and it can fool our testing.”
‘Just a matter of time’
Last week, an Anthropic senior safety researcher put the odds of AI killing every human on Earth at more than one in 10 over the next decade. Some have dismissed the figure as exaggerated and aimed at grabbing attention, but Krueger calls the number “way too low.”
“If we don’t course correct, it’s more likely than not that humanity will be permanently disempowered from rogue AI within the next five years,” he said. “And then human extinction at that point is just a matter of time.”
By disempowered, he means a world in which AI is effectively running society, much the way humans now dominate every other species on the planet.
“AI (would be) controlling physical infrastructure, including every step of the process of building more AI and building robots and other physical systems,” he said.
“Imagine a world where all companies are run by AI, where there are robots all over the place, where basically humans don’t have any work except for nominal work, and even politicians and everyone is deferring to AI because the AI just seems to have such good judgment,” he added. “That’s the kind of future that we’re headed towards.”
He warns the window to act is narrow. Today, he said, a rogue AI system could still be unplugged. But once one gets loose, it could copy itself across many computers, and “we wouldn’t even know which computers to shut down.”
“We’d have to try and shut down all of them, and that becomes very, very difficult,” he said. “So, we have a very limited time window in which to act to prevent those kinds of things.”
‘It’s a state of emergency’
Still, he says he is more optimistic than ever that governments will act, calling the past week the first time in 15 years of warnings that “the world is really responding in a sane manner.”
“I think people are actually finally understanding just how serious of a crisis this is. It’s a state of emergency.”
In an interview with CTV News chief political correspondent Vassy Kapelos on Monday, federal AI minister Evan Solomon was asked whether he agreed there is a 10 per cent chance humanity could be destroyed by AI.
“Are there risks to AI? Absolutely,” he replied. “Yes. We know about these risks.”
Solomon said he spoke with OpenAI CEO Sam Altman on Sunday morning.
“AI will be regulated to keep Canadians safe,” Solomon said. “It is our No. 1 responsibility. It is our No. 1 concern.”
Solomon also said he will make an AI safety announcement on Wednesday alongside Yoshua Bengio, a Canadian computer scientist known as one of the “Godfathers of AI.”
