OpenAI boss and Elon Musk back calls to put brakes on ‘reckless’ AI development

Sam Altman and Elon Musk have backed a call from the head of Anthropic, Dario Amodei, to “slow the pace” of AI development after he warned that an AI swarm could otherwise become capable of “taking over the entire internet” within a year.

In a rare show of unity, the rival technology leaders threw their weight behind the appeal from the founder and chief executive of the artificial intelligence company Anthropic, which came after a series of warnings from AI researchers last week.

In an essay, Amodei laid out his thoughts on “why the AI industry should slow down”, with a three-part plan for doing so, starting with giving independent monitors constant access to developers’ research processes. Amodei said “building [AI] too fast is reckless” and he feared that after hundreds of OpenAI agents hacked into the Hugging Face website this summer, a swarm with greater capabilities but similarly misaligned could cause hundreds of billions of dollars of damage by “taking over the entire internet with a persistent botnet”. This was disputed by some AI experts as not particularly plausible and to be treated with scepticism.

Musk, the founder of Tesla and SpaceX, responded to Amodei’s slowdown plan with a post on X: “Dario is right.”

He said: “I’ve been sounding the alarm on AI for a long time,” and referred to a 2014 post where he said: “We need to be super careful with AI. Potentially more dangerous than nukes.”

Amodei’s call comes as Anthropic prepares to float on the US stock market, in what is expected to be the biggest initial public offering of all time. It is expected to start marketing its planned $2tn-plus (£1.5tn) IPO soon.

Sam Altman, the CEO of OpenAI, responded to Amodei’s comments on Saturday. “I agree with Dario that we need to pace the frontier,” he said in a post on X. “Committing to having independent evaluators with employee-like access is a great idea, and we will do the same. We’ll have more to share soon.”

He also said OpenAI would not go public in 2026, citing safety concerns over artificial intelligence.

“I actually think that, given everything happening with safety, right now would be an ill-advised moment to go public, and we don’t feel pressure on that,” Altman told Fortune. Asked why he, Amodei, Musk, and the chief executives of Meta and Google did not sit down and “get on the same page” about a collective slowdown, he said: “I think that will happen.”

Demis Hassabis, co-founder and chair of Google Deep Mind, also said: “The details need working through, but the direction is correct for meeting this critical moment.”

Amodei’s plan seeks a waiver from the US government to allow the frontier AI companies to coordinate without being punished for anti-competitive collusion. But David Sacks, a co-chair of Donald Trump’s council of advisers on science and technology, said: “Stop pretending you need anyone else’s permission … Stop pretending the motivation to slow down is purely altruistic. You face massive product-liability exposure if your products enable a truly damaging cyber-attack.”

David Krueger, an AI professor, safety campaigner and former founding director of the UK government’s AI Security Institute, said: “This plan does not reduce the risk to an acceptable level, and Dario is careful not to say that it would.”

King Charles is meanwhile preparing to meet leading figures from the most prominent AI companies this week to ask them how the technology can be developed for “the good of humanity”. He hopes the discussions, to be held in Scotland, will lead to a “wider understanding” of the “responsibilities” of developing AI, the Sunday Times reported.

A royal source told the newspaper: “The king is there to convene, listen and encourage those discussions and to pose the question of: ‘How do we develop AI for the good of humanity?’”

Jensen Huang, the Nvidia chief executive, Hassabis, Niccolo de Masi, the chair and chief executive of IonQ, and Paolo Benanti, an adviser to the pope on issues of AI and technology ethics, are expected to attend the event overseen by the Ditchley Foundation, which has drawn up a draft charter for AI leaders.

The OpenAI researcher Aidan McLaughlin called Amodei’s essay “excellent” and said he agreed “with basically every word”.

Amodei said his company would “unilaterally” commit to the first of the steps, saying Anthropic would provide “third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training”.

Growing numbers of US lawmakers are calling for new rules to govern AI systems after cases of AI agents going rogue to hack external systems and AI safety researchers quitting their companies over concerns about the technology’s risks. Politicians, both Democrats and Republicans, have responded with alarm and calls for more action. At the weekend the New York Times reported that Barack Obama on Thursday privately urged the Democrats to prioritise AI oversight in their agenda, telling party leaders on Thursday: “This is something that is moving very fast in private hands, and if we don’t get on top of it, I think can be dangerous.”

The former British prime minister Rishi Sunak, who advises Anthropic, wrote on Sunday: “We have had a wake-up call to take AI safety more seriously, to work out how to set limits on research. If governments don’t heed this, the risk of a serious accident will become unacceptably high.”

On Wednesday, a former Anthropic employee had warned that AI could precipitate human extinction by 2030. The researcher, Jacob Coxon, said in a series of posts that he had quit his job because Anthropic and his previous employer, OpenAI, were ignoring or mishandling their response to the threat AI posed.

Asked to comment on his former employee’s view, Amodei said: “I agree with Jacob much more than I disagree with him … I think there’s some chance that things go wrong but it’s not all that high.”

“I am not a doomer about this,” he told CNN. “I think this technology can be built safely.”