Even the King of England Has His Hesitations About AI

Kings aren’t supposed to have hot takes. The whole point of a constitutional monarchy is staying above the argument, not wading into it. That’s what made it notable when King Charles III used a private AI summit on Thursday to warn some of the industry’s most powerful people that humanity needs real control over the technology, and soon. That’s not a garden-opening quote. That’s a monarch getting uncharacteristically specific about an industry most heads of state still discuss in vague, hopeful platitudes.

The setting was Dumfries House, the King’s estate in Cumnock, Scotland, and the guest list read like a fever dream of the AI industry’s org chart. Nvidia’s Jensen Huang was there. So was DeepMind’s Demis Hassabis, UK AI minister Kanishka Narayan, the head of Britain’s foreign intelligence service, and Paolo Benanti, the Vatican’s own AI advisor, because apparently even the Pope wanted eyes in the room. OpenAI sent its CFO, Sarah Frair. Anthropic sent Tino Cuéllar, its global affairs chief, who afterward praised the King for bringing industry and civil-society voices together to talk about AI’s role in the public good, the kind of line that could describe nearly any conference with a name tag table.

The speech itself leaned on “Harmony,” a philosophy the King wrote about back in 2010 emphasizing balance between humanity and the natural world, using it to frame AI’s rise. He noted that AI is already saving lives through medicine and science, then pivoted to the part that actually made headlines: the same people building these systems, he said, are now warning AI could develop far darker capabilities, up to and including the power to take life. His closing question to the room, reproduced in full on the Royal Family’s website, was blunt for a man whose day job usually involves ribbon-cutting: “Surely, we need sufficient means of control before it is all too late?”

This isn’t new territory for him. He sent a similar message to the 2023 AI Safety Summit at Bletchley Park and has returned to the theme since. What’s shifted is the temperature in the room he’s addressing. Weeks before this summit, an Anthropic researcher resigned and published a warning that AI labs are racing to build systems smarter than any human, and that humanity “may not survive this.” Language like that doesn’t stay contained to one corner of the internet. It filters, apparently, all the way up to Buckingham Palace’s speechwriting desk.

The Regulatory Reality Behind the Royal Rhetoric

Here’s the part that matters more than the pageantry: what is Britain actually prepared to do about any of this?

Since Bletchley Park in 2023, the UK’s approach has been almost entirely voluntary. Successive governments bet that light-touch, informal agreements with AI vendors would keep London attractive to labs and capital, rather than risk scaring them off with binding rules. Long-promised legislation targeting the most powerful frontier models still hasn’t materialized. Britain has instead reorganized its own bureaucracy around the problem: Prime Minister Andy Burnham dismantled the Department for Science, Innovation and Technology this summer, folding AI policy into a restructured business department while handing oversight of the UK’s AI Security Institute directly to the Cabinet Office.

Narayan, who picked up a literal cabinet seat in that reshuffle, recently told Reuters that regulation of advanced models remains on the table if the voluntary system proves inadequate. That’s a hedge, not a commitment, but a meaningful one from a minister whose government just spent months rewiring Whitehall specifically to give AI more institutional weight.

The King’s remarks landed in the middle of that ambiguity, and in the middle of an industry that can’t agree on how fast to move, either. Sam Altman, Elon Musk and Anthropic’s Dario Amodei called for a deliberate slowdown in AI development this month. Jensen Huang and President Trump dismissed the idea almost immediately, arguing that whichever nation leads in AI wins outright. Layered on top of that is the fallout from Anthropic’s decision earlier this summer to restrict its Fable and Mythos models to US customers only, a move that rattled London’s AI scene and revived long-simmering worries about Europe’s dependence on American labs.

None of that gets resolved by a monarch asking nicely from a remote corner of Scotland, however sincerely he means it. But it’s a useful gauge of how far this anxiety has traveled. When the one head of state whose entire job description is “don’t rock the boat” starts talking about existential risk and control before it’s too late, that’s a decent sign the debate has moved past the AI conference circuit and into the actual machinery of government, however slowly that machinery is choosing to move.