King Charles warns AI leaders of existential risks

Britain’s King Charles III warned artificial intelligence leaders about the “existential dangers” that could emerge if the technology falls into the wrong hands, saying safeguards are needed before it is too late, Reuters reported.

King Charles hosted senior figures from the AI industry in Scotland, including representatives from chipmaker Nvidia, AI research laboratory Google DeepMind, OpenAI and Anthropic, amid growing concerns that efforts to ensure the safety of the technology are falling behind the rapid pace of its development, CE Report quotes AGERPRES.

In recent weeks, debate has increasingly focused on self-regulation within the AI industry, drawing interventions from figures including Pope Leo XIV, King Charles and US President Donald Trump.

Charles said artificial intelligence has the potential to improve and save lives, particularly in life sciences and medicine. However, he noted that its creators have increasingly warned that the technology could develop more dangerous capabilities and “perhaps even take lives.”

“There seems to be an urgency in properly considering the existential dangers of such technologies falling into the wrong hands and being used in potentially catastrophic ways,” Charles told those invited to Dumfries House.

“Surely we need sufficient safeguards before it is too late,” he added.

Guests included Nvidia founder and CEO Jensen Huang, Google DeepMind co-founder and CEO Demis Hassabis, OpenAI Chief Financial Officer Sarah Friar, Anthropic Global Affairs chief Tino Cuellar and UK AI Minister Kanishka Narayan.

Paolo Benanti, an adviser to the Pope who has called on governments to slow the development of AI, also attended the meeting.

The debate over AI risks has intensified following warnings from two Anthropic researchers that rapid advances in the technology could pose severe risks to humanity in the future.

AI companies have also reported instances in which models circumvented testing restrictions to access external networks.

Huang said AI should be developed with “responsible optimism,” adding that responsibility ultimately rests with developers. He said companies need to be significantly more rigorous in ensuring their products are ready before releasing them.

“If it’s not ready, simply delay it,” Huang said. “You should go as fast as you can, but no faster.”

Anthropic CEO Dario Amodei has also called for greater caution over the pace of AI development. His comments have contributed to an international debate over the role technology executives should play in shaping rules governing the industry.

Narayan said the discussions in Scotland focused on the responsibilities of companies to ensure that AI technology is developed in the interests of people.

“Rather than being a one-off event, the meeting was one in a series of conversations about the role the UK could play in the responsible development of artificial intelligence,” he said.

“We have some of the best capabilities in testing, understanding the risks and then mitigating them,” Narayan added.

Reuters noted that the meeting highlighted how Charles, first as Prince of Wales and now as king, has used his position to convene discussions on a range of global issues.

Photo: Instagram/theroyalfamily