The co-founder of one of the world’s leading artificial intelligence companies has warned that the technology is approaching a point where it could develop without any human input, as he called for an urgent “brake pedal” to slow its progression.
Jack Clark, who co-founded Anthropic, the company behind the popular chatbot Claude, told BBC Newsnight that the AI industry has “a gas pedal, but it doesn’t have a brake pedal”. He stressed that people, through government policy, need to retain control of systems that will only become more powerful.
“Anthropic co-founder Jack Clark warns AI could develop without humans, calling for a 'brake pedal' to slow progress.”
“The world needs to do some thinking and we need to eventually develop some new regulations that allow us to be confident in these systems,” Clark said.
His warning comes as Anthropic prepares to debut on the public stock market, with private investors valuing the firm at nearly $1tn (£745bn), making it one of the most valuable stock listings in history. Despite its rapid growth since being founded five years ago, the company has positioned itself as a vocal advocate for addressing AI risks.
Clark highlighted the accelerating pace of the technology: Claude, Anthropic’s chatbot, is already operating on code that is 80% written by the system itself. Reaching 100% could happen within two years, he said, which “would have huge implications”.
He drew a parallel between AI and the early oil industry, arguing that society responded to the oil boom with a regulatory framework that gave people confidence and removed the need to worry about the personalities leading companies. “That’s clearly where we end up here,” he said.
Yet, despite these concerns, Anthropic this week welcomed an executive order on AI from US President Donald Trump that was relatively hands-off, requiring no mandatory safety testing by the government. Major AI companies, including Anthropic, OpenAI and Google, have also not said they will pause their own research.
Clark said his motivation for speaking out is not to boost Anthropic’s reputation. “We just want to tell the world what we’re seeing inside these companies with this unusual technology,” he said.