Jack Clark, co-founder of artificial intelligence company Anthropic, has warned the technology is approaching a stage where it could develop without any human input — and said the industry urgently needs a “brake pedal”.
Speaking to BBC Newsnight, Clark said: “You want the option to be able to take your foot off the gas and put your foot on the brake. Right now, it’s like the AI industry has a gas pedal, but it doesn’t have a brake pedal.” He stressed that people, through government policy, must keep control of AI systems, which will only become more powerful and have broader impacts on society. “The world needs to do some thinking and we need to eventually develop some new regulations that allow us to be confident in these systems,” he said.
“Anthropic co-founder Jack Clark warns AI could soon develop without human input and needs a 'brake pedal'.”
Clark’s warning comes as his own company’s popular chatbot, Claude, already operates on code of which 80% was written by the system itself. He told the BBC that reaching 100% is possible within two years and “would have huge implications”.
Drawing a historical parallel, Clark compared the current state of AI to the oil boom and barons of the turn of the last century. “Society’s response was to come up with a sensible policy and regulatory framework that gave people confidence in oil and the benefits that oil could provide to the world, and meant that you didn’t have to worry about the personalities of the people leading the companies,” he said. “That’s clearly where we end up here.”
Yet, despite his call for regulation, Anthropic this week welcomed an executive order on AI from US President Donald Trump that was relatively hands-off. The order did not require AI companies to submit to safety testing by the government; such testing remains a voluntary effort. Major AI firms pursuing advances — including Anthropic, OpenAI and Google — have also not said they will pause their own research.
Clark insisted his motivation for publicly discussing the growing capability of AI technology is not to burnish Anthropic’s reputation with paying customers. He said he simply wants to “tell the world what we’re seeing inside these companies with this unusual technology”.
Since its founding five years ago by chief executive Dario Amodei, Clark and a handful of other executives, Anthropic has positioned itself as outspoken about potential risks stemming from AI. The company is now preparing to debut on the public stock market, poised to be one of the first public listings by a newer AI firm and one of the most valuable stock listings in history. Private investors estimate Anthropic’s valuation at nearly $1tn (£745bn).
As the company races toward that landmark, the question of whether a “brake pedal” can be fitted in time remains unanswered.