The co-founder of AI safety firm Anthropic has warned that artificial intelligence could reach a point where it develops without human input, calling for a 'brake pedal' to be built into the technology.
Jack Clark told BBC's Newsnight that the rapid pace of AI advancement risks creating systems that operate beyond human oversight. "We are approaching a stage where AI could effectively drive itself," he said. "We need a brake pedal — some mechanism to pause or slow down development when risks become too great."
“Anthropic co-founder Jack Clark warns AI could develop without human input and calls for a built-in 'brake pedal' to control risks, as UK regulators face pressure to keep pace with rapid technological change.”
His comments come as governments worldwide, including the UK, grapple with how to regulate a technology that is evolving faster than legislation. The UK government has positioned itself as a global leader on AI safety, hosting the first international AI Safety Summit at Bletchley Park in November 2023 and establishing the world's first AI Safety Institute. However, critics argue that regulatory efforts are lagging behind commercial deployment.
Clark's warning is particularly resonant for the UK, which is home to a thriving AI sector including companies like DeepMind and ventures backed by major tech firms. The UK also has a strong university research base in AI. Yet the very speed of progress, Clark suggests, could outstrip the ability of regulators and the public to keep up. "We are not just talking about better chatbots," he said. "We are talking about systems that could design other systems, potentially leading to exponential growth in capability without meaningful human control."
The concept of a 'brake pedal' is not new in AI safety discussions, but Clark's framing emphasises the urgency. He is not advocating for a halt to development, but for built-in safeguards that can be triggered automatically or by human overseers when certain thresholds of risk are detected. This might include limiting the autonomy of AI systems or requiring human approval for significant actions.
The UK's approach to AI regulation has so far been light-touch, favouring guidance over binding rules. But Clark's intervention adds weight to calls for more robust measures. The AI Safety Institute, based in London, is tasked with evaluating frontier models and advising the government. Its work could inform where a 'brake pedal' might be applied.
Looking ahead, Clark's warning will likely intensify debate at upcoming international forums on AI governance. The UK has committed to further summits and is pushing for global coordination. For now, the question remains whether the metaphorical brake pedal will be fitted before the car is going too fast to stop.