AI researcher Jacob Coxon has resigned from Anthropic with an unusually stark warning: he believes the competition to build more capable artificial intelligence systems is moving faster than the safeguards needed to control them.
Coxon previously worked at OpenAI and spent about three years at Anthropic. In a public statement and interviews reported by the Associated Press, Wired and The Guardian, he described the next one or two years as “crunch time” and argued that leading laboratories are caught in a race that makes it difficult for any one company to slow down.
The warning spread widely because it comes from somebody who worked inside two frontier AI companies. It also contains the most frightening possible claim — that advanced systems could contribute to human extinction this decade.
That claim needs exact framing. Coxon is expressing a risk assessment, not reporting a scheduled event. Other Anthropic researchers have discussed double-digit probabilities for catastrophic outcomes, but those numbers are judgments under deep uncertainty. They cannot be checked in the way a weather forecast or election result can.
What Coxon says he fears
The core concern is not that today’s chatbot will become irritated by a spelling correction and launch a missile. It is that increasingly autonomous systems could become capable of planning, coding, persuasion and cyber operations across long time horizons, then be deployed before researchers know how to make their behaviour reliably match human intentions.
Coxon argues that competitive pressure makes the problem harder. If one company believes a rival — or a state-backed laboratory — is close to a major breakthrough, delaying its own model can look commercially and strategically dangerous. Every participant can sincerely prefer caution while the combined incentives still produce a sprint.
He has called for coordination and stronger oversight rather than trusting individual companies to solve the dilemma alone.
What Anthropic says
Anthropic presents itself as a safety-focused company and points to work on interpretability, evaluations, responsible scaling policies and public discussion of model risks. In reporting on Coxon’s departure, the company defended its approach and said serious safety research is central to its work.
That does not end the argument. Critics say a laboratory cannot be both an impartial judge of acceptable risk and a commercial participant spending billions to build the next system. Supporters reply that people with direct access to powerful models are best placed to understand and reduce the danger.
Both points can be true. Internal expertise is essential, and independent scrutiny is necessary precisely because incentives are not neutral.
Why experts disagree so sharply
Some researchers see rapid progress in coding, reasoning and agent-like behaviour as evidence that highly capable systems could arrive soon. Others believe current methods remain brittle, dependent on human-generated data and far from the broad competence implied by “superintelligence”.
There is also disagreement about the pathway from capability to catastrophe. A system might cause harm through misuse by people, unintended autonomous action, concentration of power, cyberattacks, biological design assistance or disruption to critical infrastructure. These are different risks requiring different evidence and controls.
Combining them into a single percentage can create false precision. Yet dismissing all of them because the probability is uncertain would also be poor risk management. Governments regulate aviation, medicines and nuclear materials without demanding proof that a disaster is guaranteed.
What useful policy would look like
The practical questions are less cinematic than the extinction headline:
- Which capabilities require independent testing before release?
- Who must report dangerous model behaviour or security failures?
- What access should external auditors receive?
- How can whistleblowers raise concerns without sacrificing their careers?
- Which uses — such as autonomous cyber operations or biological weapon design — should face firm restrictions?
The UK has invested in AI safety testing but still faces the central issue of legal authority. Voluntary agreements can move quickly, while enforceable rules provide accountability when commercial pressure rises.
For more coverage, visit OutOut’s Technology section and our report on Apple’s new foldable iPhone, which represents the consumer-facing side of the same accelerating industry.
The OutOut verdict
When people building the machines leave the building to warn about the machines, listening is not hysteria. It is basic due diligence.
That does not mean converting a researcher’s personal probability into a countdown clock. The responsible response sits between “the robots kill everyone on Tuesday” and “the market will sort it out”.
Coxon’s departure adds weight to demands for independent testing, incident reporting and rules that do not disappear when a competitor announces a larger model. If the industry genuinely believes the technology could transform civilisation, it cannot complain that civilisation would like a proper risk assessment.
