Concerns over artificial intelligence and humanity’s safety have intensified at Anthropic after a senior researcher warned that advanced AI could eventually threaten human existence.
Evan Hubinger, Anthropic’s head of alignment science, said several employees genuinely believe AI could pose an existential threat to humanity. He also acknowledged that the company has yet to find a clear solution for aligning AI systems’ goals with human interests.
Hubinger’s comments came shortly after researcher Jacob Coxon resigned from Anthropic and accused the company of failing to act responsibly enough.
Coxon, who has worked on pre-training research at both OpenAI and Anthropic, warned that competitors are moving dangerously quickly towards self-improving superintelligence and “playing with people’s lives.”
AI Could Threaten Humanity Within a Decade
In posts on X, Hubinger said he and his colleagues “genuinely believe that AI could wipe out all humans”, estimating that the probability of such an outcome exceeds 10% over the next decade.
He stressed that, despite Anthropic’s efforts, there is still no clear plan to solve the problem of “superintelligence alignment”.
Coxon separately argued that people working on AI development believe the technology could threaten humanity by the end of the decade, insisting that this is not a “publicity stunt”.
He also criticised OpenAI, saying it had not fully understood what was at stake for civilisation. At the same time, he said Anthropic had a better understanding of the risks but remained trapped in a competitive race with other AI companies.
Concern Focuses on Self-Improving Superintelligence
Responding to the concerns, Hubinger referred to Anthropic’s latest risk assessment, which concludes that current AI models pose a low level of risk.
His main concern, however, involves the development of superintelligence through recursive self-improvement, which he said is progressing faster than researchers initially expected.
Growing Calls for AI Regulation and Oversight
The developments come as leading AI researchers and industry executives increasingly call for a slowdown in the development of advanced systems.
In July, prominent figures including Anthropic co-founders Dario Amodei and Jared Kaplan, OpenAI chief scientist Jakub Pachocki and Meta AI chief scientist Shengjia Zhao signed a statement warning that AI development may need to slow down to allow adequate safety and oversight measures to emerge.
The statement called for international cooperation and government intervention to establish safeguards around the uncontrolled development of automated AI systems.
In a recent post, Pachocki reiterated that no AI laboratory has fully solved the challenges of alignment and monitoring.
He warned that the industry cannot continue expanding AI systems at such speed indefinitely without adequate mechanisms to monitor and control the technology.
