\n<\/p>\n
Evan Hubinger, Anthropic\u2019s head of alignment science, said several employees genuinely believe AI could pose an existential threat to humanity. He also acknowledged that the company has yet to find a clear solution for aligning AI systems\u2019 goals with human interests.<\/p>\n
Hubinger\u2019s comments came shortly after researcher Jacob Coxon resigned from Anthropic and accused the company of failing to act responsibly enough.<\/p>\n
Coxon, who has worked on pre-training research at both OpenAI and Anthropic, warned that competitors are moving dangerously quickly towards self-improving superintelligence and \u201cplaying with people\u2019s lives.\u201d<\/p>\n
AI Could Threaten Humanity Within a Decade<\/span><\/em><\/p>\n In posts on X, Hubinger said he and his colleagues \u201cgenuinely believe that AI could wipe out all humans\u201d, estimating that the probability of such an outcome exceeds 10% over the next decade.<\/p>\n He stressed that, despite Anthropic\u2019s efforts, there is still no clear plan to solve the problem of \u201csuperintelligence alignment\u201d.<\/p>\n Coxon separately argued that people working on AI development believe the technology could threaten humanity by the end of the decade, insisting that this is not a \u201cpublicity stunt\u201d.<\/p>\n He also criticised OpenAI, saying it had not fully understood what was at stake for civilisation. At the same time, he said Anthropic had a better understanding of the risks but remained trapped in a competitive race with other AI companies.<\/p>\n Concern Focuses on Self-Improving Superintelligence<\/em><\/span><\/p>\n Responding to the concerns, Hubinger referred to Anthropic\u2019s latest risk assessment, which concludes that current AI models pose a low level of risk.<\/p>\n His main concern, however, involves the development of superintelligence through recursive self-improvement, which he said is progressing faster than researchers initially expected.<\/p>\n Growing Calls for AI Regulation and Oversight<\/span><\/em><\/p>\n The developments come as leading AI researchers and industry executives increasingly call for a slowdown in the development of advanced systems.<\/p>\n In July, prominent figures including Anthropic co-founders Dario Amodei and Jared Kaplan, OpenAI chief scientist Jakub Pachocki and Meta AI chief scientist Shengjia Zhao signed a statement warning that AI development may need to slow down to allow adequate safety and oversight measures to emerge.<\/p>\n The statement called for international cooperation and government intervention to establish safeguards around the uncontrolled development of automated AI systems.<\/p>\n In a recent post, Pachocki reiterated that no AI laboratory has fully solved the challenges of alignment and monitoring.<\/p>\n He warned that the industry cannot continue expanding AI systems at such speed indefinitely without adequate mechanisms to monitor and control the technology.<\/p>\n \u00a0<\/p>\n