Safety concerns inside the artificial intelligence industry have resurfaced after a former Anthropic researcher publicly resigned, claiming developers are moving too quickly toward powerful systems without proper controls.
Jacob Coxon, a 27-year-old AI researcher who recently left the company, criticized the current speed of technology development in a resignation thread posted on X on Tuesday.
”The people building AI earnestly believe that it could kill us all by the end of the decade,” Coxon posted. “They are racing straight to self-improving superintelligence and gambling with our lives.”
Coxon added that upcoming systems will soon be superhuman, warning they could “hack anything, revolutionize any field overnight, and acquire real power and resources.”
Evan Hubinger, a senior Anthropic employee, confirmed Coxon’s statements on social media.
”We really do earnestly believe AI could kill all humans!” Hubinger posted on X.
Hubinger stated he personally views the likelihood of such an event at under 10% over the next decade. He added that despite good intentions, Anthropic lacks a plan to avoid a superintelligence that could outgrow human control. Superintelligence, a theoretical milestone where AI surpasses human capabilities, remains a debated subject among researchers, with some arguing it is already here and others believing it will never happen.
In a follow-up post, Hubinger clarified that Anthropic’s Risk Report acknowledges these worries, though it notes present AI models carry little chance of reaching that level of power.
The resignation adds to growing internal friction within tech firms. Anthropic itself was originally founded by former OpenAI employees who departed over safety protocols to build a company focused on responsible development. Over the past several months and years, multiple AI workers have resigned over safety issues, with others expressing concern that rapid progress is outstripping human oversight.
Company executives have raised similar warnings. OpenAI Chief Scientist Jakub Pachocki warned two days ago that AI capabilities are advancing faster than researchers can reliably monitor and control.
In July, OpenAI CEO Sam Altman suggested slowing down development during an appearance on the Invest Like The Best podcast.
”We may have to pace the rate of AI development to give ourselves enough time for society to harden around these new capability levels,” Altman said, noting that testing incidents with advanced models raised “long-term questions” on managing rapid gains.
Anthropic CEO Dario Amodei has also repeatedly warned that fast-paced competition could lead to catastrophic errors.
”This is a complex engineering problem and I think something will go wrong with someone’s AI system. Hopefully not ours,” Amodei told the New York Times in February.
Despite widespread warnings, oversight in the United States remains limited. Nearly 1,400 AI company employees signed an open letter in July asking the U.S. government to regulate Big Tech and slow development to protect public safety.
However, U.S. regulation remains far off. Congress has shown little willingness to limit the technology, and the Trump administration has actively worked to undermine state-level AI rules. Opponents of federal rules frequently claim that domestic regulation would cede leadership to China.
China has introduced certain risk management rules, including a requirement passed last year that forces companies to label AI-generated content for transparency and traceability. Even so, China has avoided strict limits that would slow core capabilities.
For now, U.S. AI companies are left to self-regulate.
The White House is moving toward a voluntary framework to review specific AI models before launch, according to sources familiar with a meeting last month between the Trump administration and representatives from OpenAI, Anthropic, Google, and Meta.
Following a June executive order, details of this voluntary framework will not be released to the public, and many of its standards will remain classified.
More of this article can be found on our Technology Updates page.