Jacob Coxon, a former researcher at OpenAI and Anthropic, has left Anthropic, accusing the leaders in the AI sector of irresponsibility and posing a potential risk to humanity.
I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives. More thoughts below.
— Jacob Coxon (@hilbertspaess) September 9, 2026
Coxon stated that both organizations are engaged in a dangerous competition, primarily aimed at developing self-improving superintelligence. He warned that superhuman systems capable of hacking networks, drastically altering industries, and accumulating substantial resources could emerge soon.
The developer expressed concern that algorithm creators genuinely expect humanity could face extinction by the end of the decade. While this fear is often discussed within the industry, executives tend to downplay it publicly. He emphasized that no other industry carries such significant risks.
He also shared insights on the attitude towards safety at the two leading companies:
- At OpenAI, staff do not grasp the civilization-scale threat.
- At Anthropic, the risks are acknowledged, yet management continues to push forward to outpace competitors and gain control over AI.
Coxon characterized the current race as a "reckless gamble" and urged American laboratories to agree on slowing down development. He believes that the industry needs a temporary halt on scaling new models. He advised colleagues against initiating training for superintelligence until there is a complete understanding of its neural processes.
Users have reacted skeptically to predictions of extinction. One commenter dismissed the fears as absurd, arguing that humanity has survived hundreds of thousands of years of evolution and will not perish simply because a "token prediction model" becomes sentient.
With all respect, this is bizarre. It’s a ridiculous take. Humans have evolved over hundreds of thousands of years. We aren’t going to die out because a token prediction model gained sentience.
Get a grip. All of you.
— alias (@loadingalias) September 9, 2026
Another user questioned how AI would actually exterminate humans—whether through combat drones or by causing fires via home routers.
I am ignorant on the topic but I have no understanding of exactly *how* the superintelligent AI will be killing us. Will it exterminate via drones? Will it make our household routers somehow into bombs or start fires? Help
— halscioN 🌹 (@HalscionRose) September 9, 2026
A third participant raised the issue of legal liability, asking who would be responsible if an algorithm were to harm or kill someone—the developer, the implementing organization, or the end-user of the program?
if you really believe this you stay on the job and sabotage the progress, what does resigning do? you think it's going to kill everybody and you decided not to fight? so leave just to virtue signal and show humans are too pussy to fight?
just not understanding the game theory
— bone (@boneGPT) September 9, 2026
“AGI is Already Here”
Nvidia CEO Jensen Huang announced that OpenAI's new AI model, Astra, has achieved a level of artificial general intelligence (AGI). However, investors and experts have responded with skepticism.
GPT-6 Astra, trained on ~100K+ NVIDIA Grace Blackwell NVLink72. From ChatGPT to o1 to Astra in 4 years.
AGI has arrived. Congratulations @OpenAI team.
400K GPUs coming online next.
— Jensen Huang (@JensenHuang) September 6, 2026
On September 8, during the first trading session after the long Labor Day weekend in the U.S., shares of CoreWeave surged by 15%, SoftBank increased by 2%, while Nvidia’s stock fell by 2%.
Gil Luria, head of technology research at D.A. Davidson, explained that Nvidia's scale plays a part. The company recently reported a record quarterly revenue of $96 billion. The market perceives that Nvidia may have reached its peak, prompting investors to lock in profits and shift funds to other ventures, such as CoreWeave.
Experts doubt that Astra qualifies as true AGI. Previously, Huang stated that genuine AGI should be capable of independently creating a company valued at $1 billion. Until AI demonstrates such an achievement, discussions about AGI remain theoretical, Luria noted.
Basil Halperin, an economist at the University of Virginia, also disagreed with Huang's claims, asserting that current neural networks lack the cognitive abilities of an adult human and require ongoing user supervision.
Halperin believes that the stock market is a poor indicator of technological progress. A more accurate marker for the emergence of AGI would be real interest rates. If AI significantly accelerates global economic growth, rates could rise to 10%. Currently, the economy does not reflect such metrics.
According to Halperin, AI development in the next five years will lead to a boom reminiscent of the dot-com era, contributing about 0.5% to global GDP growth, but it is still early to discuss technological singularity.
In September, Jeffrey Hinton, a Nobel Prize-winning physicist and one of the industry pioneers, warned of the existential risks posed by AI.