Anthropic expresses concerns over AI's potential for self-improvement as over 80% of its code is now generated by AI systems.
Anthropic is increasingly relying on AI systems for the development of its models, with over 80% of the code for its current products generated by its AI, Claude. This trend raises alarms about the potential for recursive self-improvement in AI technology.
Marina Favaro, head of the Anthropic Institute, along with co-founder Jack Clark, warned that if computational power continues to grow, AI systems could autonomously create their successors. They stated, “We have not yet reached a point of no return, and recursive self-improvement is not inevitable. However, it may occur sooner than most institutions are prepared for.”
Impressive Metrics
Recent data revealed that Claude completed over 800 code fixes in April, a task that would have taken a human engineer four years. The AI’s success rate for open tasks surged to 76% by May 2026, a significant increase from six months prior.
Furthermore, the time needed for AI to perform tasks independently has halved, now doubling every four months. Claude Opus 4 achieved a threefold speed increase in training a small AI model, while Mythos Preview delivered a staggering 52 times speed increase in April 2026.
In tests, the Mythos Preview model effectively tackled AI safety research tasks, achieving a 97% resolution rate in 800 hours, far surpassing the 23% completion rate by two human researchers in a week.
Changing Developer Roles
Despite the advancements in AI coding capabilities, Anthropic believes that human developers will shift from writing code to reviewing AI outputs. This change may create a bottleneck in the speed of new model development due to the need for human oversight.
The company also suggested that it may be prudent to establish mechanisms for slowing or pausing advanced AI development to allow societal institutions to catch up. However, they warned that unilateral action could disadvantage cautious players, as less risk-averse competitors could gain an edge in the rapidly evolving landscape.
In May, Anthropic published its first report on Project Glasswing, aimed at identifying vulnerabilities in AI systems, and released updates to its Claude models, including new features to enhance workflow efficiency.
