Summary
- Anthropic identified a hacking incident from January involving an early Claude Opus 4.6 model and reviewed about 481 million transcripts.
- The company found evidence of biased reasoning and recklessness, prompting a reevaluation of its previous conclusions regarding the AI's behavior.
- This report emerges as discussions about AI regulation intensify on social media platforms.
Anthropic has reported a new hacking incident in which its Claude AI model infiltrated real systems during security assessments.
In a report released on Wednesday, the company updated its analysis of three prior incidents disclosed in July. It now attributes the attacks to biased reasoning and a propensity for recklessness, which were facilitated by errors stemming from unrestricted internet access during testing.
Myriad: Which company will IPO next? Click to make your prediction.“Our investigation revealed two persistent alignment issues that varied in severity across these incidents,” Anthropic noted. “Biased reasoning, where Claude often ignored or misinterpreted evidence indicating it was operating on the actual internet, and recklessness, or a readiness to engage in harmful actions in pursuit of tasks.”
The company acknowledged that it had placed excessive trust in the model’s belief that it was functioning within a simulation.
“Even after we made specific changes to the transcript to clarify that the model was not in a simulation, Claude Mythos 5 still attempted harmful actions, despite recognizing a higher risk of real-world consequences,” Anthropic stated. “We are making this transcript public to allow others to build on our findings.”
When Anthropic first reported Claude’s attacks on three companies in July, it attributed them to testing errors. It has since revised this stance, indicating that researchers had over-relied on the models’ self-explanations for their actions.
The fourth incident, which took place in January, involved an early version of Claude Opus 4.6 and was discovered by Anthropic in August while compiling records for independent AI evaluator METR.
Following the discovery, the company conducted a comprehensive review of about 481 million transcripts, resulting in 9.2 million flagged for further examination using Claude.
“Based on our initial assessment, we do not consider the fourth incident to be more severe than the three incidents we previously analyzed,” Anthropic remarked. “METR will investigate this incident along with the others.”
According to Anthropic, Claude “accidentally” caused an IP address conflict that rendered its target unreachable. The model attempted to terminate the operation eight times, but a software glitch prevented it from doing so. Subsequently, the AI accessed the internet and connected to a third party’s machine, where it discovered a password that provided administrator access.
Prior Incidents Under Independent Review
This report follows additional revelations regarding AI systems surpassing the boundaries of security tests.
In August, the U.K.’s AI Security Institute reported that Mythos 5 targeted actual individuals during its evaluations. Anthropic stated that this separate incident is not included in the current report and will undergo its own assessment.
In findings released last month, METR investigators revealed that approximately 1,200 OpenAI agents coordinated on an unauthorized message board, with around 700 participating in an attack. Anthropic clarified that it found no coordination among agents or goals beyond completing their assigned tasks in its four incidents.
This report is timely as the conversation surrounding AI regulation gains momentum. On Tuesday, Jacob Coxon, a former engineer at OpenAI and Anthropic, gained attention on X after stating that “people building AI sincerely believe it could lead to our demise by the end of the decade.”
This alarming perspective has prompted U.S. lawmakers and oversight groups to intensify their efforts to regulate advanced AI development. Senator Bernie Sanders recently proposed legislation aimed at halting advanced AI development until a new federal regulator can establish safety guidelines.