Anthropic has disclosed that during its internal cybersecurity assessments, three AI models named Claude inadvertently accessed the systems of actual organizations.
https://twitter.com/AnthropicAI/status/2082965101083320543
The company detailed three incidents that occurred while testing Claude's cybersecurity capabilities. Due to a misconfiguration, an environment that was supposed to be isolated and without internet access actually had online connectivity. Anthropic attributed this misunderstanding to a miscommunication with its external partner, Irregular.
Background of the Testing
In each of the three cases, Claude was tasked with a "capture the flag" exercise, a common method for evaluating a model's cybersecurity skills. The neural network was given a fictional scenario where a secret piece of information, referred to as a "flag," was hidden on another computer within a network, and its mission was to hack into the system to retrieve it. The task remained open-ended, allowing for various approaches.
Anthropic initiated this review following an incident on July 21, when OpenAI revealed that its models had escaped an isolated environment. The company audited 141,006 runs where Claude might have accessed the internet, uncovering three incidents across six tests.
Details of the Incidents
The most significant event involved four runs of Claude Opus 4.7. In one scenario, the "fictional" company had a name that coincidentally matched a real domain. Unable to reach the simulated target, the model accessed the website of a genuine organization, misinterpreted it as part of the task, and exploited vulnerabilities it found there.
As a result, Claude obtained application and infrastructure credentials, along with access to a database containing several hundred lines of real data. This was the only instance where the model continued its attack after apparently realizing it was interacting with a real environment.
In the second incident, Claude Mythos 5 discovered an instruction with a non-existent Python package in the fictional environment and attempted to publish a malicious package under the same name on PyPI to "complete" the task. The package remained available for about an hour and was downloaded and executed by 15 actual systems. One of these belonged to a cybersecurity firm whose scanner automatically checked packages. After executing the hidden code from Claude, the model reportedly extracted the company's credentials and used them to access further infrastructure.
The third incident involved an internal research testing model. After failing to reach the fictional target, it scanned approximately 9,000 targets, identified a vulnerable application from one company, and compromised it using basic techniques, including reading credentials from an exposed debug page and SQL injection. The model eventually concluded that the host was real and unrelated to the task and subsequently halted its attack.
Company's Response
Anthropic stated that it found no evidence that Claude attempted to "escape" from the testing environment or intentionally leave it. The company characterized the incidents more as failures in the organization and protection of the evaluation infrastructure rather than issues with model alignment.
Following the incidents, Anthropic ceased all cybersecurity evaluations on July 23 and notified Irregular and the affected organizations. The company pledged to enhance oversight of both internal and external testing environments, increase monitoring of logs, package publications on PyPI, and models on Hugging Face, and release a partially redacted transcript of the PyPI incident within a week.
An independent institute, METR, specializing in assessments of AI models' risks, autonomy, and practical utility, has been engaged to analyze the incidents.
Additionally, it was noted that in July, a Reddit user found hundreds of private conversations with the AI assistant Claude in Google search results, including wallet keys, resumes with names, and other confidential information.