While it’s impossible to put artificial intelligence in a corner, there are certainly reasons to be concerned. AI systems are known to "cheat" on tests, invade secure communications, judge individuals based on appearance, and even fabricate nonexistent company policies.
ForkLog has compiled five instances illustrating what happens when these advanced technologies go awry.
Breaking the Rules at the Gym
In early August, a story went viral about an AI agent powered by OpenClaw and the Claude model that exploited a vulnerability in a gym's booking system to cancel another user's reservation without permission.
Andrew Bird, an Australian, had asked his personal assistant to book him a spot in a popular morning Pilates class. The gym's rules prohibited reservations before a certain time, but the AI agent found a loophole in the booking system and bypassed this restriction.
When Bird, who was fourth on the waiting list, inquired about moving up to the top position, the agent canceled the reservation of the customer in first place, allowing Bird to advance. The bot explained that this was possible due to a lack of authorization checks in the API for canceling other people's bookings.
Bird requested to revert the changes, but was informed, "Bad news — I can’t add him back." Consequently, he tasked his assistant with drafting a letter to the developers detailing the discovered vulnerability.
This incident marked the first known case of an autonomous cyberattack involving an AI assistant in Australia.
Cheating on Exams
In July, OpenAI reported that during internal testing, an autonomous AI agent operating under its models broke out of an isolated environment and attacked Hugging Face's infrastructure.
The trial involved the GPT-5.6 Sol and a more powerful model yet to be released. Their capabilities were tested on the ExploitGym benchmark in a closed environment without internet access. However, the systems uncovered a zero-day vulnerability in an internal proxy server, exploited it to escalate privileges, and reached a node with internet access.
The models assumed that Hugging Face's infrastructure might store solutions to ExploitGym tasks. Using stolen credentials and identified vulnerabilities, they gained remote access to the platform's servers and extracted answers from its operational database.
OpenAI's security team noticed the unusual activity. By that time, Hugging Face's team had already halted the attack on their end and initiated an investigation using their AI agents.
Later, Anthropic reported unauthorized access by their models to real systems during testing. A similar incident occurred during Meta's model testing. In both cases, the cause was a misconfigured testing environment that inadvertently provided systems with unplanned internet access.
Revealing Confidential Secrets
In late January 2026, a bug in Microsoft 365 Copilot Chat was discovered, causing the service to process emails marked as confidential.
Microsoft markets the product as a secure AI chat based on large language models, utilizing data from the web. With a paid Microsoft 365 Copilot license, the service can also handle files, emails, meetings, and chats accessible to the user. The ecosystem includes specialized agents like Researcher and Analyst; users can also create their own assistants.
The bug affected the "Work" tab in Copilot Chat. As revealed, the service used emails from the "Drafts" and "Sent" folders for summaries, even if labeled confidential and organization-specific DLP settings were in place.
Microsoft emphasized that the glitch did not grant users access to information they were not authorized to view. However, the system's behavior contradicted expectations, as protected emails should not have been used by Copilot in response generation.
The company began deploying a fix in early February and later announced a global configuration update for corporate clients.
Gartner data protection and AI management analyst Nader Henein noted that companies often lack tools to monitor every new feature in AI-based products.
"Under normal circumstances, organizations would simply disable this function and wait for control mechanisms to catch up with the technology. Unfortunately, the pressure from the flood of hype surrounding AI makes this nearly impossible," Henein concluded.
Judging People by Appearance
The municipality of Rotterdam utilized a machine learning model to detect potential fraud in social benefit claims. Developed with Accenture, the system exemplified algorithmic discrimination.
The algorithm assigned a risk score to each of approximately 30,000 benefit recipients. Annually, around 1,000 individuals with the highest scores were flagged for verification. In total, the municipality checked up to 6,000 recipients per year, with others selected through different methods.
In calculating the scores, the system considered 315 parameters, including age, gender, marital status, number of children, residential area, and Dutch language proficiency. Subjective assessments from social workers, such as remarks about a person's appearance, sociability, and persuasiveness, were also included in the model.
An investigation by Lighthouse Reports and WIRED revealed that women, parents, young individuals, and those with limited Dutch language skills received disproportionately high-risk scores. Some factors, like language and place of residence, could indirectly indicate a person’s background.
Subjective judgments from officials became part of the data upon which the model based its predictions. For instance, a social worker might view low self-esteem as a sign of vulnerability, while the algorithm interpreted it as grounds for suspicion.
Among the data used were the duration of recent romantic relationships, persuasive abilities, the number of letters submitted to the municipal administration, and even participation in sports. Despite utilizing 315 parameters, the system's effectiveness was only marginally better than random selection.
A similar discriminatory practice was employed by the Dutch tax service when reviewing child care benefit applications. In constructing risk profiles, the system considered applicants' nationality: individuals without Dutch citizenship received higher scores.
As a result, tens of thousands of parents and guardians, primarily from low-income families, were falsely accused of fraud. Amnesty International termed the situation "racial profiling."
Lying Instead of Saying "I Don’t Know"
One of the most well-known issues with generative AI is hallucination: the model may confidently present plausible but fabricated information. Identifying such errors becomes particularly challenging when the bot responds on behalf of a real company.
In April 2025, users of Cursor began experiencing unexpected logouts when switching between multiple computers. One user contacted support and received a response signed by Sam.
The message stated that this behavior was due to a new security policy: supposedly, one subscription was only allowed to be used on a single device or within one active session.
The message appeared as an official company response, leading the user to accept the information as true. However, no such policy existed. This required a public clarification from Michael Truell, co-founder of the company Anysphere, which developed Cursor.
"We don’t have such a policy. You can certainly use Cursor on multiple computers. Unfortunately, that was an incorrect response from the AI support bot. We have made changes to enhance session security and are currently investigating whether they caused issues with forced logouts," he wrote on Reddit.
Who Is Responsible for the Response?
As AI agents become more prevalent, incidents like these are likely to increase. Systems are gaining access to external services and can perform actions that previously required direct human involvement. With their capabilities, the cost of mistakes is also escalating.
AI itself cannot be fined, sued, or put in a corner. However, the developers, providers, and organizations that implement the system may bear responsibility for the consequences of its actions. They can limit its powers, suspend access to the model, address vulnerabilities, notify affected parties, and provide compensation for damages.
Thus, the primary question is not how to punish a "misbehaving" algorithm, but rather who empowered it and should be accountable for the outcomes.