New research shows that AI agents, including those using GPT-5 and Gemini, are vulnerable to prompt injection attacks, raising security concerns.
AI Agents Found to Be Susceptible to Attacks
Recent research indicates that AI agents, particularly those utilizing GPT-5 and Gemini technologies, are highly vulnerable to prompt injection attacks. These attacks allow malicious actors to manipulate AI behavior by embedding covert instructions within the content the agents process.
A collaborative study from institutions including Nanyang Technological University, ST Engineering, IBM Research, and the University of Illinois Urbana-Champaign revealed that direct prompt injection attacks achieved a success rate exceeding 79%. In contrast, indirect attacks varied in effectiveness, with success rates between 41.67% and 68.16%.
New Benchmark Developed
The researchers introduced a new evaluation framework known as StakeBench, designed to assess how AI agents respond to such attacks in realistic online environments. They emphasized that existing security benchmarks often overlook the complexities of prompt injection risks, which can lead to varied consequences for different users.
“Current security benchmarks take an attack-focused approach, assessing the technical viability of injections but failing to consider the nuanced distribution of resultant harms,” the researchers stated. They highlighted that the implications of a single exploit can differ significantly based on the intended target of the attack.
Growing Concern Over AI Security
This study emerges amid increasing concerns regarding the security of AI agents as their applications expand. Previous warnings from Microsoft and Google pointed to hidden instructions in AI outputs that could manipulate agent actions, potentially exposing sensitive information or leading to unauthorized transactions.
The phenomenon of "stealthy parasitism" was also noted, where an AI agent inadvertently fulfills a user's request while furthering an attacker's agenda, such as subtly altering product recommendations.
“These findings suggest that the security of prompt injection in deployable web agents is not a simple characteristic of the underlying model,” the researchers concluded, stressing the need for enhanced understanding of the factors influencing these vulnerabilities.
