Overview
- Google introduced Gemini 4 Argon on Wednesday, achieving a score of 77.9% on the DeepSWE v1.1 benchmark and outperforming 12 out of 18 metrics in its comparison chart.
- The model recorded a 0.7% success rate in Gray Swan's prompt injection test, outperforming Claude Opus 5.5 and Claude Fable 5.1, both of which scored 1.0%.
- Initially, Argon will be available to selected cybersecurity professionals through the Fairwind Program, lacking standard cyber guardrails, before expanding to paying API clients and Google AI Ultra members.
Gemini 4 has officially launched, just a week after the debut of Claude Opus 5.5 and a day following the release of GPT 6.1 Sol, underscoring the commitment of American tech companies to advancing AI technology. We say this with a hint of irony.
On Wednesday, Google unveiled Gemini 4 Argon, branding it as their most advanced model, designed for coding, office tasks, and cybersecurity applications.
In the DeepSWE v1.1 evaluation, which assesses an AI's ability to handle complex software engineering tasks, Argon scored 77.9%. For context, Claude Opus 5.5 achieved 74.2%, GPT-6 Astra scored 74.1%, and Claude Fable 5.1 came in at 67.4%.
To provide perspective, Gemini 3.6 Flash scored only 49% on the same assessment back in July. Argon now has the capability to generate up to 1 million tokens in a single response, a significant increase from the previous limit of 64,000 tokens. A token refers to a segment of text, roughly equivalent to three-quarters of a word, amounting to around 750,000 words compared to about 48,000.
However, it's important to approach these figures with caution. Google derived its own DeepSWE score, while its competitors' results were based on public leaderboards and corporate disclosures. The comparison also shows that while Argon excels in 12 of the 18 benchmarks, it is tied in one and falls short in five, which encompass areas like coding, scientific research, and computer control.
The standout feature of this model is its cybersecurity capabilities. It can execute indirect prompt injection attacks, where a hidden instruction is embedded in an email, allowing the AI to potentially follow the instruction of an unauthorized user instead of its intended operator. This poses a significant risk for those considering using AI for tasks like managing emails or shopping carts.
In the Gray Swan's Indirect Prompt Injection assessment, which measures how often these hidden malicious instructions succeed within 15 attempts, Argon achieved a score of 0.7%, indicating a lower success rate is preferable. In comparison, both Claude Opus 5.5 and Claude Fable 5.1 scored 1.0% while GPT-6 Astra recorded 8.5%. Grok 4.6 and Kimi K3 were misled more than half the time, with scores of 51.8% and 52.7%, respectively.
Argon will first be released to vetted cybersecurity teams through the Fairwind Program, which is a selective cyber defense initiative launched on September 2, partnering with over 650 organizations, including governmental bodies and critical infrastructure operators. Notably, it will be distributed "without cyber guardrails," meaning it lacks the usual safeguards that prevent an AI from assisting in hacking activities.
The rationale for this approach is that cybersecurity defenders need to think like attackers to identify vulnerabilities before they can be exploited. However, Google emphasizes that a phased rollout is essential for safety. The company is also participating in a voluntary pre-release model access initiative with the U.S. government.
Google is not alone in adopting this cautious approach to cybersecurity models. An earlier iteration of Anthropic's Claude Mythos identified 271 vulnerabilities in Firefox, prompting Mozilla to implement necessary patches. Similarly, OpenAI has established a Trusted Access for Cyber program to manage access to its models for cybersecurity purposes.
Argon's cybersecurity performance surpasses that of Gemini 3.8 Flash Cyber, the restricted model initially provided through Fairwind. In the Wiz Penetration Test Benchmark, an internal Google assessment that challenges an AI to create effective exploits against real-world web application vulnerabilities without access to the code, Argon scored 70.9%, significantly higher than the 58.2% achieved by its predecessor.
Additionally, Google reports that Argon assisted the security firm Wiz in uncovering a critical vulnerability in healthcare software used globally by hospitals, which had previously gone unnoticed by earlier models.
This launch follows a challenging summer for Google, during which it released smaller Flash models but omitted the anticipated Gemini 3.5 Pro, leading to a 4.4% decline in Alphabet's stock price. Notably, Argon was released on the same day that President Trump announced a voluntary, non-penalizing AI agreement that Google's executives signed.
Google anticipates a broader release of Argon as soon as feasible, beginning with paid API clients and subscribers of Google AI Ultra.
Introductory pricing is set at $2 for every million input tokens and $10 for every million output tokens. Google has not specified when this introductory phase will conclude, only stating that standard rates will be $4 and $20, respectively.