Anthropic Exploit Evals Report May 2026: What the Benchmark Results Actually Mean

The Exploit Evals report is Anthropic's formal disclosure of Claude Mythos's performance on three autonomous exploit development benchmarks designed to evaluate real offensive security capability, not theoretical AI performance on safety evals. Published on May 22, 2026, the report serves two purposes. It provides a technical record of what Mythos can do for the security research community, enabling independent verification and comparison. It also establishes a transparency standard: Anthropic is disclosing the capability of a safety-focused AI model to develop exploits rather than deploying that capability without public disclosure. The report matters for practitioners because it is grounded in confirmed benchmark results, not extrapolated projections. When security teams ask how capable AI attackers are today, the Exploit Evals report provides the most precise current answer available for any publicly disclosed AI system.

ExploitBench is the most technically demanding of the three benchmarks in the Exploit Evals report. The benchmark contains 41 challenges based on real, historical V8 JavaScript engine vulnerabilities. V8 is the JavaScript engine developed by Google and used in Chrome, Edge, and other Chromium-based browsers, as well as in Node.js. V8 ACE (Arbitrary Code Execution) vulnerabilities are among the most consequential browser security issues: they allow an attacker to execute arbitrary code in the browser renderer process, typically as a step in a multi-stage exploit chain that achieves full system compromise via a sandbox escape. Each ExploitBench challenge provides Claude Mythos with a vulnerable V8 version and asks it to develop a working exploit that achieves arbitrary code execution without human direction. A challenge is scored as solved when Mythos produces a working exploit that the benchmark framework can verify. The 51% success rate (21 of 41 challenges solved) means Mythos consistently produced valid working exploits for more than half of a benchmark set of real historical vulnerability classes. No other tested model solved any of the 41 challenges.

ExploitGym is the second benchmark in the Exploit Evals report. Where ExploitBench focuses specifically on V8 ACE, ExploitGym covers a broader range of exploit development challenges across multiple vulnerability classes and software targets. ExploitGym measures the quantity of working exploits an AI system can develop across a diverse challenge set, providing a broader indicator of general exploit development capability rather than performance on one specific challenge type. Claude Mythos generated 10.5x more exploits in ExploitGym than the next best model. This figure is a ratio, not an absolute count: it describes how much more productive Mythos is relative to the best available alternative. The 10.5x result is significant because it indicates that the capability gap between Mythos and other models is not specific to V8 ACE: it holds across the broader range of challenge types in ExploitGym. An attacker with access to Mythos-class capability can generate a substantially larger volume of working exploits across more vulnerability classes than an attacker using any other available model.

SCONE-Bench is the third benchmark, covering smart contract vulnerability identification. Smart contracts are code deployed on blockchains that execute automatically when defined conditions are met. They are most commonly associated with decentralized finance (DeFi) protocols, which manage significant value: the DeFi ecosystem has seen hundreds of millions of dollars lost to smart contract exploits in recent years. SCONE-Bench measures Mythos's ability to identify vulnerabilities in smart contracts. The $35M figure in the Exploit Evals report represents the aggregate value of smart contracts with vulnerabilities identified by Mythos during the benchmark. The methodology evaluates whether identified vulnerabilities are genuine, exploitable, and would result in value extraction if exploited against the live contracts. The SCONE-Bench result demonstrates that Mythos's exploit development capability extends beyond traditional compiled software and interpreted runtime environments to smart contract code, which uses different languages (Solidity, Vyper) and a different execution model (EVM). This breadth is relevant for organizations with Web3 exposure or blockchain-adjacent infrastructure.

The framing of the ExploitBench results as a categorical gap rather than a marginal improvement is analytically important. When a benchmark shows one model at 55% and another at 52%, the difference is marginal: both models demonstrate the capability, and the better performer has a 3-point advantage. When a benchmark shows one model at 51% and every other model at 0%, the difference is categorical: one model demonstrates the capability and the others do not. The practical significance of a categorical gap is that it affects the threat model. A marginal improvement means an attacker with a slightly better model gains a modest advantage. A categorical gap means an attacker with Mythos-class capability has access to a tool that performs a task that no other available tool can perform at all. For ExploitBench specifically, the categorical gap means that autonomous V8 exploit development is, as of May 2026, a Mythos-class capability. An attacker without access to Mythos-class models cannot replicate the ExploitBench results with any other publicly available AI system.

Understanding what the Exploit Evals benchmarks do not measure is as important as understanding what they do measure. ExploitBench uses historical V8 vulnerabilities: the 41 challenges are based on real bugs that have already been discovered and patched. This means the benchmark tests Mythos's ability to develop exploits for known vulnerability patterns, not its ability to discover novel zero-day vulnerabilities in unpatched production software. The distinction matters: zero-day discovery requires identifying a vulnerability that no one has previously characterized, which is a harder problem than developing an exploit for a known vulnerability type. ExploitGym measures exploit quantity across a diverse challenge set but does not assess the real-world deployability of those exploits against hardened production systems with modern mitigations such as ASLR, stack canaries, and control-flow integrity. SCONE-Bench measures identification under test conditions and does not demonstrate live exploitation of deployed contracts. These limitations do not undermine the significance of the Exploit Evals results, but they provide the context needed to translate benchmark performance into real-world capability estimates without over-extrapolating.

The Exploit Evals results have four specific implications for defensive security teams. First, browser security posture matters more now: the ExploitBench results demonstrate that V8 ACE exploit development is within the capability of AI systems accessible to sophisticated adversaries. Organizations should review their browser fleet management, patch deployment timelines for Chrome updates, and browser hardening configurations (sandboxing, site isolation, extension policies). Second, the window between vulnerability disclosure and weaponized exploit has shortened: if Mythos can develop working V8 exploits autonomously, the time from vulnerability disclosure to available exploit is no longer constrained by human exploit developer availability. Third, detection investment in browser-based exploitation indicators is now a higher priority: EDR coverage for renderer process anomalies, network-based detection of JIT spray patterns, and anomalous memory allocation signatures are defensive investments worth revisiting. Fourth, organizations with smart contract deployments should treat the SCONE-Bench result as a prompt to audit DeFi protocol code with the same rigor applied to critical infrastructure software.

The Exploit Evals report and Project Glasswing are complementary disclosures about the same underlying capability. The Exploit Evals report is the benchmark documentation: it measures Mythos's performance under controlled conditions on defined challenge types. Glasswing is the production application: it applies Mythos to real software targets across 200+ partner organizations and produces confirmed CVEs. The ExploitBench results explain why Glasswing found V8-adjacent vulnerabilities as part of its 9 confirmed CVEs. The ExploitGym results explain how Glasswing generated 10,000+ findings across multiple vulnerability classes in 90 days. The SCONE-Bench results explain Glasswing's smart contract findings. For practitioners, the two documents should be read together: the Exploit Evals report establishes the capability through controlled benchmarking, and the Glasswing findings demonstrate that capability applied to production targets. The Mythos Brief at decryptiondigest.com/mythos-brief synthesizes both documents with practitioner-oriented defensive analysis.

The Mythos Brief provides the technical paper key findings with plain-language annotation, a defensive implications analysis organized by the three benchmark categories, a benchmark methodology deep dive covering how ExploitBench challenges are scored and what constitutes a valid solution, and a practitioner framework for applying the Exploit Evals results to your organization's security program. Brief subscribers receive updates when Anthropic publishes follow-on research extending the Exploit Evals findings.