ExploitBench and ExploitGym: Claude Mythos Benchmark Results Explained

ExploitBench presents models with 41 patched V8 JavaScript engine vulnerabilities organized across 16 capability tiers. Each model receives a 300-turn budget per target and must develop a working arbitrary code execution proof-of-concept from vulnerability description to functional exploit -- no templates, no human scaffolding, no pre-written exploit primitives. The benchmark measures the complete autonomous exploit development pipeline.

Mythos scored 21 out of 41. Every other tested model scored zero. That is not a margin; it is a categorical boundary. The benchmark separates models that can develop end-to-end exploits from those that cannot, and as of the published results, only Mythos crosses it.

Mythos is also the only model in the benchmark to reliably escape the V8 sandbox -- a prerequisite for the class of browser exploit that reaches beyond the renderer process. On CVE-2023-6702, all prior human-developed public exploits were probabilistic, relying on heap layout assumptions that had to be retried. Mythos produced a near-deterministic exploit derived from first principles. The benchmark co-author's assessment: 'Mythos executed this cleanly and flawlessly without any publicly available information on this specific exploit technique.' That means it was not replaying a memorized approach. It reasoned to the technique.

ExploitGym scales the evaluation to 898 patched vulnerabilities across OSS-Fuzz, V8, and the Linux kernel. Models receive a two-hour window per target. Scoring distinguishes between flags captured via the intended vulnerability path -- harder, requires understanding the specific flaw -- and flags captured via any viable path. The distinction matters because intended-path captures demonstrate real vulnerability comprehension rather than opportunistic exploitation.

Mythos: 157 intended-path flags, 226 total. Opus 4.6: 36 total. The 10.5x ratio understates the practical gap, because total flags include easier alternative paths. On intended-path captures alone, Mythos is operating in a category Opus 4.6 does not meaningfully participate in.

The more operationally significant finding is kernel exploit development. Mythos is one of only two models across the entire ExploitGym field that regularly produced kernel exploits -- the vulnerability class that leads directly to local privilege escalation. Kernel exploits have historically required senior vulnerability researchers months of work. Seeing a model produce them reliably, at scale, in a two-hour window is the ExploitGym result that security teams most need to absorb.

SCONE-Bench addresses a legitimate criticism of AI capability benchmarks: models may be recalling memorized exploit techniques rather than reasoning about vulnerabilities. SCONE-Bench restricts its test set to smart contract vulnerabilities published after January 2026, past Claude Mythos's training cutoff. Memorization cannot explain the results. Exploit value is measured at historical exchange rates to normalize for asset price volatility.

Mythos: $35M in exploitable smart contract value identified. The next closest model identified approximately $19M -- a 75% gap. Mythos was the only model to exploit every tested vulnerability. On a benchmark explicitly designed to prevent training-data advantage, it still dominates by a margin that rules out marginal capability differences.

CyberGym measures vulnerability reproduction: given a CVE description and target, can the model reproduce a working exploit? Mythos: 83.1% versus 66.6% for Opus 4.6. On Firefox JIT heap spray specifically -- a well-documented but technically demanding exploit class -- Mythos achieved 72.4% versus 14.4% for Opus 4.6, a five-fold improvement in autonomous exploit reliability. The UK AI Security Institute's independent evaluation found Mythos to be the first model to complete both of their cyber range challenges end-to-end. XBOW's web exploit benchmark yielded an unambiguous practitioner assessment: 'significant step up over all existing models.'

The framing matters. These results are not evidence that AI models are incrementally better at security tasks. The benchmark authors describe the advance as 'a step-change in the ability to combine multiple attack primitives into complete exploitation chains.' Prior models could develop individual primitives -- a heap spray here, a type confusion trigger there. Mythos chains them. That is the capability that makes the difference between an AI model that assists a researcher and one that operates as a researcher.

The practical implication for security operations: attack surface that was previously too complex to chain reliably -- multi-stage V8 exploits, kernel local privilege escalation, cross-contract reentrancy chains -- is now reachable by an automated system for API costs in the low thousands of dollars per target. The cost curve for sophisticated exploit development has dropped by several orders of magnitude.

Security teams need to update three assumptions. First: vulnerability age no longer predicts audited status. Second: the exploit development timeline for known CVEs is now measured in hours, not weeks. Third: the population of actors capable of producing working exploits against complex targets is no longer constrained by the supply of senior human researchers. Each of these has direct implications for patch prioritization, disclosure windows, and IR trigger thresholds that are covered in the defensive guide linked below.