Code Index
Code Index is a live and evolving coding-benchmarks leaderboard composed of novel, contamination-free collections. It currently combines 3 private task sets from traditional benchmarks: SWE-Lancer, Terminal Bench 2.1, Terminal Bench 3.0. More task sets soon to come, including proprietary benchmarks. Not in the index for now: SWE-Bench-Pro Hard.
Code Index leaderboard
claude-opus-5
gpt-6-astra
claude-fable-5
gpt-5.6-sol
deepseek-v4.1-flash
gemini-3.8-flash
kimi-k3
claude-opus-4.8
gpt-5.6-terra
gpt-5.6-luna
qwen3.8-max
grok-4.5
glm-5.2
deepseek-v4-flash-0731
gemini-3.6-flash
deepseek-v4-pro
hy3
minimax-m3
mimo-v2.5-pro
nemotron-3-ultra-550b-a55b
The Index is the mean pass rate of the 3 benchmarks in the index, each with equal weight. Read the methodology.
Per benchmark
Our results show that models can have different standings for different benchmarks. See the detailed results per benchmark.
claude-opus-5
gpt-6-astra
claude-fable-5
gpt-5.6-sol
deepseek-v4.1-flash
gemini-3.8-flash
kimi-k3
claude-opus-4.8
gpt-5.6-terra
gpt-5.6-luna
qwen3.8-max
grok-4.5
glm-5.2
deepseek-v4-flash-0731
gemini-3.6-flash
deepseek-v4-pro
hy3
minimax-m3
mimo-v2.5-pro
nemotron-3-ultra-550b-a55b
claude-opus-5
gpt-6-astra
claude-fable-5
gpt-5.6-sol
deepseek-v4.1-flash
gemini-3.8-flash
kimi-k3
claude-opus-4.8
gpt-5.6-terra
gpt-5.6-luna
qwen3.8-max
grok-4.5
glm-5.2
deepseek-v4-flash-0731
gemini-3.6-flash
deepseek-v4-pro
hy3
minimax-m3
mimo-v2.5-pro
nemotron-3-ultra-550b-a55b
claude-opus-5
gpt-6-astra
claude-fable-5
gpt-5.6-sol
deepseek-v4.1-flash
gemini-3.8-flash
kimi-k3
claude-opus-4.8
gpt-5.6-terra
gpt-5.6-luna
qwen3.8-max
grok-4.5
glm-5.2
deepseek-v4-flash-0731
gemini-3.6-flash
deepseek-v4-pro
hy3
minimax-m3
mimo-v2.5-pro
nemotron-3-ultra-550b-a55b
Three trials ran for each task and the mean of all 'n' tasks composes the per-benchmark index. Read the methodology.
Efficiency tradeoffs
The frontier is not only the highest-scoring models, but those that can achieve best results more efficiently. The following charts present the mean agent time and token cost for each model, and which of them compose the frontier line. Better results appear toward the upper left.
Score vs. steps
Tha number of steps needed to reach a given Index score shows which models make the most of their earlier agent steps, which may be crucial under time/cost-limited scenarios.
Accessible summary
| Model | Score at ≤ ceil(P95) steps | ceil(P95) steps |
|---|---|---|
| claude-opus-5 | 45.6% | 116 |
| gpt-6-astra | 43.3% | 47 |
| claude-fable-5 | 40.6% | 58 |
| gpt-5.6-sol | 37.2% | 61 |
| deepseek-v4.1-flash | 37.2% | 149 |
| gemini-3.8-flash | 34.4% | 161 |
| kimi-k3 | 33.9% | 65 |
| claude-opus-4.8 | 33.9% | 68 |
| gpt-5.6-terra | 28.9% | 40 |
| gpt-5.6-luna | 28.9% | 50 |
| qwen3.8-max | 27.2% | 128 |
| grok-4.5 | 26.1% | 42 |
| glm-5.2 | 25.6% | 46 |
| deepseek-v4-flash-0731 | 25.0% | 43 |
| gemini-3.6-flash | 22.8% | 79 |
| deepseek-v4-pro | 16.7% | 70 |
| hy3 | 13.9% | 57 |
| minimax-m3 | 13.9% | 112 |
| mimo-v2.5-pro | 5.6% | 47 |
| nemotron-3-ultra-550b-a55b | 2.8% | 27 |
Difficulty distribution
Difficulty was categorized according to the results of each task across all measured models. Full methodology →
Talk to us
Revelo works with model labs on:
- Training datasets for code generation
- Evaluations of new and unreleased models
- Leaderboard analysis, including failure modes and agent trajectories