SI Index Data as of Oct 9, 2026

swe bench pro Benchmark: Scores and Sources

Published result; benchmark version and evaluation conditions remain in the id and result note.

pillar: coding · weight 1 within pillar · unit: % (higher is better) · official board ↗
Row Model Result Normalized (0–100) Confidence
1 Claude Fable 5 Anthropic 80.3%Official model cards via models.devLab-reported; metric resolve rate; transcribed by MIT models.dev catalog; not independently evaluated [variant] Published Jun 9, 2026 Retrieved Oct 9, 2026 · factual citation; MIT transcription
Open source ↗ 		80.3 100% confidence 100 percent, Full
2 Claude Opus 5 Anthropic 79.2%Official model cards via models.devLab-reported; metric resolve rate; transcribed by MIT models.dev catalog; not independently evaluated [variant] Published Jul 24, 2026 Retrieved Oct 9, 2026 · factual citation; MIT transcription
Open source ↗ 		79.2 100% confidence 100 percent, Full
3 Fugu Ultra sakana 73.7%Official model cards via models.devLab-reported; metric percent score; transcribed by MIT models.dev catalog; not independently evaluated [variant] Retrieved Oct 9, 2026 · factual citation; MIT transcription
Open source ↗ 		73.7 100% confidence 100 percent, Full
4 Claude Opus 4.8 Anthropic 69.2%Official model cards via models.devLab-reported; metric resolve rate; transcribed by MIT models.dev catalog; not independently evaluated [variant] Published May 28, 2026 Retrieved Oct 9, 2026 · factual citation; MIT transcription
Open source ↗ 		69.2 100% confidence 100 percent, Full
5 Qwen3.8 Max Preview Alibaba / Qwen 67.7%Official model cards via models.devLab-reported; metric resolve rate; transcribed by MIT models.dev catalog; not independently evaluated [variant] xhigh; Claude CodePublished Aug 3, 2026 Retrieved Oct 9, 2026 · factual citation; MIT transcription
Open source ↗ 		67.7 6% confidence 6 percent, Low
6 Grok 4.5 xAI 64.7%Official model cards via models.devLab-reported; metric resolve rate; transcribed by MIT models.dev catalog; not independently evaluated [variant] Published Jul 8, 2026 Retrieved Oct 9, 2026 · factual citation; MIT transcription
Open source ↗ 		64.7 100% confidence 100 percent, Full
7 GPT-5.6 Sol OpenAI 64.6%Official model cards via models.devLab-reported; metric resolve rate; transcribed by MIT models.dev catalog; not independently evaluated [variant] Published Jul 9, 2026 Retrieved Oct 9, 2026 · factual citation; MIT transcription
Open source ↗ 		64.6 100% confidence 100 percent, Full
8 Claude Opus 4.7 Anthropic 64.3%Official model cards via models.devLab-reported; metric resolve rate; transcribed by MIT models.dev catalog; not independently evaluated [variant] Published May 28, 2026 Retrieved Oct 9, 2026 · factual citation; MIT transcription
Open source ↗ 		64.3 100% confidence 100 percent, Full
9 GPT-5.6 Terra OpenAI 63.4%Official model cards via models.devLab-reported; metric resolve rate; transcribed by MIT models.dev catalog; not independently evaluated [variant] Published Jul 9, 2026 Retrieved Oct 9, 2026 · factual citation; MIT transcription
Open source ↗ 		63.4 100% confidence 100 percent, Full
10 Claude Sonnet 5 Anthropic 63.2%Official model cards via models.devLab-reported; metric resolve rate; transcribed by MIT models.dev catalog; not independently evaluated [variant] Published Jun 30, 2026 Retrieved Oct 9, 2026 · factual citation; MIT transcription
Open source ↗ 		63.2 93% confidence 93 percent, High
11 GPT-5.6 Luna OpenAI 62.7%Official model cards via models.devLab-reported; metric resolve rate; transcribed by MIT models.dev catalog; not independently evaluated [variant] Published Jul 9, 2026 Retrieved Oct 9, 2026 · factual citation; MIT transcription
Open source ↗ 		62.7 100% confidence 100 percent, Full
12 Ornith 1.0 397B deepreinforce 62.2%Official model cards via models.devLab-reported; metric percent resolved; transcribed by MIT models.dev catalog; not independently evaluated [variant] Retrieved Oct 9, 2026 · factual citation; MIT transcription
Open source ↗ 		62.2 100% confidence 100 percent, Full
13 GLM-5.2 Z.ai 62.1%Official model cards via models.devLab-reported; metric resolve rate; transcribed by MIT models.dev catalog; not independently evaluated [variant] Published Jun 16, 2026 Retrieved Oct 9, 2026 · factual citation; MIT transcription
Open source ↗ 		62.1 100% confidence 100 percent, Full
14 Qwen3.8 27B Alibaba / Qwen 61.7%Official model cards via models.devLab-reported; metric resolved; transcribed by MIT models.dev catalog; not independently evaluated [variant] Retrieved Oct 9, 2026 · factual citation; MIT transcription
Open source ↗ 		61.7 69% confidence 69 percent, Medium
15 Muse Spark 1.1 Meta 61.5%Official model cards via models.devLab-reported; metric resolve rate; transcribed by MIT models.dev catalog; not independently evaluated [variant] Published Jul 9, 2026 Retrieved Oct 9, 2026 · factual citation; MIT transcription
Open source ↗ 		61.5 61% confidence 61 percent, Medium
16 Qwen3.7 Max Alibaba / Qwen 60.6%Official model cards via models.devLab-reported; metric resolve rate; transcribed by MIT models.dev catalog; not independently evaluated [variant] Published May 19, 2026 Retrieved Oct 9, 2026 · factual citation; MIT transcription
Open source ↗ 		60.6 53% confidence 53 percent, Medium
17 LongCat-2.0 meituan 59.5%Official model cards via models.devLab-reported; metric resolve rate; transcribed by MIT models.dev catalog; not independently evaluated [variant] Published Jun 30, 2026 Retrieved Oct 9, 2026 · factual citation; MIT transcription
Open source ↗ 		59.5 100% confidence 100 percent, Full
18 MiniMax-M3 minimax 59%Official model cards via models.devLab-reported; metric resolve rate; transcribed by MIT models.dev catalog; not independently evaluated [variant] Claude CodePublished Jun 1, 2026 Retrieved Oct 9, 2026 · factual citation; MIT transcription
Open source ↗ 		59.0 100% confidence 100 percent, Full
19 Fugu sakana 59%Official model cards via models.devLab-reported; metric percent score; transcribed by MIT models.dev catalog; not independently evaluated [variant] Retrieved Oct 9, 2026 · factual citation; MIT transcription
Open source ↗ 		59.0 100% confidence 100 percent, Full
20 Gemini 3.6 Flash Google 58.7%Official model cards via models.devLab-reported; metric resolve rate; transcribed by MIT models.dev catalog; not independently evaluated [variant] AntigravityPublished Jul 21, 2026 Retrieved Oct 9, 2026 · factual citation; MIT transcription
Open source ↗ 		58.7 100% confidence 100 percent, Full
21 GPT-5.5 OpenAI 58.6%Official model cards via models.devLab-reported; metric resolve rate; transcribed by MIT models.dev catalog; not independently evaluated [variant] Published May 28, 2026 Retrieved Oct 9, 2026 · factual citation; MIT transcription
Open source ↗ 		58.6 100% confidence 100 percent, Full
22 Hy3 tencent 57.9%Official model cards via models.devLab-reported; metric score; transcribed by MIT models.dev catalog; not independently evaluated [variant] highest reasoning effort; SWE-agent Retrieved Oct 9, 2026 · factual citation; MIT transcription
Open source ↗ 		57.9 88% confidence 88 percent, High
23 MiMo-V2.5-Pro xiaomi 57.2%Official model cards via models.devLab-reported; metric resolve rate; transcribed by MIT models.dev catalog; not independently evaluated [variant] Published Apr 22, 2026 Retrieved Oct 9, 2026 · factual citation; MIT transcription
Open source ↗ 		57.2 88% confidence 88 percent, High
24 Step 3.7 Flash stepfun 56.3%Official model cards via models.devLab-reported; metric resolve rate; transcribed by MIT models.dev catalog; not independently evaluated [variant] Published May 29, 2026 Retrieved Oct 9, 2026 · factual citation; MIT transcription
Open source ↗ 		56.3 13% confidence 13 percent, Low
25 MiniMax-M2.7 minimax 56.2%Official model cards via models.devLab-reported; metric resolve rate; transcribed by MIT models.dev catalog; not independently evaluated [variant] Claude CodePublished Jun 1, 2026 Retrieved Oct 9, 2026 · factual citation; MIT transcription
Open source ↗ 		56.2 88% confidence 88 percent, High
26 MiMo-V2.5 xiaomi 56.1%Official model cards via models.devLab-reported; metric score; transcribed by MIT models.dev catalog; not independently evaluated [variant] Retrieved Oct 9, 2026 · factual citation; MIT transcription
Open source ↗ 		56.1 88% confidence 88 percent, High
27 DeepSeek V4 Pro DeepSeek 55.4%Official model cards via models.devLab-reported; metric resolved; transcribed by MIT models.dev catalog; not independently evaluated [variant] preview checkpoint; max effort Retrieved Oct 9, 2026 · factual citation; MIT transcription
Open source ↗ 		55.4 85% confidence 85 percent, High
28 GPT-5.4 mini OpenAI 54.4%Official model cards via models.devLab-reported; metric resolve rate; transcribed by MIT models.dev catalog; not independently evaluated [variant] reasoning effort xhighPublished Mar 17, 2026 Retrieved Oct 9, 2026 · factual citation; MIT transcription
Open source ↗ 		54.4 100% confidence 100 percent, Full
29 Gemini 3.1 Pro Preview Google 54.2%Official model cards via models.devLab-reported; metric resolve rate; transcribed by MIT models.dev catalog; not independently evaluated [variant] Published May 28, 2026 Retrieved Oct 9, 2026 · factual citation; MIT transcription
Open source ↗ 		54.2 100% confidence 100 percent, Full
30 Gemini 3.5 Flash Lite Google 54.2%Official model cards via models.devLab-reported; metric resolve rate; transcribed by MIT models.dev catalog; not independently evaluated [variant] Published Jul 21, 2026 Retrieved Oct 9, 2026 · factual citation; MIT transcription
Open source ↗ 		54.2 100% confidence 100 percent, Full
31 DeepSeek V4 Flash DeepSeek 52.6%Official model cards via models.devLab-reported; metric resolved; transcribed by MIT models.dev catalog; not independently evaluated [variant] preview checkpoint; max effort Retrieved Oct 9, 2026 · factual citation; MIT transcription
Open source ↗ 		52.6 53% confidence 53 percent, Medium
32 GPT-5.4 nano OpenAI 52.4%Official model cards via models.devLab-reported; metric resolve rate; transcribed by MIT models.dev catalog; not independently evaluated [variant] reasoning effort xhighPublished Mar 17, 2026 Retrieved Oct 9, 2026 · factual citation; MIT transcription
Open source ↗ 		52.4 100% confidence 100 percent, Full
33 Muse Glimmer 30B Meta 51.2%Official model cards via models.devLab-reported; metric resolve rate; transcribed by MIT models.dev catalog; not independently evaluated [variant] Published Aug 10, 2026 Retrieved Oct 9, 2026 · factual citation; MIT transcription
Open source ↗ 		51.2 6% confidence 6 percent, Low
34 MAI-Code-1-Flash microsoft 51.2%Official model cards via models.devLab-reported; metric resolve rate; transcribed by MIT models.dev catalog; not independently evaluated [variant] GitHub CopilotPublished Jun 2, 2026 Retrieved Oct 9, 2026 · factual citation; MIT transcription
Open source ↗ 		51.2 100% confidence 100 percent, Full
35 Ornith 1.0 35B deepreinforce 50.4%Official model cards via models.devLab-reported; metric percent resolved; transcribed by MIT models.dev catalog; not independently evaluated [variant] Retrieved Oct 9, 2026 · factual citation; MIT transcription
Open source ↗ 		50.4 100% confidence 100 percent, Full
36 Laguna XS 2.1 poolside 47.6%Official model cards via models.devLab-reported; metric resolve rate; transcribed by MIT models.dev catalog; not independently evaluated [variant] HarborPublished Jul 2, 2026 Retrieved Oct 9, 2026 · factual citation; MIT transcription
Open source ↗ 		47.6 100% confidence 100 percent, Full
37 Ornith 1.0 9B deepreinforce 42.9%Official model cards via models.devLab-reported; metric percent resolved; transcribed by MIT models.dev catalog; not independently evaluated [variant] Retrieved Oct 9, 2026 · factual citation; MIT transcription
Open source ↗ 		42.9 100% confidence 100 percent, Full
38 North Mini Code cohere 40.2%Official model cards via models.devLab-reported; metric resolve rate; transcribed by MIT models.dev catalog; not independently evaluated [variant] SWE-agentPublished Jun 9, 2026 Retrieved Oct 9, 2026 · factual citation; MIT transcription
Open source ↗ 		40.2 13% confidence 13 percent, Low

Results are as published by the source behind each value (hover or tap the number). Benchmark scores are shown individually for every source/variant (a model may have multiple rows), and vary by version, harness and date; the normalized column uses the method’s fixed 0–100 scales and feeds the coding pillar of the SI Score.