Benchmark explorer.
AutomationBench-AAAA Index 4.3.2 collection · Higher is better
Source ↗Method
Artificial Analysis evaluation. Index 4.3.2 collection.
Score × cost
Where performance meets price
AutomationBench-AA · 1M input + 250K output tokens · uncached USD
18 priced resultsPareto frontier
Google: Gemini 4 Argon ↗77.51% AutomationBench-AA · $2.25 workload · Google AI Studio · via OpenRouter
Evaluation: Effort: high · High. Price: $1.00 input / $5.00 output per 1M tokens · google-ai-studio/flex · tier=flex.
Numbers match the model key. Shared positions show an index range.
Model key Score / workload
- OpenAI: GPT-6 Luna53.17% / $0.1125
- DeepSeek: DeepSeek V4.1 Flash68.89% / $0.135
- GLM-5.3-Flash60.37% / $0.1375
- Qwen3.8-27B48.24% / $0.145
- Reflection: Beam 1.0 501B A23B48.46% / $0.50
- MiniMax-M321.25% / $0.555
- Xiaomi: MiMo-V2.6-Pro58.62% / $0.6475
- GLM-5.362.2% / $0.8763
- Step-5-Preview50.96% / $1.675
- Thinking Machines: Inkling4.98% / $1.9625
- Google: Gemini 4 Argon77.51% / $2.25
- OpenAI: GPT-6.1 Sol64.87% / $2.25
- Meta: Muse Spark Open49.71% / $2.3125
- Kimi-K358.27% / $2.529
- Anthropic: Claude Sonnet 5.571.77% / $4.50
- Anthropic: Claude Opus 5.569.54% / $9.00
- OpenAI: GPT-6 Astra68.49% / $11.25
- Anthropic: Claude Fable 5.159.38% / $22.50
Pareto frontier
- OpenAI: GPT-6 LunaScore53.17%Workload cost$0.1125OpenAI · via OpenRouter · openai/flex · tier=flex · prompt ≤ 272000 tokensEvaluation: Effort: max · Max. Price: $0.05 input / $0.25 output per 1M tokens · openai/flex · tier=flex · prompt ≤ 272000 tokens.
- DeepSeek: DeepSeek V4.1 FlashScore68.89%Workload cost$0.135Decart · via OpenRouter · decart/fp4 · quantization=fp4Evaluation: Effort: max · Max. Price: $0.09 input / $0.18 output per 1M tokens · decart/fp4 · quantization=fp4.
- Google: Gemini 4 ArgonScore77.51%Workload cost$2.25Google AI Studio · via OpenRouter · google-ai-studio/flex · tier=flexEvaluation: Effort: high · High. Price: $1.00 input / $5.00 output per 1M tokens · google-ai-studio/flex · tier=flex.
Frontier: no result is cheaper with an equal or better score, or equally priced with a better score. Each point uses its lowest generally available workload price from one endpoint; evaluation and provider configurations remain visible.
| # | Model | AutomationBench-AA | Configuration | API rate / unit | Source |
|---|---|---|---|---|---|
| 1 | Google: Gemini 4 Argongoogle | 77.51% | Effort: high · HighIndependent result | $1.00 in / $5.00 out/ 1M tokens · Google AI Studio · via OpenRoutergoogle-ai-studio/flex · tier=flex | Result ↗Oct 18, 2026 |
| 2 | Anthropic: Claude Sonnet 5.5anthropic | 71.77% | Effort: max · Max, Default FallbackIndependent result | $2.00 in / $10.00 out/ 1M tokens · Amazon Bedrock · via OpenRouteramazon-bedrock | Result ↗Oct 18, 2026 |
| 3 | Muse Wonder FullMeta | 71.33% | Effort: max · MaxIndependent result | See provider quotes | Result ↗Oct 18, 2026 |
| 4 | Anthropic: Claude Opus 5.5anthropic | 69.54% | Effort: max · Max, Default FallbackIndependent result | $4.00 in / $20.00 out/ 1M tokens · Amazon Bedrock · via OpenRouteramazon-bedrock | Result ↗Oct 18, 2026 |
| 5 | DeepSeek: DeepSeek V4.1 Flashdeepseek | 68.89% | Effort: max · MaxIndependent result | $0.09 in / $0.18 out/ 1M tokens · Decart · via OpenRouterdecart/fp4 · quantization=fp4 | Result ↗Oct 18, 2026 |
| 6 | OpenAI: GPT-6 Astraopenai | 68.49% | Effort: max · MaxIndependent result | $5.00 in / $25.00 out/ 1M tokens · OpenAI · via OpenRouteropenai/flex · tier=flex · prompt ≤ 272000 tokens | Result ↗Oct 18, 2026 |
| 7 | Muse WonderMeta | 68.09% | Effort: max · MaxIndependent result | See provider quotes | Result ↗Oct 18, 2026 |
| 8 | Grok 4.7SpaceXAI | 65.56% | Effort: xhigh · XhighIndependent result | See provider quotes | Result ↗Oct 18, 2026 |
| 9 | OpenAI: GPT-6.1 Solopenai | 64.87% | Effort: max · MaxIndependent result | $1.00 in / $5.00 out/ 1M tokens · OpenAI · via OpenRouteropenai/flex · tier=flex · prompt ≤ 272000 tokens | Result ↗Oct 18, 2026 |
| 10 | GLM-5.3zai-org | 62.2% | Effort: max · MaxIndependent result | $0.0632 in / $4.40 out/ 1M tokens · Relace · via OpenRouterrelace | Result ↗Oct 18, 2026 |
| 11 | GLM-5.3-Flashzai-org | 60.37% | Effort: maxIndependent result | $0.06 in / $1.60 out/ 1M tokens · Reka · via OpenRouterreka | Result ↗Oct 18, 2026 |
| 12 | Gemini 3.8 FlashGoogle | 59.93% | Effort: high · HighIndependent result | See provider quotes | Result ↗Oct 18, 2026 |
| 13 | Mistral Large 4 PreviewMistral | 59.9% | Effort not specifiedIndependent result | See provider quotes | Result ↗Oct 18, 2026 |
| 14 | Anthropic: Claude Fable 5.1anthropic | 59.38% | Effort: max · Max, Default FallbackIndependent result | $10.00 in / $50.00 out/ 1M tokens · Amazon Bedrock · via OpenRouteramazon-bedrock | Result ↗Oct 18, 2026 |
| 15 | Xiaomi: MiMo-V2.6-Proxiaomi | 58.62% | Effort not specifiedIndependent result | $0.43 in / $0.87 out/ 1M tokens · DeepInfra · via OpenRouterdeepinfra/fp8 · quantization=fp8 | Result ↗Oct 18, 2026 |
| 16 | Kimi-K3moonshotai | 58.27% | Effort: max · MaxIndependent result | $0.27 in / $14.00 out/ 1M tokens · Relace · via OpenRouterrelace/fp4 · quantization=fp4 | Result ↗Oct 18, 2026 |
| 17 | Qwen3.8 Max (0902)Alibaba | 56.19% | Effort not specifiedIndependent result | See provider quotes | Result ↗Oct 18, 2026 |
| 18 | OpenAI: GPT-6 Lunaopenai | 53.17% | Effort: max · MaxIndependent result | $0.05 in / $0.25 out/ 1M tokens · OpenAI · via OpenRouteropenai/flex · tier=flex · prompt ≤ 272000 tokens | Result ↗Oct 18, 2026 |
| 19 | Step-5-Previewstepfun-ai | 50.96% | Effort not specifiedIndependent result | $1.00 in / $2.70 out/ 1M tokens · DeepInfra | Result ↗Oct 18, 2026 |
| 20 | Meta: Muse Spark Openmeta | 49.71% | Effort: xhigh · XhighIndependent result | $1.25 in / $4.25 out/ 1M tokens · baseten · via Hugging Face | Result ↗Oct 18, 2026 |
| 21 | Reflection: Beam 1.0 501B A23Breflection | 48.46% | Effort not specifiedIndependent result | $0.25 in / $1.00 out/ 1M tokens · Nebius · via OpenRouternebius/fp8 · quantization=fp8 | Result ↗Oct 18, 2026 |
| 22 | Qwen3.8-27BQwen | 48.24% | Effort: xhigh · XhighIndependent result | $0.02 in / $0.50 out/ 1M tokens · Tenstorrent · via OpenRoutertenstorrent/fp4 · quantization=fp4 | Result ↗Oct 18, 2026 |
| 23 | K2 Horizon 375B A23BInstitute of Foundation Models | 37.21% | Effort not specifiedIndependent result | See provider quotes | Result ↗Oct 18, 2026 |
| 24 | Claude Haiku 5.5Anthropic | 35.41% | Effort: max · MaxIndependent result | See provider quotes | Result ↗Oct 18, 2026 |
| 25 | MiniMax-M3MiniMaxAI | 21.25% | Effort not specifiedIndependent result | $0.28 in / $1.10 out/ 1M tokens · deepinfra · via Hugging Face | Result ↗Oct 18, 2026 |
| 26 | Nemotron 3.5 Super VLNVIDIA | 14.82% | Effort not specifiedIndependent result | See provider quotes | Result ↗Oct 18, 2026 |
| 27 | Thinking Machines: Inklingthinkingmachines | 4.98% | Effort: xhigh · XhighIndependent result | $0.95 in / $4.05 out/ 1M tokens · deepinfra · via Hugging Face | Result ↗Oct 18, 2026 |