Kimi K2.6 Benchmarks: How It Compares to Claude Opus 4.6, GPT-5.4 and Gemini 3.1 Pro

The Model Nobody Expected to Be This Competitive

Open-source frontier models have a pattern. They close the gap on benchmarks, then fall short on the tasks that matter to engineers: long agent runs, complex coding pipelines and multi-step reasoning under real-world constraints. Kimi K2.6 from Moonshot AI breaks that pattern in some places. In others, it confirms it.

The model uses a 1-trillion-parameter Mixture-of-Experts architecture. Per token, it activates 32B parameters, which is enough to run competitive inference on 8x H100-80G hardware while keeping the compute cost closer to a dense 32B model than to a dense 1T model. The context window is 256K tokens. The vision encoder (MoonViT, 400M parameters) handles image and video input natively.

That frontier-scale parameter count, along with the dense-model inference cost, is exactly what makes the benchmark results worth reading carefully.

Where Kimi K2.6 Leads

Start with software engineering, because that's where the results are least expected. 

• On SWE-Bench Pro, the harder, newer variant of the standard software engineering benchmark, K2.6 scores 58.6. Claude Opus 4.6 scores 53.4. GPT-5.4 scores 57.7. Gemini 3.1 Pro scores 54.2. K2.6 leads the entire field here, including proprietary models running at maximum effort or highest reasoning settings.
• SWE-Bench Verified tells a slightly different story. Claude Opus 4.6 edges ahead at 80.8, Gemini at 80.6, K2.6 at 80.2. The gap is less than one percentage point. For practical purposes, these three models are performing identically on real-world GitHub issue resolution.
• On DeepSearchQA, the separation is sharper. K2.6 scores 92.5 f1 and 83.0 accuracy. Claude Opus 4.6 scores 91.3 and 80.6. GPT-5.4 scores 78.6 and 63.7. Gemini scores 81.9 and 60.2. For any workload that requires deep factual retrieval across long contexts with search tools, K2.6 is the current leader by a clear margin.
• LiveCodeBench v6 continues the pattern. K2.6 at 89.6, Claude at 88.8, Gemini at 91.7. K2.6 beats Claude on live competitive coding but trails Gemini by two points.

On software engineering and deep search, K2.6 is not competing with proprietary models. It's ahead of most of them.

Where Closed Models Hold Their Ground

Pure reasoning is where the story changes.

• On HLE-Full without tools, Humanity's Last Exam, one of the most difficult knowledge benchmarks currently available, K2.6 scores 34.7. Claude Opus 4.6 scores 40.0. GPT-5.4 scores 39.8. Gemini 3.1 Pro scores 44.4. The gap here is real: K2.6 trails the field by 5 to 10 points.
• On AIME 2026, competition-level mathematics, GPT-5.4 scores 99.2 and K2.6 scores 96.4. On HMMT 2026, GPT-5.4 scores 97.7 and Claude Opus 4.6 scores 96.2 while K2.6 scores 92.7. On GPQA-Diamond, graduate-level science, Gemini leads at 94.3, followed by GPT-5.4 at 92.8, then Claude at 91.3, then K2.6 at 90.5.

This pattern is consistent. On tasks where the primary requirement is deep domain knowledge and mathematical reasoning from memory, the closed frontier models maintain a lead. It's not a wide lead. But it exists.

Worth Noting: when tools are added to HLE-Full, K2.6 jumps to 54.0, beating Claude (53.0), GPT-5.4 (52.1), and Gemini (51.4). The model's raw knowledge retrieval is weaker than its competitors but its ability to use tools effectively to compensate for that is stronger than any of them.

Vision: Strong, Not Dominant

K2.6 supports native image and video input via MoonViT. On MathVision with Python tool use, it scores 93.2, trailing only GPT-5.4 (96.1) and Gemini (95.7). On CharXiv with Python, it scores 86.7, again close to GPT-5.4 (90.0) and Gemini (89.9).

Claude Opus 4.6 lags significantly on vision benchmarks. On MathVision with Python, Claude scores 84.6 against K2.6's 93.2. On BabyVision with Python (spatial and physical reasoning from images), Claude scores 38.4 against K2.6's 68.5.

If your pipeline involves complex chart interpretation, visual reasoning or multimodal inputs at scale, K2.6's vision performance is meaningfully better than Claude's. It trails GPT-5.4 and Gemini on most vision tasks, but not by much.

Benchmark Results at a Glance

All results use maximum reasoning effort. K2.6 and K2.5 use thinking mode; Claude Opus 4.6 uses max effort; GPT-5.4 uses xhigh; Gemini 3.1 Pro uses high thinking. Asterisked results (*) were re-evaluated independently by Moonshot AI.

What the Numbers Mean for Real Workloads

Benchmarks are a proxy. The question that matters is: for what actual tasks does K2.6 change the decision?

• For coding agents and software engineering pipelines, K2.6 is now the strongest open-source option available, and it's competitive with the best closed models. If you're building a coding agent that runs autonomously against a real codebase, K2.6 at 58.6 on SWE-Bench Pro beats Claude Opus 4.6 by more than five points.
• For deep search and retrieval-augmented workloads, K2.6's DeepSearchQA lead over GPT-5.4 (92.5 vs 78.6 on f1) is significant enough to justify the infrastructure cost of self-hosting. A model running on your own H100 nodes that outperforms GPT-5.4 by 14 f1 points on search tasks is not a marginal improvement.
• For pure reasoning and knowledge retrieval without tools, the closed models still win. GPT-5.4 at 39.8 and Claude at 40.0 on HLE-Full beat K2.6 at 34.7. If your workload is heavy on graduate-level science or competition mathematics and the task can't use external tools, the closed models remain the right call.
• For multimodal workloads, K2.6 is the strongest open option and meaningfully outperforms Claude on vision tasks. If image or video understanding is part of your pipeline, K2.6 closes most of the gap to GPT-5.4 and Gemini while running on infrastructure you control.

The Open-Source Angle is Real This Time

Released under a Modified MIT license, K2.6 is fully open for commercial use, self-hosting, and modification. The weights are on Hugging Face. The architecture is the same as K2.5, so deployment tooling carries over directly.

Most “open” frontier models still come with constraints. Whether through non-commercial licensing, restrictions on deployment, or limited access to reasoning traces, K2.6 removes those barriers.

For organisations that need full control, such as regulated industries, data-sensitive environments or teams that require auditable inference, this matters more than any single benchmark score.

A 1-trillion-parameter MoE model at this performance level, with full commercial rights, was not available six months ago. That's the actual story here.

Deploying Kimi K2.6 on Hyperstack

Running a 1-trillion-parameter model in production is a different infrastructure problem than running a dense 7B or 70B model. K2.6 requires 8x H100-80G at a minimum. The weight is 595 GB. First-token latency, throughput per GPU and storage I/O all matter in ways that a standard cloud VM setup doesn't handle well.

Hyperstack is a cloud platform built specifically for this class of workload. Here's why:

• Multi-GPU nodes for trillion-parameter MoE workloads. K2.6 needs 8x H100-80G at a minimum. Hyperstack provides on-demand access to NVIDIA H100 VMs purpose-built for this scale of MoE deployment.

• High-speed ephemeral NVMe storage. The 6 TB ephemeral NVMe disk downloads K2.6's 595 GB weights in roughly 15 minutes. That's fast enough to iterate on multiple model versions in a single afternoon without rebuilding your environment.

• Pre-configured CUDA and Docker images. The Ubuntu 22.04 + CUDA 12.2 + Docker image removes the driver-and-runtime setup phase entirely. Deployment goes straight from SSH to Docker run without a half-day of environment debugging.

• Seamless scale-up. Move from an 8x H100-80G smoke-test node to an 8x H200-141G production node using the same Docker image and the same launch flags. No environment rebuild.

• Hibernation for cost control. Pay only when actively serving. Hibernate the VM between batches and your GPU billing stops while the entire setup is preserved.

• Native compatibility with the full inference stack. vLLM, SGLang, KTransformers, and Hugging Face — all inference engines Moonshot recommends for K2.6 — run on Hyperstack without extra configuration

If K2.6's benchmark profile fits your workload like coding agents, deep search or multimodal pipelines where you need model weights on your own infrastructure, Hyperstack gives you the cloud environment to run it at the scale the model requires.