Kimi K3: A Technical Deep Dive into the First Open 3T-Class Model

Kimi K3 is the most capable model Moonshot AI has released, a 2.8 trillion parameter system built on two architectural updates, Kimi Delta Attention and Attention Residuals, with native vision and a one million token context window. It is the world's first open 3T-class model, designed for frontier intelligence across long-horizon coding, knowledge work and reasoning, and Moonshot AI has committed to releasing the full model weights by 27 July 2026. In this deep dive, we walk through the architecture, the training and inference infrastructure, the benchmark results and the case studies from the official Kimi K3 release blog, then look at what it takes to be ready to run the model on your own infrastructure the day the weights land.

While its overall performance still trails the strongest proprietary systems, Claude Fable 5 and GPT 5.6 Sol, Moonshot AI reports that Kimi K3 demonstrated frontier-level performance across its evaluation suite, consistently outperforming the remaining models tested. That places K3 in new territory for the open ecosystem: a model you will be able to download, inspect and serve yourself, competing on the same benchmarks as the largest closed models. At launch, Kimi K3 runs at max thinking effort by default, with low and high effort modes planned for subsequent updates, and a full technical report on the architecture, training and evaluations is to follow.

What Makes Kimi K3 an Open 3T-Class Model

Kimi K3 is the first open model to reach 2.8 trillion parameters. It is also the latest step in a sustained push at the scaling frontier: for nine of the past twelve months, Kimi models have set the upper bound of open-model sizes. The chart below shows the latest flagship model size for each major open-model lab as of July 2026.

Raw parameter count is only half the story. The release states that the architectural changes in K3, together with refined training and data recipes, yield an approximate 2.5x improvement in overall scaling efficiency compared with Kimi K2, allowing the model to convert compute into intelligence more effectively. Here is the model at a glance:

Kimi K3 Architecture: KDA, Attention Residuals and Stable LatentMoE

Kimi K3 is built on Kimi Delta Attention (KDA) and Attention Residuals (AttnRes), two architectural updates designed to improve how information flows across sequence length and model depth. KDA provides an efficient foundation for scaling attention, while AttnRes selectively retrieves representations across depth rather than accumulating them uniformly. Together, they form the architectural backbone of a model designed to scale well beyond the trillion-parameter regime.

At the sparsity level K3 operates at, 16 experts activated out of 896, routing and optimisation become first-order challenges. The release describes four techniques that make training stable at this scale:

• Quantile Balancing derives expert allocation directly from router-score quantiles, eliminating heuristic updates and a sensitive balancing hyperparameter.
• Per-Head Muon extends the Muon optimiser by optimising attention heads independently, for more adaptive learning at scale.
• Sigmoid Tanh Unit (SiTU) improves activation control.
• Gated MLA improves attention selectivity.

Together, these advances enable stable and efficient training at the 2.8 trillion parameter scale.

Quantisation and Inference Infrastructure

Kimi K3 applies quantisation-aware training from the SFT stage onward, using MXFP4 weights with MXFP8 activations for broad hardware compatibility. To prevent expert imbalance from degrading throughput at large expert-parallel scales, Moonshot AI introduces a fully balanced expert-parallel training method with static shapes and no host synchronisation on the critical path.

Two infrastructure details matter if you plan to serve this model yourself:

• Because inference efficiency benefits from larger high-bandwidth communication domains, Moonshot AI recommends deploying Kimi K3 on supernode configurations with 64 or more accelerators.
• KDA poses new challenges for conventional prefix caching, so Moonshot AI has contributed a corresponding implementation to the vLLM community, to be released alongside the model. KDA with prefill cache allows Kimi K3 to be served at a highly competitive token price despite its scale and long context.

Kimi K3 Coding Performance

Kimi K3 has strong long-horizon coding performance. Operating with minimal human oversight, it can sustain long engineering sessions, navigate massive repositories and orchestrate terminal tools. It also excels at tasks that blend software engineering with visual reasoning, leveraging screenshots and visuals to optimise game development, frontend work and CAD.

The chart below takes a representative slice of the published results, comparing Kimi K3 at max thinking effort against Claude Fable 5, GPT 5.6 Sol and Claude Opus 4.8 across coding, agentic, reasoning and document-understanding benchmarks. The full results, including GPT 5.5 and GLM-5.2, follow later in this article.

Benchmarks only frame the picture. What sets the Kimi K3 release apart is the set of case studies showing the model doing sustained, open-ended engineering work. The next five sections walk through them in detail.

Case Study 1: GPU Kernel Optimisation

Moonshot AI tested each model's capability to optimise GPU kernels. Every model worked independently in an identical sandbox, with up to 24 hours to profile, rewrite and benchmark four tasks spanning AttnRes, DSA, KDA and a 512-head-dimension MLA kernel, across NVIDIA H200 and a GPGPU from an alternative vendor. Kimi K3 performed competitively with Claude Fable 5 (with fallback) and substantially outperformed Claude Opus 4.8, GPT 5.6 Sol and GPT 5.5.

AttnRes Kernel Optimisation

Given the FLA Triton implementation of AttnRes at its production shape, 96 layers, model dimension 8192 and 8192 tokens, the task is to make the training-side operation as fast as possible without changing the numerics. Across 15 hours of nonstop iteration, Kimi K3 designed a novel two-phase kernel algorithm and fused kernels while preserving the same numerics, cutting forward and backward time from 283.6 ms to 114.4 ms. Kimi K3 and Claude Fable 5 both reached similar final performance, with K3 optimising faster per iteration.

DSA Kernel Optimisation

Starting from a TileLang implementation of the DSA kernel end to end, this task targets a realistic training setting with one million token sequences and model hyperparameters aligned with DeepSeek-V3.2. Each model optimised the DSA training kernel while preserving the numerics. Kimi K3 cut end-to-end time by 55.1% versus the baseline, second to Claude Fable 5 at 57.3%.

MLA-512 Kernel, Written From Scratch

This arena offers no kernel baseline at all. Each model writes an MLA kernel with a 512 head dimension from scratch, at the production shape of batch 1, 64 heads and 8192 tokens, and the scoreboard is raw achieved throughput rather than a ratio. Kimi K3's kernel reached 517.8 TFLOPS on forward and backward, over half of the H200's theoretical BF16 peak, ahead of the next best model at 492.7 TFLOPS.

KDA Kernel Optimisation on Alternative-Vendor GPGPU

The same rules applied to the FLA Triton implementation of KDA, but on a GPGPU from an alternative vendor. The software stack and memory behaviour differ enough that CUDA tuning experience does not carry over, so the performance model has to be rebuilt from profiling. Kimi K3 cut forward and backward time by 73.6% versus the baseline, with headroom likely still remaining.

There is also a telling production detail in the release: in the late stages of Kimi K3 development, an early version of K3 handled the majority of the team's own kernel optimisation work.

Case Study 2: Building a GPU Compiler from Scratch

Moonshot AI went a step further and tested whether Kimi K3 could build a GPU programming system from scratch. The result is MiniTriton, a compact Triton-like compiler with its own tile-level IR layer over MLIR, optimisation passes and a PTX code-generation pipeline.

The results are notable for a from-scratch system:

• Across supported roofline benchmarks, MiniTriton delivers performance on par with or better than Triton and torch.compile, beating Triton on certain workloads. The published CUDA-core roofline figure was measured on an NVIDIA L20.
• Beyond microbenchmarks, MiniTriton sustains end-to-end nanoGPT training with stable convergence, the loss curve closely tracking the reference with only minor divergence, validating the full pipeline on a realistic workload.
• Its from-scratch Tensor Core path already rivals Triton's extensively optimised stack.

Case Study 3: Game Development and Digital Creation

Kimi K3 combines strong 3D reasoning, coding and vision capabilities to turn concepts, images and videos into fully playable interactive experiences. The release describes this as vision in the loop: the model iterates between code and live screenshots, seeing and refining its own output.

The flagship demo is a fully procedural, browser-based 3D exploration game built with Three.js WebGPU and GPU compute. Kimi K3 procedurally generated the environment, producing an expansive open world with forests, a log-cabin village, snowy mountains and dynamic weather, and used a 3D asset generation tool for the rider and horse models. Moonshot AI lists the animated cowboy and horse models and the terrain data as external assets. You can play it in your browser at the live demo link.

A rider exploring Kimi K3's procedural 3D open world. Video: Moonshot AI, Kimi K3 release blog.

The same capability set produced nine interactive demos in total, every one of them running in a browser. Each card below links to the live build.

3D Open World

A procedural browser-based world in Three.js WebGPU, with forests, a village, snowy mountains and dynamic weather.

3D GBA Emulator

A Game Boy Advance emulator built around the mGBA WASM core, with the interface and gameplay refined through recursive self-improvement.

Cyberpunk Web-Swinging Game

Pendulum swing physics with energy-conserving rope constraints, polished to a smooth 120 FPS.

3D Mechanical Typewriter

A 1930s mechanical typewriter built to a museum-grade spec, with 242 individually animated parts and OOXML (.docx) export.

Voxel-Art Colosseum

A voxel-art Roman Colosseum with gladiator duels, animated crowds and physics-driven interactions.

3D Fighting Game

A browser-based 3D fighting game on a custom WebGPU engine, with combo combat, AI opponents and procedurally generated arenas.

Wuxia-Style RPG

A browser-based 3D martial arts role-playing game with melee combat, quests, inventory and explorable interiors.

4v4 FPS Arena

A four against four first-person shooter arena with eight weapons, AI teammates and opponents, and destructible environments.

Recreating Interstellar's Gargantua

A recreation of the Gargantua black hole from Interstellar, arrived at through 62 self-screenshots, each one read, diagnosed and acted on.

Nine interactive demos built by Kimi K3. Thumbnails and live demos: Moonshot AI, Kimi K3 release blog.

Case Study 4: A Chip Built by a Model, for a Model

As an early proof of concept, Kimi K3 designed a chip to serve a nano model built on its own architecture. In a single 48-hour autonomous run, K3 built, optimised and verified the chip using open-source EDA tools on the Nangate 45 nm library. The numbers below tell the story of how far a long-horizon agent can carry a hardware project without human intervention.

The 48-hour autonomous chip design run. Video: Moonshot AI, Kimi K3 release blog.

Case Study 5: Coding for Research

Kimi K3 bridges scientific literature and executable code, autonomously implementing, validating and analysing complex computational research workflows. In the flagship example, K3 reproduced the I-Love-Q universal relations in computational astrophysics, completing in about two hours what would typically require one to two weeks of work by an experienced researcher. Along the way it:

• Reviewed and cross-validated more than 20 papers, identifying inconsistencies in published formulas.
• Implemented the full numerical pipeline and evaluated more than 300 equations of state.
• Generated more than 3,000 lines of Python code.
• Produced an interactive HTML dashboard for exploring the results.

The live dashboard Kimi K3 produced for the I-Love-Q reproduction. Open it full screen at coding-science.ok.kimi.link. Built by Kimi K3, published by Moonshot AI.

Kimi K3 for Knowledge Work

Kimi K3 advances end-to-end knowledge work. Beyond public benchmarks, Kimi K3 at max effort demonstrates consistent gains across Moonshot AI's internal evaluations, which are derived from recurring patterns and challenges observed in production user-agent workflows. These consistent advantages across distinct production-oriented workflows reflect a broad improvement in K3's agentic knowledge work capabilities. Three published case studies show the scale it operates at.

42 Years of the AI ASIC Industry, as an Interactive Website

Kimi K3 produced an interactive research report you can drill into, covering 42 years of the AI ASIC industry, created through more than 120 rounds of recursive self-improvement. It transformed evidence into bespoke charts, animated diagrams and interactive visual narratives. The research effort behind it is worth pausing on:

The interactive research website itself, embedded live. Open it full screen at asic42.ok.kimi.link. Built by Kimi K3, published by Moonshot AI.

Consulting-Grade and Scientific Reports

Two further cases round out the picture. A fusion industry research report in consulting style pairs interactive visualisations, including timelines, a funnel chart, a range bar chart and Gantt charts, with publication-quality slides. And a GWTC-5 gravitational-wave analysis examined 391 events using more than 20 concurrent subagents, producing 7 scientific visualisations, 2 tables and a literature synthesis from more than 10 papers. The release also highlights K3's strength at infographic-style output, including a fully editable heatmap and an annual report.

Widgets and Dashboard in Kimi Work

Alongside the model, Kimi Work introduces two features that make interactions with Kimi K3 more visual and persistent. Widgets let you generate interactive components directly within a chat, with connections to local data or external plugins for continuous updates. Dashboard brings the widgets you care about most into one persistent, personalised view organised around a topic, project or goal.

Widgets and Dashboard in Kimi Work. Video: Moonshot AI, Kimi K3 release blog.

Video Editing and Native Multimodality

Kimi K3 excels at motion design, animation and video editing because its native multimodal architecture understands text, images and video within the same model. Two examples from the release make the point vividly.

First, K3 created a 3Blue1Brown-style motion-graphics explainer of its own architecture, translating the architecture described above into animated diagrams and transitions:

A 3Blue1Brown-style explainer of the Kimi K3 architecture, made by Kimi K3. Video: Moonshot AI.

Second, Kimi K3 edited its own teaser video from 56 source clips, handling clip selection, motion-matched cuts, frame-accurate beat synchronisation, audio processing and multiple rounds of revision. Moonshot AI notes that a high-density short video like this would typically take an experienced editor one to two working days, or a beginner three to five:

The Kimi K3 teaser, self-edited from 56 source clips. Video: Moonshot AI, Kimi K3 release blog.

Kimi K3 Benchmarks in Full

Below are the complete benchmark results from the Kimi K3 release, split into four tables, one per family, so each can be read on its own. All Kimi K3 results are obtained with reasoning effort set to max, temperature 1.0 and top-p 1.0. Depending on the benchmark, each model is evaluated under one of three agentic harnesses, Kimi Code, Claude Code or Codex. An asterisk denotes the footnoted evaluation conditions in the original post, and a dash indicates a score was not reported.

Coding Benchmarks: Repository, Terminal and ML Engineering

The coding suite spans repository-level software engineering, terminal operation and machine-learning engineering work. Kimi K3 posts the top score of any model on Program Bench and SWE Marathon, and lands within six points of the best score on every other row.

SWE Marathon is the row worth dwelling on, because it is the long-horizon test: K3's 42.0 is two points clear of Claude Opus 4.8, three clear of GPT 5.6 Sol and seven clear of Claude Fable 5, although Moonshot AI notes that Claude Fable 5 hit fallbacks on 35% of the SWE Marathon tasks in its evaluation, which may have depressed that score. That ordering matches the case studies earlier in this article, where the model's strength showed up in runs measured in hours rather than minutes.

Agentic Benchmarks: Tool Use, Browsing and Office Work

These rows measure tool use, autonomous browsing and office-style workflows. Kimi K3 takes the top reported score on BrowseComp, DeepSearchQA, Automation Bench and SpreadsheetBench 2, which is the strongest cluster of outright wins in the whole benchmark set.

Two qualifications belong with these numbers. Toolathlon-Verified is the one row where K3 sits behind the other four frontier models rather than among them. And on BrowseComp the published figure uses the context-compaction strategy from the Claude model cards, triggered at 300K tokens; run instead with the full one million token window and no context management, K3 scores 90.4.

Reasoning and Knowledge Benchmarks

This is the shortest family in the set, three rows, and it carries the widest percentage-point gaps anywhere in the four tables. Kimi K3 reaches 93.5 on GPQA-Diamond, six tenths of a point behind the best result and level with GPT 5.5.

Humanity's Last Exam is where the distance to the strongest proprietary model is clearest: K3 trails Claude Fable 5 by 9.8 points without tools and by 7.0 points with them. This is the gap Moonshot AI refers to when it says overall performance still trails the most powerful closed models.

Vision Benchmarks: Documents, Charts and Visual Mathematics

Kimi K3 has native vision, with text, images and video handled inside the same model, and these rows test document parsing, chart reading and visual mathematics. Rows marked "w/ python" allow the model to write and run code while answering, which lifts every model's score.

K3 leads outright on OmniDocBench at 91.1, the document-parsing row, and ties Claude Fable 5 at the top of ZeroBench_main (pass@5). Across the remaining vision rows it sits second or third, and never more than six points behind the leader.

Taken together, the four families point the same way. Kimi K3 beats GLM-5.2, the only other open model with reported scores, on every one of the nineteen rows where both are measured, and on most of the rest it sits within a few points of Claude Fable 5 and GPT 5.6 Sol. Open weights with results in that range are what the release means by frontier-level performance from an open 3T-class model.

Working with Kimi K3 in Practice

The release includes two pieces of practical guidance that anyone integrating Kimi K3 into an agent stack should design around from day one:

• Preserve the thinking history. K3 was trained in preserved thinking history mode, so your agent harness should pass back all historical thinking content across turns, and a session should start on K3 rather than being switched over from another model midway. Moonshot AI recommends a harness with verified compatibility, such as Kimi Code.
• Set explicit boundaries for scoped tasks. K3's training places particular emphasis on long-horizon, challenging tasks, so when it meets minor issues or ambiguous intent during a task it may make decisions on the user's behalf. If your application requires the agent to operate within well-defined boundaries, state those behavioural constraints explicitly in the system prompt or in AGENTS.md.

Kimi K3 Availability and API Pricing

Kimi K3 is live today across the Kimi product line, and the API is the route most engineering teams will take first. On the Kimi API Platform you select kimi-k3 as the model name. Pricing reflects the KDA prefill cache work described earlier: powered by Mooncake's disaggregated inference architecture, the official Kimi API achieves a cache hit rate above 90% in coding workloads, which makes the cache-hit input price the number that matters most in practice.

Across the rest of the product line: the Kimi app ships K3 on iOS, Android and HarmonyOS and at kimi.com; the Kimi Work desktop app carries it from version 3.1.0 on Windows and Apple silicon Macs; Kimi Code exposes it in the terminal through the /model command; and Kimi Enterprise adds enterprise-grade data privacy and member management with complete separation between personal and organisation accounts.

Getting Ready for Kimi K3 on Hyperstack

Here is the part that matters most for teams that self-host. The full Kimi K3 weights arrive by 27 July 2026, and the release already tells you what serving it well will take: the model uses MXFP4 weights for broad hardware compatibility, Moonshot AI recommends supernode configurations with 64 or more accelerators for inference, and a KDA prefill cache implementation is landing in vLLM alongside the model. In other words, Kimi K3 is arriving with a self-hosting playbook attached, and Hyperstack already provides every piece of it.

The practical preparation path looks like this:

• Build your serving pipeline today on Kimi K2.6. The current open Kimi flagship deploys on a single 8x NVIDIA GPU Hyperstack VM with vLLM, and our step-by-step guide, Deploy Kimi K2.6 on Hyperstack, walks through the whole flow, from ephemeral NVMe weight storage to the OpenAI-compatible endpoint. The tooling, parsers and agent integrations you set up for K2.6 are the same ones K3 will slot into.
• Practise multi-node serving with Kubernetes. A 2.8 trillion parameter model with a recommendation of 64 or more accelerators is a multi-node deployment by definition. Hyperstack On-Demand Kubernetes provisions production-ready NVIDIA GPU clusters, and our LLMs on Kubernetes quick-start guide covers running vLLM API servers across cluster nodes.
• Size the hardware ahead of the drop. MXFP4 packs two parameters per byte, so 2.8 trillion parameters imply a checkpoint in the region of 1.4 TB before overheads, comfortably beyond a single 8x 80 GB node. Hyperstack offers NVIDIA H100 SXM and NVIDIA H200 SXM nodes on demand, with NVIDIA Blackwell reservations available for teams planning serious K3 serving capacity.

Get ready for the weights

Be ready for the Kimi K3 weights

The full 2.8 trillion parameter weights land by 27 July 2026. Prove your serving stack on Hyperstack now, so the day they drop is a deployment rather than a scramble.

8x H100 and H200 nodes on demand On-Demand Kubernetes NVMe for terabyte checkpoints

Launch an NVIDIA GPU cluster on Hyperstack today.

FAQs

What is Kimi K3?

Kimi K3 is Moonshot AI's most capable model, a 2.8 trillion parameter system built on Kimi Delta Attention and Attention Residuals, with native vision capabilities and a one million token context window. It is the world's first open 3T-class model, designed for frontier intelligence across long-horizon coding, knowledge work and reasoning.

Is Kimi K3 open source?

Kimi K3 is currently available through Kimi.com, Kimi Work, Kimi Code and the Kimi API, and Moonshot AI has committed to releasing the full model weights by 27 July 2026. The company is working with inference partners and open-source maintainers to align technical details and ensure a reliable rollout across the ecosystem.

What architecture does Kimi K3 use?

Kimi K3 combines Kimi Delta Attention (KDA), Attention Residuals (AttnRes) and a Stable LatentMoE framework that effectively activates 16 of 896 experts. Training at this scale is stabilised by Quantile Balancing, Per-Head Muon, the Sigmoid Tanh Unit (SiTU) and Gated MLA. Together with refined training and data recipes, these changes deliver an approximate 2.5x improvement in scaling efficiency over Kimi K2.

How much does the Kimi K3 API cost?

On the Kimi API Platform, kimi-k3 is priced at $0.30 per million tokens for cache-hit input, $3.00 per million tokens for cache-miss input and $15.00 per million tokens for output. The official API reports a cache hit rate above 90% in coding workloads, powered by Mooncake's disaggregated inference architecture.

What hardware will Kimi K3 need for self-hosting?

Kimi K3 ships with MXFP4 weights and MXFP8 activations from quantisation-aware training, which implies a checkpoint in the region of 1.4 TB before overheads for the 2.8 trillion parameters. Moonshot AI recommends deploying on supernode configurations with 64 or more accelerators, which on Hyperstack maps to multi-node NVIDIA GPU clusters orchestrated with On-Demand Kubernetes and served with vLLM, where Moonshot AI has contributed KDA prefix caching support.

How does Kimi K3 compare with Claude Fable 5 and GPT 5.6 Sol?

Moonshot AI reports that Kimi K3 trails the strongest proprietary models overall while demonstrating frontier-level performance across its evaluation suite. In the published benchmark table it posts the top score among models with reported results on Program Bench, SWE Marathon, BrowseComp, DeepSearchQA, Automation Bench, SpreadsheetBench 2 and OmniDocBench, and sits within a few points of Claude Fable 5 and GPT 5.6 Sol on most other benchmarks.