Kimi K3 Technical Report: What It Reveals

The Kimi K3 technical report runs 47 pages, and Moonshot published it alongside the weights on July 27, 2026. Almost all the coverage that followed reused the same four numbers off the model card. There is considerably more in the document: a benchmark result no open model has ever posted, a cost comparison that undercuts the pricing everyone is actually paying, and a security evaluation that US labs structurally cannot publish.

This is a read of what's in the document, not a rewrite of the spec sheet. For the specs, what is Kimi K3 covers those.

Key Takeaways:

• K3 ranks first of 99 models on WebDev Arena at 1,678 Elo, the first open model ever to top that leaderboard.
• The architecture claims roughly 2.5x better scaling efficiency than Kimi K2, from three changes: Kimi Delta Attention, Attention Residuals, and a much sparser MoE.
• On BrowseComp it posts the best score in the report at $2.03 per task, about half what GPT-5.6 Sol costs for a slightly lower score.
• Moonshot's own abstract concedes K3 "still trails the most powerful proprietary models."
• A cybersecurity section reports 16 previously unknown vulnerabilities found across six projects, including two in the Linux kernel.
• Training token count, knowledge cutoff, training data, and training code are all still undisclosed in the Kimi K3 technical report.

The Architecture: Three Changes, One Efficiency Claim

The Kimi K3 technical report's headline architectural claim is a roughly 2.5x improvement in overall scaling efficiency over Kimi K2, meaning more capability per unit of training compute rather than per parameter. Three pieces carry it.

Kimi Delta Attention (kernels now open-sourced as FlashKDA) handles 69 of the model's 93 layers, with 24 Gated MLA layers interleaved to preserve global interaction. The split matters: linear-attention variants are cheap over long sequences but lose the ability to attend precisely across the whole context, so Moonshot pays for periodic full-attention layers rather than going all-in on the cheap mechanism. Attention Residuals let each layer selectively pull from the representations of every preceding layer instead of only the one below it. And Stable LatentMoE pushes routing to 896 experts with just 16 active per token plus 2 shared, which is far sparser than the K2 generation.

Extreme sparsity is where MoE training usually falls over, because a handful of experts hog the routing and the rest never learn. The report's answer is Quantile Balancing: instead of tuning a load-balancing loss, it sets each expert's routing bias from the router-score quantile that matches its target load, derived in a single forward pass by taking Top-(k+1) and using the (k+1)-th score as the cutoff each expert must clear. There's also a custom optimizer variant, Per-Head Muon, which the report says is cheaper than running Newton-Schulz iterations over the full projection matrix.

The rest of the spec, for reference: 2.8T total parameters with 104B activated, a 160K vocabulary, MoonViT-V2 as the vision encoder at 401M parameters, SiTU-GLU activations, and MXFP4 weights with MXFP8 activations trained quantization-aware from the supervised fine-tuning stage onward. That last detail is the reason the published checkpoint is 1.56 TB rather than several times that; the quantized form isn't an afterthought applied at release, it's what the model was tuned to live in.

The Benchmark Result Everyone Skipped

Independent evaluators, not Moonshot's own harness, produced the most interesting line in the Kimi K3 technical report. On the crowdsourced human-preference boards, K3 ranks first of 99 models on WebDev Arena at 1,678 Elo, ahead of Claude Fable 5 at 1,634. No open-weight model had topped that leaderboard before.

Read alongside the rest of the third-party table, the picture is consistent rather than flukey:

Front-end and web-dev work is where K3 wins outright; general text and agent orchestration are where it sits behind. That's a narrower and more useful claim than "third-best model in the world," and it happens to match what builders were posting during launch week, when the demos that went viral were overwhelmingly websites, games, and 3D scenes rather than research or long-horizon planning.

The self-reported numbers deserve the usual caution, though the report is unusually careful about harness disclosure: K3 runs on Moonshot's own Kimi Code harness for most coding benchmarks while competitors run on Claude Code or Codex, and the footnotes say so explicitly, including the detail that Claude Fable 5 hit fallbacks on 35% of tasks in their SWE-Marathon evaluation. Vendor benchmarks that document their own confounders are rarer than they should be.

Figure 1, Kimi K3 Technical Report (Moonshot AI). K3 in blue; every model run at maximum thinking effort.

The Cost Curves Argue Against the Price You'll Pay

Section 6.4 compares score against per-task cost across four suites, and it's the part of the report most relevant to anyone actually budgeting.

On Kimi Code Bench 2.0, K3 lands 4.0 points behind Claude Fable 5 while costing 38% as much, and at high reasoning effort it matches Claude Opus 4.8's maximum-effort score for roughly a third of the price. On BrowseComp it posts the best score in the report, 91.2%, at $2.03 per task, about half what GPT-5.6 Sol spends to score 90.4% and an order of magnitude below the Claude models at max effort. On GDPval-AA v2 it finishes within 50 Elo of Sol at 13% lower cost. On AA-Briefcase it takes second place behind Fable 5 at roughly half the cost.

Table 5 and Figure 13, Kimi K3 Technical Report (Moonshot AI). K3 is the star marker; note it sits left of Claude Fable 5 on every panel.

There's a real irony sitting under those charts. The cost-efficiency argument is K3's strongest, and the open-weights release did nothing to strengthen it: every hosted provider charges exactly $3 input and $15 output per million tokens, matching Moonshot's own API to the cent, with no free tier anywhere. The efficiency shows up in tokens consumed per task, not in the rate you're billed. We dug into why the price never moved in Kimi K3 free access.

One case study is worth pulling out because it's unusually concrete. Given twenty hours to optimize a GPU kernel for the model's own Attention Residuals layer, K3 reached a 59.7% speedup over the flash-linear-attention Triton baseline, against 57.1% for Claude Fable 5, 30.8% for GPT-5.5, and 17.3% for GPT-5.6 Sol. A model tuning the kernels for its own architecture is not a neutral test, and the report doesn't pretend otherwise, but the gap over Sol is wide enough to be interesting.

Figure 14, Kimi K3 Technical Report (Moonshot AI). K3 also gets most of its gain in the first five hours.

The Security Section US Labs Can't Publish

Section 6.2.2 is the part of the report with no equivalent in an Anthropic or OpenAI system card, and the reason is stated plainly: frontier models from both labs refuse cyber-related tasks, which makes a comparable evaluation impossible, so Moonshot excluded them. K3 does not refuse.

On vulnerability discovery, the model was pointed at operating-system kernels, databases, AI services, web frameworks, blockchain, and VPN software, and produced hundreds of candidate findings. Of those that went to human review, roughly 70% were confirmed genuine, including 16 previously unknown vulnerabilities across six projects.

Section 6.2.2, Kimi K3 Technical Report (Moonshot AI), reproduced because the exclusion of Anthropic and OpenAI models is stated outright rather than buried.

Two Linux kernel results show the ceiling: a remotely triggerable heap out-of-bounds write introduced by an incomplete upstream fix and present in every release since, confirmed by security experts as a remote denial-of-service primitive; and a Dirty-COW-class bug in the RDMA subsystem where an earlier fix had quietly dropped a permission check, giving a deterministic local privilege-escalation primitive.

Exploit development is where it stalls. Against an in-house suite of 36 tasks, all verified solvable by human experts and estimated at roughly 540 expert-hours in total, K3 solved 14 (38.9%) versus GLM-5.2's 8 (22.2%). But 10 of those 14 came from the user-space track; on the Linux kernel track neither model cracked three-quarters of the problems. The report attributes the gap to four recurring failures: not closing out the final stage of an exploit chain from primitives it already had, picking bad strategies under mitigations (persisting with control-flow hijacking where a data-only attack would be simpler), sinking into unproductive debugging loops, and submitting deliverables it hadn't verified. Anyone who has watched an agent work a long task will recognise at least three of those.

A joint assessment by the UK AI Security Institute and NIST's Center for AI Standards and Innovation reached consistent conclusions independently. For anyone doing procurement, the finding that matters isn't the score; it's that a capable model which does not decline this category of work is now downloadable, and your own guardrails have to do the job the vendor's refusal behaviour used to. We cover that and the rest of the trade-offs in Kimi K3 limitations.

Four Repositories Shipped the Same Day

The weights got the attention, but Moonshot open-sourced four other things on July 27, and two of them are more immediately useful to practitioners than the 1.56 TB checkpoint:

• FlashKDA, a CUTLASS implementation of the Kimi Delta Attention kernels, reporting a 1.72x to 2.22x prefill speedup over the flash-linear-attention baseline on H20 and usable as a drop-in FLA backend.
• MoonEP, the expert-parallel communication library used for K3's own pre-training, aimed at cutting communication overhead in large MoE workloads.
• AgentENV, built with kvcache-ai, a distributed system for running agent environments at scale with snapshot, resume, and fork; it powered K3's agentic RL training.
• PerceptionBench, a benchmark derived backwards from how frontier models fail across 42 existing benchmarks, reduced to 10 atomic perceptual capabilities and 3,000 verified questions. K3 scores 58.5 on it; GPT-5.6 Sol scores 59.7.

Releasing the training infrastructure alongside the model is a meaningfully different posture from releasing weights alone, even though the training data and the training code for the model itself stayed private.

What the Report Still Doesn't Tell You

Four gaps survive all 47 pages of the Kimi K3 technical report, and they're the ones a careful reader should note.

The pre-training corpus is described only by domain: web text, code, mathematics, knowledge, plus a large vision corpus covering captions, interleaved image-text documents, OCR, perception, video, and programmatic visual data such as SVG, 3D assets, CAD schematics, and rendered webpages. No dataset names, no sources, no licensing discussion. There is no training token count anywhere in the document, which is unusual for a report that spends pages on scaling laws. There is no knowledge cutoff. And neither the training data nor the training code was released, so the model can be run and modified but not reproduced.

Combine that with a licence carrying revenue-triggered conditions, covered in our Kimi K3 license breakdown, and "open" is doing narrower work here than the word usually implies. Which doesn't diminish the release. It just means the accurate phrase is open-weight, and the report is a description of how the model was built rather than a recipe for building it.

FAQ

Where can I read the Kimi K3 technical report?

It's a PDF in the model's GitHub repository at github.com/MoonshotAI/Kimi-K3, published July 27, 2026 alongside the weights. It runs 47 pages across architecture, pre-training, post-training, infrastructure, evaluations, and case studies. Moonshot also posted a shorter tech blog covering the same ground.

What is Kimi Delta Attention?

The attention mechanism K3 is built on, used in 69 of its 93 layers, with 24 Gated MLA layers interleaved so the model keeps a path for global interaction. It's designed for efficient mixing over long sequences, which is what makes a 1M-token context economical to serve. Moonshot separately open-sourced FlashKDA, a CUTLASS kernel implementation reporting up to 2.22x faster prefill than the flash-linear-attention baseline.

Does the technical report say Kimi K3 beats Claude and GPT?

No, and the abstract says the opposite: K3 "still trails the most powerful proprietary models, namely Claude Fable 5 and GPT-5.6 Sol," while beating every other model in its comparison set. The report's stronger claims are narrower ones, chiefly that K3 tops WebDev Arena and sits on or near the cost-efficiency frontier across four benchmark suites.

How much training data was Kimi K3 trained on?

The report doesn't say. It describes the corpus by domain and explains the filtering and rephrasing pipeline, but publishes no token count, no dataset names, and no knowledge cutoff. Training data and training code were not released.

To go further: what is Kimi K3 for the specs and pricing, Kimi K3 license for the commercial terms, Kimi K3 limitations for the trade-offs, and Kimi K3 free access for what it costs to run.