Kimi K3.1 Pro Nearly Matches Claude Opus 5.5
Kimi K3.1 Pro Nearly Matches Claude Opus 5.5 — Weights Free in November
Three weeks after OpenAI accused people linked to Moonshot AI of trying to extract the hidden reasoning out of its models, Moonshot has answered the loudest way a lab can: its new Kimi K3.1 Pro sits one point behind the best closed model in America on the most-watched independent leaderboard — and the company promises to hand the whole thing over, weights and all, free, on November 4. Whether that date is a promise or a provocation is the story.
Some outlets datelined the launch October 24 — a timezone artifact; the announcement actually landed on the 23rd, through Moonshot's launch blog and a LinkedIn post that made the rounds before most American analysts had had coffee. "Introducing Kimi K3.1 Pro: our most capable 3T-class open-weight model," Moonshot wrote. Then the line that detonated the benchmark tables: "Kimi K3.1 Pro and K3.1 Pro Highspeed (2–3× faster) are now live on Kimi.ai, Kimi Work, Kimi Code, and the Kimi API. Open weights by Nov 4, 2026."
We cross-checked the benchmark claims across six outlets before printing them — and on the two numbers that matter most, independent verification splits exactly where you'd expect: the leaderboard score is third-party measured; almost everything else is the company's own homework. That split, more than any single score, is what this launch is really about.
Photo: A neural-network diagram showing the "weights" — the parameters Moonshot AI says it will publish for Kimi K3.1 Pro on November 4. The actual weights are not public yet. Credit: Geetika Saini via Wikimedia Commons, CC BY-SA 4.0
The headline number — and who actually measured it
Artificial Analysis, the independent benchmark outfit whose Intelligence Index has become the closest thing this industry has to a neutral scoreboard, puts Kimi K3.1 Pro at 55 — fourth overall out of 215 models, behind Claude Opus 5.5 (58), Claude Sonnet 5.5 (56) and Muse Wonder Full (56), and ahead of GPT-6 Astra, Gemini 4 Argon, Claude Fable 5.1 and GPT-6.1 Sol (all 52–53). It is eleven points clear of its own predecessor, Kimi K3, at 44. That measurement is real, third-party, and the reason this story is news.
Artificial Analysis also put the model's agentic Elo at 1,846 on GDPval-AA — third in the world behind Opus 5.5 (1,866) and Muse Wonder Full (1,857), up 312 on K3. Its launch-day analysis (we read a reproduced copy; the original page wasn't directly retrievable this session) makes the open-weight case plainly: once the weights ship, the leading open-versus-proprietary gap on the index shrinks from twelve points to three. Secondhand paraphrase — but the numbers underneath are AA's own, independently measured.
Then there is the coding-agents number everyone is quoting: 65.7% on Terminal-Bench 4.0, against 66.4% for Opus 5.5 and 58.2% for GPT-6 Astra. Here's the catch, and Moonshot deserves credit for disclosing it in its own chart: it ran its model inside Anthropic's Claude Code harness and lifted its rivals' scores from their published reports and the public leaderboard. The rivals were not re-run. The one-point gap is Moonshot-graded homework, not a neutral measurement — which is why the AA index, not the chart, is the number to trust.
Same model, far more training
The first thing to get straight: K3.1 Pro is not a new base model. Moonshot's launch blog describes it as a post-trained version of Kimi K3 "put through far more reinforcement learning across what it calls 'hundreds of downstream domains'." The architecture is unchanged — 2.8 trillion total parameters, 104 billion active (mixture-of-experts), a million-token context window — and thinking is now always on.
What one pass of extra reinforcement learning apparently bought is startling: on Moonshot's chart the predecessor scored 13% on Terminal-Bench against K3.1 Pro's 65.7% — though that 13% traces to an older version we couldn't cross-check independently, so take the size of the jump as the company's story. SWE-Marathon comes in at 51.7 against Opus 5.5's 46.3 and GPT-6 Astra's 42.5 — Moonshot leading the pack, on its own chart, again. Kimi Code Bench 3.0, the lab's internal long-horizon test, moved from 55.95 to 61.2. BenchCAD's 3D tool-use test: 92.7 against Opus 5.5's 96.2. OfficeQA Pro v2, office-document work, is claimed as state of the art at 64.4 — "claimed" doing heavy lifting, since no independent run exists. Every one of these, mark it down, is self-reported.
There is something shoddy about an industry where the loudest numbers on launch day are always the ones the vendor graded itself. That is not an accusation of fraud — Moonshot disclosed its methodology more openly than most — but it is why the "within a point of Opus 5.5" framing should live or die on the AA index, which genuinely measured the model, and nowhere else.
The weaknesses, in Moonshot's own charts
To the lab's credit, the weak spots are in the same slides: Terminal-Bench Science 0.1 is 38.1 for K3.1 Pro against 68.1 for GPT-6 Astra and 63.3 for Opus 5.5 — a roughly thirty-point deficit in a domain that matters enormously to research labs, and Moonshot published it anyway. Handyai's "Model Drop" newsletter caught the launch-day sceptics' mood precisely: "Early reactions: Impressed by the ranking, wary of Moonshot's test setup."
The other number worth a hard look is the hallucination claim. Moonshot's launch blog says claim-level errors on its internal "Kimi Hallu Bench Hard" fell from 23.4% to 8.3% — a 65% drop. But the API quickstart documentation shows 6.85% against 23.35%. Moonshot's own two documents disagree with each other, and nobody at the company has explained which test is the real one. One caveat: this is a stress-test hallucination benchmark, not an everyday error rate — but the inconsistency should have been caught before launch, and it wasn't.
The price is not what the headline says
The launch-day framing, including in one widely shared write-up, is that Kimi K3.1 Pro costs half of Opus 5.5. API pricing is $2/$10 per million input/output tokens, down from K3's $3/$15, with cache reads at $0.20 and a Highspeed variant at $4/$20 that Moonshot says runs two to three times faster with "the same abilities" — no separate benchmarks published, so that one's on trust. Vercel listed both model IDs on its AI Gateway the same day, independently corroborating the price and the million-token context.
But the "half of Opus" framing is aimed at the wrong comparison. The real like-for-like US price match is OpenAI's GPT-6.1 Sol — the same $2/$10 rate Kimi is now matching, as we reported when it launched. And then there is the verbosity tax, measured independently by Artificial Analysis: K3.1 Pro emitted 150 million tokens across AA's index runs against a comparable-model median of 87 million, working out to roughly $2 per task (one outlet reads it as $1.98, another as $2.11 — call it two dollars) versus $0.72 for GPT-6.1 Sol at the identical per-token price. Cheap per token is not cheap per task — price tasks, not tokens.
"Open weights by Nov 4" is a promise, not a fact
Every outlet has reported November 4 as though it were already on the calendar. It isn't. It is a promise — and until it ships, the model is API-only, every benchmark above is unreplicable by outsiders, and the weight licence is undisclosed. K3's Hugging Face weights shipped under Moonshot's own MIT-style licence, which requires a separate deal for API providers making more than $20 million in twelve months; whether K3.1 Pro gets the same terms or something with teeth in it is currently unknown.
Here is what nobody in the launch-day coverage itemised: if November 4 comes and goes, the "highest-scoring open-weight model" story is vapor, the twelve-to-three gap stays twelve, and developers who planned around downloadable weights are back on the $2/$10 API — still decent, still not revolutionary. A promise is not a release. The coverage should say so, every time, until the weights are on Hugging Face.
The hacking cut
The most consequential paragraph in Moonshot's launch materials is not a benchmark at all. The company says it "identified and removed cyber-related data" from mid-training and from the reinforcement-learning stage specifically for the open-weight release, while keeping general coding tasks. Its stated aim, in the launch blog's words: "not to release open weights that would extend the absolute frontier of cyber capability."
The cost of that scrub is quantified on an internal, unpublished test, xdev-bench-hard: the uncleaned checkpoint passes at 90.5% (pass@3) while the released K3.1 Pro manages 81.0%; K3 scored 33.3%, DeepSeek's V4.1-Flash 85.7%. All self-reported — no outsider can re-run it, which is precisely the watchdog press's complaint.
What the launch blog does not address is the obvious next question: can fine-tuning put the removed capability back? A downloadable 2.8-trillion-parameter model with its general coding ability intact is, by definition, a few training runs away from being something else. The scrub presumably happened because Mindgard researchers jailbroke Kimi K2.6 and K3 in July. Scrubbing the weights before release is responsible; pretending the scrub is irreversible would be marketing.
And then the backdrop Moonshot's post never mentions: weeks ago, OpenAI said people linked to Moonshot tried to extract its models' hidden reasoning; Anthropic said new safeguards cut what copycat labs can get from Claude by up to 99%. Washington accused Moonshot of distilling Anthropic's Fable to build Kimi K3 — and now Moonshot is publishing open weights anyway, daring both claims to be tested. If the weights land on November 4, they become a live experiment in whether Anthropic's safeguards held.
What it means for the US labs
Strip out the vendor charts and what remains is this: a Chinese lab is selling frontier-adjacent agentic coding at OpenAI's price, with downloadable weights eleven days out, in a market where Chinese models already run most developer AI traffic. The open-weight wave out of China is no longer a curiosity; it is the pricing floor. US labs' API margins — the thing that pays for the next training run — are being set in Beijing.
Here's the judgment call: whether Moonshot's November 4 weights slip will decide whether this was the week open-weight AI caught the frontier or just another launch day of vendor-graded charts. If the weights ship on time, with a licence that actually lets developers build, the American labs lose their last argument — that the best models stay closed because they have to. If they slip, or arrive with tripwires, Kimi K3.1 Pro is a very good API product with a great benchmark story, and the frontier stays where it was: in California, behind an API key. Both outcomes are live. Only one is November 4.
Update, Oct. 24: the independent numbers got better — and the hype needs a note
The independent measurement has moved since launch. Artificial Analysis's own launch analysis now puts Kimi K3.1 Pro at 67% on Terminal-Bench 4.0 — its highest score of any model, ahead of Claude Sonnet 5.5 at 64% — and 1,846 Elo on GDPval-AA, third behind only Claude Opus 5.5 (1,866) and Meta's Muse Wonder Full (1,857), a 312-point climb over Kimi K3. Artificial Analysis says the gap between the leading open-weights and proprietary models on its Intelligence Index will narrow from 12 points to 3 once the weights ship.
The independent voices are noticing. Nathan Lambert, who writes the Interconnects newsletter on open models, wrote on X that the scores "look to be clearly the strongest current open weight model, if the weights are released as they claim by November 4th" — and added "it looks like they figured out their RL, where the initial release was a very impressive base model with lighter post-training." Independent AI writer Simon Willison called it "the first time a (soon-to-be) open weight model has scored this highly against the frontier."
The launch-day buzz is already running ahead of verification. A viral X post by investor @goodalexander claiming a top U.S. law firm had "cut off Claude and installed Kimi" passed 1,800 likes and 140,000 views — though the poster said he had only "heard of" it, and the claim is unverified. A separate viral claim that Anthropic CEO Dario Amodei requested an emergency session with lawmakers over distillation allegations was hit with an X Community Note: no credible reports confirm it. The benchmarks are real; the mythology is not.
One more asterisk on the "open" label: Intelligent Living notes Kimi ships under custom licenses that permit commercial use, not a standard OSI-approved open-source license — "open weights" is the more accurate term. The weights' actual licence terms remain unconfirmed in Moonshot's launch materials.
Update, Oct. 26: the launch's promotion is now the story too
Update, Oct. 26: with nine days left until the promised November 4 weights release, the launch's promotion has become a story of its own. X user Elshayib posted screenshots he described as instructions for a Kimi promotional campaign telling creators "Please DO NOT put on the paid partnership label"; he said he declined to participate on those terms (VentureBeat, via aiweekly.co). The document has not been independently authenticated, and no confirmed Moonshot response had surfaced as of October 25 — so treat the screenshots as alleged, not established. VentureBeat's Carl Franzen keeps the two threads apart: the allegations do not by themselves invalidate the independent benchmark results. That distinction is the point. The numbers — Artificial Analysis 55 against Opus 5.5's 58, Terminal-Bench 4.0 at 65.7% against 66.4% — stand or fall on the weights landing on November 4 and surviving independent reproduction, not on how the launch was marketed.
Kimi K3.1 Pro: the quick version
What is Kimi K3.1 Pro?
Moonshot AI's post-trained version of Kimi K3 (same 2.8T-param base, far more post-training compute), released Oct. 23, 2026, with a 1M-token context window.
How close is it to Claude Opus 5.5?
One point behind on the Artificial Analysis Intelligence Index (55 vs 58); within a point on Terminal-Bench 4.0 (65.7% vs 66.4% in Moonshot's runs; 67% in AA's own eval). These are the closest scores an open-weight model has ever posted against the frontier.
When are the weights out?
Moonshot promises full open weights by November 4, 2026. Until then it's API-and-apps only. The licence terms are still unconfirmed.
How much does it cost?
$2 per million input tokens / $10 per million output ($0.20 cache hits); Highspeed variant at $4/$20. Real-world cost per completed task is roughly $2.
What's the catch?
All headline scores are Moonshot's own runs, not independent; Moonshot deliberately removed cyber-related training data (a 9.5-point self-measured cost); and "open-weight" before the weights exist is a promise, not a fact.
Sources: Artificial Analysis launch-day analysis (reproduced copy); Moonshot AI launch blog and LinkedIn launch post (Oct 23, 2026); madrobot; MarkTechPost; implicator.ai ("Kimi K3.1 Pro Hacking Cut Rests on Unpublished Moonshot Test"); metarix.io; oossa.com; officechai.com; genainews.tech; siliconnoon.com; handyai "Model Drop" newsletter (Substack); kingy.ai; Vercel AI Gateway model listing.
Comments
Post a Comment