Moonshot Denies K3 Distillation Claims | Cites Three In-House Architectural Innovations Behind 2.8T Parameters, OpenAI Exec Reportedly Says 'Distillation Can't Do This'
TL;DR
Moonshot's Huang Zhenxin refuted distillation allegations for Kimi K3, citing three in-house architectural upgrades; new user registration paused due to compute crunch.
Responding to market skepticism that Kimi K3 was distilled from GPT, Moonshot enterprise-business head Huang Zhenxin pushed back at a media briefing yesterday: K3's performance jump comes from ground-up architectural innovation, not distillation. He referenced an OpenAI executive's private assessment — 'distillation alone couldn't produce K3's performance.'
Huang named three in-house technologies: second-order optimizer, linear attention mechanism, residual optimization. He said the combined stack made a 2.8T-parameter regime viable for both training and inference, while materially cutting compute cost. When K3's open weights dropped last week, overseas researchers pattern-matched architecture diagrams to GPT-4/GPT-5, with attention layers and residual connections as the main similarity flashpoints. Moonshot is now walking through those origins publicly.
Short-term pressure has moves. Due to a compute crunch, Moonshot has paused new K3 user registration to preserve service for existing users. Huang confirmed the roadmap: keep open-sourcing, keep scaling parameters. That's a counterweight to the de facto ban DC is weighing — not stepping aside, but shipping bigger open weights every quarter.
via Shanghai Securities News
Huang named three in-house technologies: second-order optimizer, linear attention mechanism, residual optimization. He said the combined stack made a 2.8T-parameter regime viable for both training and inference, while materially cutting compute cost. When K3's open weights dropped last week, overseas researchers pattern-matched architecture diagrams to GPT-4/GPT-5, with attention layers and residual connections as the main similarity flashpoints. Moonshot is now walking through those origins publicly.
Short-term pressure has moves. Due to a compute crunch, Moonshot has paused new K3 user registration to preserve service for existing users. Huang confirmed the roadmap: keep open-sourcing, keep scaling parameters. That's a counterweight to the de facto ban DC is weighing — not stepping aside, but shipping bigger open weights every quarter.
via Shanghai Securities News
