Moonshot AI (月之暗面)

Contribution to AI

Context length was a line in an API specification until a Beijing startup made it the whole proposition. The assistant released in October 2023 took in roughly 200,000 Chinese characters without truncation or retrieval workarounds, more than any generally available service then offered, and the upgrade to two million characters in March 2024 drew enough traffic to break the service for days — direct evidence that reading long documents was a consumer demand and not merely a benchmark. The lineage is unusually legible: Yang Zhilin's earlier papers, Transformer-XL and XLNet, had attacked the same problem of models losing their grip on distant text, and the company was that research thread turned into a product. What followed was more consequential for everyone else. February 2025 brought MoBA, a block-sparse attention scheme for long sequences, and Moonlight, trained with a reworked version of the Muon optimiser released as open code — taking a method that had only been shown on small networks and making it dependable at production scale. Kimi K2, in July 2025, carried that to a trillion-parameter mixture-of-experts trained on 15.5 trillion tokens without the loss spikes such runs usually suffer, with the weights published rather than sold. The result was an open-weight model competitive with closed American systems on agentic and coding tasks, which removed the assumption that frontier-scale capability and downloadable weights were mutually exclusive.

Drafted with AI and edited by hand (claude-opus-5, reviewed 2026-08).