Xiaomi MiMo V2.5: A Million-Token Context Trained Up in Stages

What MiMo V2.5 is

MiMo V2.5 is Xiaomi's open-weight omnimodal model, released under the MIT licence and self-hostable: it understands text, image, video and audio in one architecture, built on the MiMo-V2-Flash backbone with added vision and audio encoders. Xiaomi publish it in two forms: MiMo-V2.5-Base carries a 256K token context window, while the post-trained MiMo-V2.5 carries 1 million. Both are available on Hugging Face and ModelScope. The roster records 1M tokens, which is the same figure written out.

How the million-token window was built

Xiaomi document the training stages: during supervised fine-tuning and agentic post-training, the context window is progressively extended from 32K to 256K to 1M. The base model stops at 256K; the 1M window comes from post-training.

Architecture

The language model is a sparse mixture-of-experts with 310 billion total parameters and 15 billion active per token. It has 48 layers, one dense and 47 mixture-of-experts, of which 9 use global attention and 39 sliding-window attention with a 128-token window, the 5:1 hybrid design inherited from MiMo-V2-Flash; Xiaomi state this cuts KV-cache storage by nearly six times, with a learnable attention sink bias to keep long-context performance. There are 256 routed experts with 8 per token, 64 attention heads and a hidden size of 4,096. The vision encoder is a 729-million-parameter ViT with 28 layers, 24 sliding-window and 4 full attention; the audio encoder has 261 million parameters across 24 layers, initialised from MiMo-Audio. Three multi-token prediction layers, 329 million parameters in total, accelerate inference through speculative decoding.

LMArena

On LMArena's text leaderboard MiMo V2.5 has 1434 Elo (rank 111); by category, 1491 Elo in Coding (rank 96), 1438 Elo in Math (rank 101), 1471 Elo in Expert (rank 91) and 1394 Elo in Creative Writing (rank 117).

Price

Roughly $0.14 per million input tokens through third-party providers, an aggregated figure rather than first-party.

Training and deployment

Xiaomi report training on about 48 trillion tokens in FP8 mixed precision, in five stages: text pre-training, a warm-up of the multimodal projectors, multimodal pre-training, supervised and agentic post-training with the context extended from 32K to 256K to 1M, and reinforcement learning with Multi-Teacher On-Policy Distillation. A larger MiMo-V2.5-Pro has 1.02 trillion parameters with 42 billion active, 70 layers of which 10 use global attention, 384 routed experts with 8 per token, 128 attention heads and a hidden size of 6,144. Both share a head dimension of 192 for queries and keys and 128 for values, a 128-token sliding window, three MTP layers and a 1M maximum context. Xiaomi publish evaluation charts for multimodal, coding and agentic benchmarks and for GraphWalks long-context retrieval. Xiaomi updated the config.json and tokenizer_config.json files after release and ask users who downloaded earlier to re-pull them. SGLang is the officially supported deployment path, and Xiaomi recommend a temperature of 1.0 and top_p of 0.95. The model is also offered through the Xiaomi MiMo API Platform and MiMo Studio, with a blog post on mimo.xiaomi.com and a model collection on Hugging Face.

Official Resources