Qwen3-Next-80B-A3B-Thinking: Training Cost and Long-Context Throughput Claims

What this model is

Qwen3-Next-80B-A3B-Thinking is the reasoning edition of Alibaba's efficiency-focused Qwen3-Next architecture: 80 billion total parameters with just 3 billion active per token, Apache 2.0 and self-hostable, with a 256K token context window. It is the counterpart to the Instruct release built on the same architecture. It is roughly $0.15 per million input tokens against $1.20 for output.

Training-cost claim

Alibaba report that Qwen3-Next-80B-A3B-Base outperforms Qwen3-32B-Base on downstream tasks with 10% of the total training cost, and delivers ten times the inference throughput for contexts over 32K tokens. These are Alibaba's own measurements against their own earlier model.

LMArena

On LMArena's text leaderboard Qwen3 Next 80B A3b Thinking has 1369 Elo (rank 201, thinking effort); by category, 1421 Elo in Coding (rank 197, thinking effort), 1392 Elo in Math (rank 178, thinking effort), 1393 Elo in Expert (rank 195, thinking effort) and 1326 Elo in Creative Writing (rank 210, thinking effort). The Instruct edition has 1399 Elo (rank 167).

Architecture

Eighty billion total parameters with three billion active means roughly one parameter in twenty-seven participates in any given token. The full eighty billion parameters must still be resident in memory to serve the model.

Architecture and training

The model uses the Qwen3-Next hybrid design: 48 layers with a hidden dimension of 2,048, arranged as 12 repeats of three Gated DeltaNet layers followed by one gated-attention layer, each feeding a mixture-of-experts block of 512 experts with 10 routed and 1 shared active. It was pre-trained on 15 trillion tokens and supports a native 262,144-token context, extensible to 1,010,000. Alibaba state that they used GSPO to address the stability and efficiency problems that the hybrid attention and high-sparsity MoE design pose in reinforcement-learning training. The model supports only thinking mode; the chat template inserts the opening thinking tag automatically, and Alibaba note it may produce longer thinking content than its predecessor.

Alibaba’s comparison table

Alibaba report MMLU-Pro 82.7, GPQA 77.2, AIME25 87.8, HMMT25 73.9, LiveCodeBench v6 68.7, CFEval 2071, Arena-Hard v2 62.3 and TAU2-Airline 60.5. In the same table Qwen3-30B-A3B-Thinking-2507 records 80.9, 73.4, 85.0, 71.4, 66.0, 2044, 56.0 and 58.0, and Gemini 2.5 Flash Thinking 81.9, 82.8, 72.0, 64.2, 61.2, 1995, 56.7 and 52.0. Serving requires SGLang 0.5.2 or vLLM 0.10.2 or later with a deepseek-r1 reasoning parser, and Alibaba recommend a context above 131,072 tokens because reasoning can need long sequences.

More of Alibaba’s figures

Alibaba also report SuperGPQA 60.8, OJBench 29.7, IFEval 88.9, WritingBench 84.6, BFCL-v3 72.0, TAU1-Retail 69.6, MultiIF 77.8 and PolyMATH 56.3. Qwen3-32B in thinking mode records 54.1, 24.1, 85.0, 79.0, 70.3, 52.8, 73.0 and 47.4 on the same tests, and Qwen3-235B-A22B-Thinking-2507 64.9, 32.5, 87.8, 88.3, 71.9, 67.8, 80.6 and 60.1. Arena-Hard v2 results are win rates judged by GPT-4.1.

Serving notes

Multi-token prediction is not generally available in Hugging Face Transformers, and Alibaba advise SGLang or vLLM for inference, with optional speculative decoding using the model's MTP layers. Alibaba's examples serve the model with tensor parallelism across 4 GPUs at a 256K context and generate up to 32,768 new tokens; flash-linear-attention and causal-conv1d can improve efficiency depending on settings.

Official Resources