Qwen3-235B-A22B-Instruct-2507: 1M-Token Context and RULER Results

What this model is

Qwen3-235B-A22B-Instruct-2507 is Alibaba's updated instruct release of the Qwen3 235B mixture-of-experts model — 235 billion total parameters with 22 billion active per token, open-weight and self-hostable. The 2507 suffix marks the July 2025 refresh, and the substantive change is long context: this release is configured for inputs up to roughly 1,010,000 tokens, where the original Qwen3-235B-A22B was not.

Long-context results

Alibaba test on a 1M-token version of the RULER benchmark and publish accuracy at every context length from 4K to 1,000K, for three configurations. The original Qwen3-235B-A22B in non-thinking mode averages 83.9%, starting at 97.7% on 4K inputs and falling to 68.0% at 1,000K. This 2507 release with full attention averages 92.5%, holding 98.5% at 4K and still managing 84.5% at 1,000K. With sparse attention it averages 91.7%, effectively matching full attention across the range — 82.5% at 1,000K against 84.5%.

Running it at a million tokens

Alibaba document the serving configuration. The flags they publish enable Dual Chunk Flash Attention as the attention backend, set the maximum input length to 1,010,000, allocate 75% of memory to static allocation for weights and the KV cache pool, use tensor parallelism across 8 devices, and chunk prefill at 131,072 tokens to avoid running out of memory on long inputs. Alibaba note that memory fraction should be reduced if out-of-memory errors appear.

LMArena

On LMArena's text leaderboard Qwen3 235B A22b Instruct 2507 has 1422 Elo (rank 131); by category, 1472 Elo in Coding (rank 125), 1417 Elo in Math (rank 136), 1447 Elo in Expert (rank 125) and 1378 Elo in Creative Writing (rank 148).

Price

Roughly $0.26 per million input tokens through third-party providers, an aggregated figure rather than first-party.

Specifications

The card lists 235 billion total and 22 billion activated parameters (234 billion non-embedding), 94 layers, grouped-query attention with 64 query heads and 4 key-value heads, and 128 experts with 8 activated. The native context length is 262,144 tokens, extendable to 1,010,000. The model supports only non-thinking mode and does not emit thinking blocks, so enable_thinking=False is no longer required. It needs Transformers 4.51.0 or later, and Alibaba's standard deployment examples use SGLang 0.4.6.post1 or vLLM 0.8.5 with tensor parallelism of 8 and a 262,144 context.

Alibaba’s comparison table

Alibaba compare the release with DeepSeek-V3-0324, GPT-4o-0327, Claude Opus 4 without thinking, Kimi K2 and the original Qwen3-235B-A22B in non-thinking mode. For this model they report MMLU-Pro 83.0, GPQA 77.5, SimpleQA 54.3, AIME25 70.3, HMMT25 55.4, ARC-AGI 41.8, ZebraLogic 95.0, LiveCodeBench v6 51.8, Aider-Polyglot 57.3, IFEval 88.7, Arena-Hard v2 79.2 and BFCL-v3 70.9. The original non-thinking release scored 75.2, 62.9, 12.2, 24.7, 10.0, 4.3, 37.7, 32.9, 59.6, 83.2, 52.0 and 68.0 on the same tests. Arena-Hard v2 figures are win rates judged by GPT-4.1.

Memory for a million tokens

Alibaba state that processing a 1 million token context requires approximately 1,000 GB of total GPU memory, covering weights, KV cache and peak activations. The 1M configuration combines Dual Chunk Attention, a length-extrapolation method, with MInference sparse attention, and Alibaba report up to a 3x speedup over standard attention on sequences approaching 1M tokens.

Official Resources