Qwen3-235B-A22B-Thinking-2507: Output Length and Alibaba's Comparison Table

What this model is

Qwen3-235B-A22B-Thinking-2507 is the reasoning edition of Alibaba's Qwen3 235B mixture-of-experts model — 235 billion total parameters with 22 billion activated per token, open-weight and self-hostable, with a 256K token context window. It is the counterpart to the Instruct release of the same weights covered elsewhere on this site. At roughly $0.23 per million input tokens and $2.30 output through third-party providers; reasoning tokens bill as output.

Recommended output length

Alibaba recommend an output length of 32,768 tokens for most queries, and for benchmarking on highly complex problems — competition mathematics and programming specifically — they suggest raising the maximum to 81,920 tokens, stating that the space is what lets the model produce comprehensive responses and improves its performance. At $2.30 per million output tokens, an 81,920-token ceiling is roughly nineteen cents per hard question if the model uses the room, and the published competition figures assume it does.

Alibaba’s comparison table

From Alibaba's comparison table: on LiveCodeBench v6 it records 74.1, the highest in its table. On CFEval it scores 2134, again the highest. On MultiIF it takes 80.6 and on PolyMATH 60.1, both table-leading, ahead of a next-best 54.7. OJBench comes in at 32.5 against a leading 38.9, MMLU-ProX at 81.0 against 84.7, and INCLUDE at 81.0 against 86.6.

LMArena

On LMArena's text leaderboard Qwen3 235B A22b Thinking 2507 has 1400 Elo (rank 166); by category, 1442 Elo in Coding (rank 170), 1396 Elo in Math (rank 171), 1468 Elo in Expert (rank 95) and 1370 Elo in Creative Writing (rank 160). The Instruct edition of the same weights has 1422 Elo (rank 131).

Specifications and serving

The card lists 234 billion non-embedding parameters, 94 layers, 64 query heads and 4 key-value heads, and 128 experts with 8 activated, with a native context of 262,144 tokens. The default chat template inserts the opening thinking tag automatically, so output normally contains only the closing tag. Alibaba's SGLang and vLLM examples use tensor parallelism of 8 and a deepseek-r1 reasoning parser, and Alibaba advise keeping the context above 131,072 tokens where possible because reasoning can need long sequences. Like the Instruct release, it can be configured for contexts up to about 1 million tokens, which Alibaba estimate needs about 1,000 GB of GPU memory.

More of Alibaba’s figures

Alibaba also report MMLU-Pro 84.4, GPQA 81.1, SuperGPQA 64.9, AIME25 92.3, HMMT25 83.9, HLE 18.2 on the text-only subset, IFEval 87.8, Arena-Hard v2 79.7 and WritingBench 88.3. The original Qwen3-235B-A22B in thinking mode scored 82.8, 71.1, 60.7, 81.5, 62.5, 11.8, 83.4, 61.5 and 80.3 on the same tests. In the same table OpenAI o3 records 85.9 on MMLU-Pro and 88.9 on AIME25, and Gemini 2.5 Pro 86.4 on GPQA.

Agentic results and tools

On Alibaba's agent tests it records BFCL-v3 71.9, TAU2-Retail 71.9, TAU2-Airline 58.0 and TAU2-Telecom 45.6, against the original thinking release's 70.8, 40.4, 30.0 and 21.9. Alibaba point to Qwen-Agent, which wraps tool-calling templates and parsers and accepts MCP configuration files, and list Ollama, LM Studio, MLX-LM, llama.cpp and KTransformers as supporting Qwen3 locally.

Official Resources