Qwen3 32B: A Dense Model With Two Output Budgets

What Qwen3 32B is

Qwen3 32B is the large dense model of Alibaba's Qwen3 generation — Apache 2.0, self-hostable, with a 128K token context window, at roughly $0.08 per million input tokens and $0.28 output through third-party providers. Dense rather than mixture-of-experts means every parameter activates on every token: more compute per token than a sparse model of similar total size, and a far simpler deployment, since the memory requirement and the compute requirement are the same number.

Recommended output length

Alibaba recommend an output length of 32,768 tokens for most queries, and for benchmarking on highly complex problems — mathematics and programming competitions specifically — they suggest raising the maximum to 38,912 tokens, stating that the extra space is what lets the model produce detailed responses and improves its performance. For comparison, they recommend 81,920 tokens for the Qwen3-235B Thinking model.

The mathematics prompt requirement

As across the Qwen line, Alibaba document that mathematics prompts must include the instruction to reason step by step and put the final answer within a box. Published mathematics figures are produced with that phrasing.

LMArena

On LMArena's text leaderboard Qwen3 32B has 1347 Elo (rank 226); by category, 1406 Elo in Coding (rank 210), 1398 Elo in Math (rank 167), 1398 Elo in Expert (rank 186) and 1304 Elo in Creative Writing (rank 236).

Specifications

The card lists 32.8 billion parameters, 31.2 billion of them non-embedding, across 64 layers, with grouped-query attention of 64 query heads and 8 key-value heads. The context is 32,768 tokens natively and 131,072 with YaRN, which Transformers, llama.cpp, vLLM and SGLang support. It needs Transformers 4.51.0 or later, and Alibaba's deployment examples use SGLang 0.4.6.post1 with the qwen3 reasoning parser or vLLM 0.8.5 with the deepseek_r1 parser. Ollama, LM Studio, MLX-LM, llama.cpp and KTransformers support it locally.

Thinking and non-thinking modes

Qwen3 32B switches between a thinking mode for reasoning, mathematics and coding and a non-thinking mode for general dialogue within one model. Thinking is on by default; enable_thinking=False in the chat template is a hard switch that disables it, and with thinking enabled, /think and /no_think in a prompt or system message switch modes turn by turn, following the most recent instruction. Alibaba recommend temperature 0.6, top_p 0.95, top_k 20 and min_p 0 for thinking mode, and warn that greedy decoding can cause degraded output and endless repetition; for non-thinking mode they suggest temperature 0.7, top_p 0.8, top_k 20 and min_p 0. Alibaba state that thinking mode surpasses QwQ, and non-thinking mode the Qwen2.5 instruct models, on mathematics, code generation and commonsense reasoning, and describe support for more than 100 languages and dialects. For tool calling they point to Qwen-Agent, which accepts MCP server configuration files.

Within the Qwen3 line

Qwen3 32B is the largest dense model in the original Qwen3 release; the larger Qwen3-235B-A22B and the smaller Qwen3-30B-A3B are mixture-of-experts models. Alibaba later refreshed the MoE models as separate Instruct-2507 and Thinking-2507 releases, which each support only one mode. Alibaba point to the Qwen3 blog, GitHub repository and documentation for benchmark results and hardware requirements.

Official Resources