QwQ-32B: Qwen's 32B Reasoning Model and Its YaRN Requirement

What QwQ-32B is

QwQ is the reasoning line of the Qwen series, and QwQ-32B is its medium-sized member — a model built to think before answering rather than to respond immediately. Alibaba position it as competitive with state-of-the-art reasoning models and name DeepSeek-R1 and o1-mini as the comparison. It went through both supervised fine-tuning and reinforcement learning in post-training, and it is open-weight and self-hostable.

Context length and YaRN

Alibaba state the context length as a full 131,072 tokens, with a condition: for prompts exceeding 8,192 tokens, YaRN must be enabled. That is a configuration step, not a default. YaRN is enabled by adding a rope_scaling entry with a factor of 4.0 and an original maximum of 32,768 positions; Alibaba note that vLLM supports only static YaRN, which can affect performance on shorter texts, so it should be added only when needed.

The architecture, in full

QwQ-32B is a causal language model with 32.5 billion parameters, or 31.0 billion excluding embeddings, across 64 layers. Attention is grouped-query with 40 query heads to 8 key-value heads. The architecture uses RoPE positional encoding, SwiGLU activation, RMSNorm and attention QKV bias.

One prompting detail Alibaba document

For mathematics, Alibaba instruct users to include the phrase "Please reason step by step, and put your final answer within \boxed{}" in the prompt. Published mathematics figures are produced with that phrasing.

LMArena

On LMArena's text leaderboard QwQ 32B has 1336 Elo (rank 244); by category, 1383 Elo in Coding (rank 236), 1363 Elo in Math (rank 206), 1362 Elo in Expert (rank 217) and 1293 Elo in Creative Writing (rank 250).

Alibaba’s usage guidelines

Alibaba recommend temperature 0.6, top_p 0.95 and min_p 0 instead of greedy decoding, which can cause endless repetition, with top_k between 20 and 40; presence_penalty between 0 and 2 can reduce repetition, though higher values may cause occasional language mixing and slightly lower performance. The model should begin its output with a thinking tag so it does not produce empty thinking content, which the chat template handles automatically. In multi-turn conversations, earlier turns should include only the final output, not the thinking content. For multiple-choice benchmarks Alibaba suggest asking for the answer letter in a JSON answer field. QwQ-32B is based on Qwen2.5 and needs Transformers 4.37.0 or later; Alibaba's example generates up to 32,768 new tokens. It can be tried in Qwen Chat and a Hugging Face demo space.

Release and training

Alibaba released QwQ-32B on 6 March 2025 under the Apache 2.0 licence. Reinforcement learning ran in two stages. The first used outcome-based rewards for mathematics and coding, checked by an accuracy verifier for maths problems and a code-execution server for code. The second used rewards from a general reward model and rule-based verifiers to improve general capabilities. Alibaba's launch post compares QwQ-32B with DeepSeek-R1-Distilled-Qwen-32B, DeepSeek-R1-Distilled-Llama-70B, o1-mini and the original DeepSeek-R1.

Within the Qwen line

QwQ-32B was later followed by the Qwen3 generation, whose Qwen3 32B and other models combine thinking and non-thinking modes in one model; Alibaba state that Qwen3's thinking mode surpasses QwQ on mathematics, code generation and commonsense reasoning.

Official Resources