Qwen2.5 72B Instruct: Alibaba's Dense 72B Model Under the Qwen Licence

What Qwen2.5 72B Instruct is

Qwen2.5 72B Instruct is Alibaba's flagship dense model of the Qwen2.5 generation, released under the Qwen licence rather than Apache 2.0; most later Qwen releases use Apache 2.0. It runs at roughly $1.20 per million tokens for both input and output through third-party providers.

Use as a comparison baseline

DeepSeek benchmarked V3 against it, recording Qwen2.5-72B-Instruct at 81.2 on Arena-Hard and 49.1 on AlpacaEval 2.0, against GPT-4o's 80.4 and 51.1.

LMArena

On LMArena's text leaderboard Qwen2.5 72B Instruct has 1303 Elo (rank 288); by category, 1355 Elo in Coding (rank 269), 1296 Elo in Math (rank 259), 1295 Elo in Expert (rank 275) and 1254 Elo in Creative Writing (rank 297).

Architecture

Alibaba's card lists 72.7 billion parameters, 70.0 billion of them non-embedding, across 80 layers. The model is a Transformer with RoPE, SwiGLU, RMSNorm and attention QKV bias, and uses grouped-query attention with 64 heads for queries and 8 for keys and values. The full context length is 131,072 tokens, with generation up to 8,192 tokens. Qwen2.5 was released as a series of base and instruction-tuned models from 0.5 to 72 billion parameters; this repository holds the instruction-tuned 72B model.

What Alibaba changed from Qwen2

Alibaba describe Qwen2.5 as adding more knowledge and improving coding and mathematics through specialised expert models in those domains. They also list better instruction following, generation of long texts over 8K tokens, understanding of structured data such as tables, and structured output, especially JSON, along with more resilience to varied system prompts for role-play and chatbot condition-setting. Multilingual support covers more than 29 languages, including Chinese, English, French, Spanish, Portuguese, German, Italian, Russian, Japanese, Korean, Vietnamese, Thai and Arabic.

Long inputs

The shipped config.json is set for contexts up to 32,768 tokens. For longer inputs Alibaba use YaRN, enabled by adding a rope_scaling entry with a factor of 4.0 and an original maximum of 32,768 positions. Alibaba note that vLLM supports only static YaRN, where the scaling factor stays constant regardless of input length, which can affect performance on shorter texts, so they advise adding the setting only when long contexts are needed. The model requires Transformers 4.37.0 or later; older versions fail with a KeyError on 'qwen2'. The default system prompt in Alibaba's example is "You are Qwen, created by Alibaba Cloud. You are a helpful assistant."

Release and Alibaba’s figures

Alibaba released Qwen2.5 on 19 September 2024, with pre-training on up to 18 trillion tokens. Alongside the open weights, Qwen-Plus and Qwen-Turbo were offered through Alibaba Cloud Model Studio. In Alibaba's instruct table, Qwen2.5-72B-Instruct scores MATH 83.1, GSM8K 95.8, LiveCodeBench 55.5 and Arena-Hard 81.2, against Qwen2-72B-Instruct's 69.0, 93.2, 32.2 and 48.1; the same table gives Llama-3.1-405B 73.8, 96.8, 41.6 and 69.3. For the 72B base model Alibaba report MMLU 86.1, MATH 62.1, GSM8K 91.5, HumanEval 59.1 and MBPP 84.7.

Running it

Alibaba advise the latest Transformers release and document deployment with vLLM, publishing GPU memory and throughput measurements in the Qwen documentation's speed benchmark. Detailed evaluation results are in the Qwen2.5 blog, and the Qwen2 technical report is on arXiv (2407.10671).

Official Resources