OLMo 2 32B Instruct: Ai2's Post-Training Stages and Datasets

What OLMo 2 32B Instruct is

OLMo 2 32B Instruct (March 2025) is Ai2's post-trained variant of their OLMo-2 32B base model, Apache 2.0 licensed with a 4.1K token context window and pricing around $0.05 per million input tokens and $0.20 output through third-party providers. On LMArena's text leaderboard OLMo 2 0325 32B Instruct has 1252 Elo (rank 324); by category, 1279 Elo in Coding (rank 324), 1227 Elo in Math (rank 313), 1223 Elo in Creative Writing (rank 314) and 1229 Elo in Instruction Following (rank 327).

Three post-training stages, each with its dataset published

Ai2 name every step and link the data behind it. First, supervised fine-tuning on an OLMo-specific variant of the Tülu 3 dataset. Then direct preference optimization on a published preference mix built for this model. Then a final stage of RLVR — reinforcement learning with verifiable rewards — on a dataset combining GSM8K, MATH, instruction following and mixed constraints. Ai2 note that Tülu 3 was designed for strong performance across a diversity of tasks beyond chat, specifically naming MATH, GSM8K and IFEval. The datasets are published for download.

RLVR

Reinforcement learning with verifiable rewards trains against problems where correctness can be checked automatically — a maths answer that is right or wrong, an instruction-following constraint that was or was not satisfied — rather than against a learned reward model guessing at human preference. It only applies where correctness is mechanically checkable, which is why the dataset is GSM8K, MATH and instruction constraints.

Successor

Each post-training stage is separately published. Its successor, Olmo 3, is published at 7B and 32B in Instruct and Think variants.

Ai2’s published results and training run

Ai2's table reports, for OLMo-2-32B-0325-Instruct, an average of 68.8, with GSM8K 87.6, IFEval 85.6, MATH 49.7, MMLU 77.3, AlpacaEval 2 LC 42.8 and a safety score of 85.9; the same table lists the SFT and DPO stages at averages of 61.7 and 68.8. The RLVR stage was trained on five 8xH100 nodes with GRPO, and Ai2 took step 320, corresponding to episode 573,440, as the final checkpoint. Intermediate RLVR checkpoints were saved every 20 steps and are published as revisions of the Hugging Face repository, alongside the exact training command. The OLMo models are trained on the Dolma dataset, and Ai2 release all code, checkpoints, logs and training details.

Use notes

The chat template does not include a beginning-of-sequence token before the conversation, unlike Ai2's earlier OLMo 2 and Tülu 3 models. Ai2's demos use the system prompt "You are OLMo 2, a helpful and harmless AI Assistant built by the Allen Institute for AI", though the model was not trained with a specific system prompt. Ai2 note that OLMo 2 has limited safety training and no in-the-loop filtering of responses, that it is intended for research and educational use, and that its fine-tuning data includes outputs from third-party models subject to the Gemma Terms of Use. Training, inference and evaluation code are in the OLMo-core, open-instruct and olmes repositories.

Licence and lineage

OLMo 2 32B Instruct is released under Apache 2.0. It is the post-trained variant of the OLMo-2 32B March 2025 base model: supervised fine-tuning on an OLMo-specific variant of the Tülu 3 dataset, DPO on Ai2's 32B preference mix, then RLVR on a mix of GSM8K, MATH and instruction-following constraint data.

Official Resources