Mistral Small 3.1 24B: Apache 2.0 Multimodal Model, 32K Context

What Mistral Small 3.1 24B Instruct is

Mistral Small 3.1 24B Instruct (release 2503) is Mistral AI's small multimodal model: 24 billion parameters, Apache 2.0 licensed and self-hostable, with a 128K token context window and pricing around $0.10 per million input tokens and $0.30 output through third-party providers. Mistral publish it alongside a base checkpoint, and both are compared in the card against named competitors.

Mistral’s multimodal comparison

Mistral's multimodal comparison puts Small 3.1 24B Instruct against Gemma 3 27B IT, GPT-4o Mini, Claude 3.5 Haiku and Cohere Aya-Vision 32B. On MMMU PRO it records 49.25% against Gemma 3 27B's 48.38% and GPT-4o Mini's 37.60%. On MathVista, 68.91% against 67.60% and 56.70%. On DocVQA it reaches 94.08% where the next best is Claude 3.5 Haiku at 90.00% and Gemma 3 27B at 86.60%. On AI2D, 93.72% against 92.10% and 84.50%. And on MM MT Bench it scores 7.3 against Gemma's 7 and Claude 3.5 Haiku's 6.5. The exceptions are narrow: Gemma 3 27B edges plain MMMU at 64.90% against 64.00%, and Claude 3.5 Haiku takes ChartQA at 87.20% against 86.24%. These are Mistral's figures.

Base model comparison

Mistral also compare their base checkpoint against Gemma 3 27B PT. On GPQA Main with 5-shot chain-of-thought, Small 3.1 24B Base records 37.50% against Gemma 3 27B PT's 24.30%. It also leads MMLU at 81.01% against 78.60%, MMLU Pro at 56.03% against 52.20% and MMMU at 59.27% against 56.10%, losing only TriviaQA at 80.50% against 81.30%. On multilingual performance it averages 71.18% against Gemma's 70.19%, leading on European and East Asian languages while trailing slightly on Middle Eastern. All vendor-reported.

LMArena

On LMArena's text leaderboard Mistral Small 3.1 24B Instruct 2503 has 1303 Elo (rank 289); by category, 1361 Elo in Coding (rank 260), 1277 Elo in Math (rank 280), 1302 Elo in Expert (rank 263) and 1270 Elo in Creative Writing (rank 280).

Features and instruction-tuned figures

Mistral describe Small 3.1 as building on Mistral Small 3 (2501) by adding vision and extending long-context support to 128K tokens; it is the instruction-tuned version of Mistral-Small-3.1-24B-Base-2503. It supports native function calling and JSON output, uses the Tekken tokenizer with a 131K vocabulary, and lists support for dozens of languages, including English, French, German, Hindi, Japanese, Korean, Arabic, Chinese and Farsi. Mistral state that once quantized it fits on a single RTX 4090 or a 32GB RAM MacBook. For the instruct model Mistral report MMLU 80.62%, MMLU Pro 66.76%, MATH 69.30%, GPQA Diamond 45.96% and HumanEval 88.41%, and on long context LongBench v2 37.18%, RULER 32K 93.96% and RULER 128K 81.20%.

Running it

Mistral recommend vLLM 0.8.1 or later, which installs mistral_common 1.5.4 or later, and treat mistral-common as the source of truth for the V7-Tekken instruct template. Running the model in bf16 or fp16 on GPU requires about 55GB of GPU memory, and Mistral's serving example uses tensor parallelism across two GPUs with up to 10 images per prompt. They recommend a low temperature such as 0.15 and publish a default system prompt that gives the model's knowledge cutoff as 1 October 2023 and states it cannot browse the web, generate images, or read audio or video.

Official Resources