Nemotron 3.5 Lightning Explained: NVIDIA's 30B Open Model

In short: Nemotron 3.5 Lightning is an open-weight AI model from NVIDIA, released on 11 August 2026. It has 30 billion parameters in total but uses only 3 billion for each token, which lets it run on a single 80GB data-centre GPU. Its weights are free to download under the OpenMDW 1.1 licence, which NVIDIA states is ready for commercial use. It reads text and has a context window of up to 1 million tokens, with 256K used on a single H100.

What is Nemotron 3.5 Lightning?

Nemotron 3.5 Lightning is a large language model trained by NVIDIA and published as downloadable weights, not sold as a pay-per-token API. NVIDIA announced it on 11 August 2026. The full-precision model is on Hugging Face as NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16 - "30B-A3B" means 30 billion total parameters, 3 billion active.

For what parameters are, see how LLMs work.

What does "30B total, 3B active" mean?

The model is split into many smaller "experts", and only a few of them are used for each token. This design is called mixture of experts (MoE). The full 30 billion parameters give the model its capacity; the 3 billion used per token set how much computation each word of output costs. Memory still has to hold all 30 billion.

Nemotron 3.5 Lightning combines three kinds of layer: Mamba-2 layers, MoE layers and a smaller number of attention layers. Mamba is an alternative to attention that processes long sequences with less memory; attention is the mechanism behind standard transformers - see Attention Is All You Need explained.

What hardware does Nemotron 3.5 Lightning need?

NVIDIA's model card lists single-GPU deployment on one H100 80GB or one A100 80GB. Supported hardware covers NVIDIA Blackwell (GB200, B200), Hopper (H100, H200) and Ampere (A100) GPUs. NVIDIA also links a GGUF checkpoint for local devices such as the RTX 5090, DGX Spark and RTX 6000 Pro.

The context length depends on the hardware. The model card's validated settings:

What are Nemotron 3.5 Lightning's specs?

How do you run Nemotron 3.5 Lightning?

Download the weights and run them with inference software such as vLLM or SGLang. The model card's quick start uses vLLM on a single H100. NVIDIA describes the BF16 release as the reference weights, intended mainly for customisation - further training and making quantised versions - and points to the NVFP4 release for faster inference. The model is also listed on build.nvidia.com. See how to run AI models locally for what these tools do.

What is NeMo Switchyard?

NeMo Switchyard is a routing library NVIDIA open-sourced alongside Nemotron 3.5 Lightning. NVIDIA describes it as directing each step of an agent workflow to the model best suited to that step.

How does Nemotron 3.5 Lightning rank?

On LMArena's text leaderboard Nemotron 3.5 Lightning has 1353 Elo (rank 217). The full ranked table is on our benchmarks page.

Words to know

• Open weights - model files published for anyone to download and run.
• Mixture of experts (MoE) - a design that uses only part of the model for each token.
• Active parameters - the parameters actually used for each token.
• Mamba - a layer type that handles long sequences with less memory than attention.
• BF16 / NVFP4 - number formats; BF16 is full precision, NVFP4 is a compressed 4-bit format.
• Quantisation - storing weights with fewer bits to save memory.

Related guides

• How to run AI models locally
• AI scaling laws explained
• Tokens and context windows explained