Nemotron 3 Ultra
What NVIDIA's 550B open reasoning model costs to run, by hosted endpoint and by GPU minimum.
Nemotron 3 Ultra is free to download under the OpenMDW-1.1 license for commercial and non-commercial work, but running its 550 billion parameters requires enterprise-grade hardware or hosted inference endpoints. NVIDIA released the model on June 4, 2026, activating 55 billion parameters per token through a Latent Mixture-of-Experts (MoE) design.
At Layer3Labs, we build custom agents for business workflows, and for those the per-token rate below usually matters more than the size of the weights.
Nemotron 3 Ultra Cost and License
Nemotron 3 Ultra model weights are free to download under the permissive OpenMDW-1.1 license, while operational expenses depend on self-hosted hardware or per-token application programming interface (API) endpoints. The Hugging Face model card calls it "ready for commercial and non-commercial use." Self-hosting the 550B-total, 55B-active model in NVFP4 takes at least 4x B200 or 8x H100 GPUs.
NVIDIA's launch blog showed a free OpenRouter listing. The OpenRouter endpoints API, read on October 18, 2026, lists no free endpoint.
Third-party inference providers host Nemotron 3 Ultra with variable rates based on speed tiers and precision levels. DeepInfra lists its standard 4-bit endpoint at $0.50 per 1 million input tokens and $2.20 per 1 million output tokens, with cached inputs priced at $0.10 per 1 million tokens. DeepInfra also offers a priority tier at $0.75 input and $3.30 output, alongside a flexible tier at $0.40 input and $1.76 output.
Baseten hosts the model at $0.60 per 1 million input tokens and $2.40 per 1 million output tokens. Venice serves an 8-bit floating point (FP8) endpoint through OpenRouter at $0.625 input and $3.125 output per 1 million tokens. Artificial Analysis lists $0.60 input and $2.70 output, with a blended price of $0.50 per 1 million tokens. Together AI lists no Nemotron 3 Ultra endpoints on its published pricing directory as of October 2026.
Size and Hybrid Architecture of Nemotron 3 Ultra
Nemotron 3 Ultra contains 550 billion total parameters and activates 55 billion parameters per token using a Latent Mixture-of-Experts architecture. NVIDIA first announced Ultra with the Nemotron 3 family on December 15, 2025 and released the completed model on June 4, 2026.
The model uses a hybrid structure that combines Mamba-2 state-space layers, MoE routing blocks, and dedicated attention layers. It incorporates Multi-Token Prediction (MTP) with a shared-weight design across prediction heads to enable native speculative decoding, and NVIDIA's serving examples use five speculative tokens.
Context handling scales up to 1 million tokens natively, as detailed in the official technical report. Standard inference engines such as vLLM and SGLang default to 256k, and need an environment variable to serve the full 1 million tokens. Hosted API providers enforce lower limits: DeepInfra exposes 262,144 tokens, Baseten caps requests at 202,800 tokens, and Venice supports 256,000 tokens.
NVIDIA's sources give two different pre-training figures. The Hugging Face model card records approximately 20 trillion pre-training tokens in 4-bit NVIDIA Floating Point (NVFP4) using Megatron-LM, while an NVIDIA developer blog post describes a 10-trillion-token base plus 212 billion new tokens. These added datasets supply 4 billion synthetic legal tokens, 35 billion synthesized Wiki-based tokens, and 173 billion refreshed GitHub tokens through September 2025.
Post-training has a data cutoff of May 2026, and this launch adds 10 million supervised fine-tuning (SFT) samples, 1 million reinforcement learning (RL) tasks, and 15 RL environments (cumulative totals: 50 million, 2 million, and 55). Post-training systems apply asynchronous GRPO and Multi-teacher On-Policy Distillation (MOPD) with more than 10 teacher models. Reasoning behavior is active by default and supports configuration through enable_thinking, medium_effort, and reasoning_budget parameters.
Hardware Requirements for BF16 and NVFP4
Self-hosting Nemotron 3 Ultra requires high-density graphics processing unit infrastructure with distinct hardware baselines depending on whether you deploy the 16-bit Brain Floating Point or 4-bit NVIDIA Floating Point checkpoint.
Deploying the BF16 checkpoint (nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16) demands a minimum cluster of 8x B200, 8x GB200, 8x B300, 8x GB300, 8x H200, or 16x H100 GPUs. NVIDIA recommends a single node containing 8x B200 GPUs, which pools roughly 1.5 terabytes (TB) of aggregate High Bandwidth Memory (HBM) to serve parameters and working memory caches.
Deploying the NVFP4 checkpoint cuts physical hardware demands in half. The minimum operational cluster requires 4x B200, 4x GB200, 4x B300, 4x GB300, or 8x H100 GPUs, allowing a single 4x B200 server to house complete model weights and key-value (KV) caches with working headroom.
Hardware compatibility introduces specific operational constraints between GPU generations. TensorRT-LLM support for the BF16 checkpoint is Blackwell-only at release, while Hopper architecture support is planned. NVIDIA says the NVFP4 checkpoint runs on Hopper, Blackwell and Ampere GPUs. Its speed gains are Blackwell-native: NVIDIA claims up to 5x higher throughput per GPU for NVFP4 versus BF16 on Blackwell at the same interactivity.
Installation and Local Serving Stacks
Deploying Nemotron 3 Ultra locally requires dedicated inference engines such as vLLM, SGLang, or TensorRT-LLM configured for tensor and expert parallelism.
NVIDIA distributes model assets across four primary Hugging Face repositories: nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 for post-trained 16-bit weights, nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4 for post-trained 4-bit weights, nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-Base-BF16 for foundation weights, and nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-GenRM for generative reward modeling used for RLHF. Pre-training code datasets and NeMo recipes are accessible via public repositories.
To serve the BF16 model with vLLM using container vllm/vllm-openai:v0.22.0, launch the server with --tensor-parallel-size 8 --enable-expert-parallel --reasoning-parser nemotron_v3 --tool-call-parser qwen3_coder. When deploying with SGLang in container lmsysorg/sglang:v0.5.12.post1, launch with --tp-size 8 --ep-size 8 and, with thinking on, set force_nonempty_content: True for coding-agent tool calls. NVIDIA also ships Ultra as an NVIDIA Inference Microservice (NIM), and docker model run hf.co/nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 pulls the BF16 checkpoint through Docker Model Runner.
The model card mentions llama.cpp, Ollama, and LM Studio only as targets for quantized builds and gives no commands for any of them. For lower-overhead setups, review our walkthrough on how to run Nemotron locally.
Benchmark Performance Across Standard Evaluations
Official benchmark evaluations for Nemotron 3 Ultra demonstrate competitive agent reasoning and long-context comprehension, though competing open-weight models lead across several software engineering and coding tests. Model performance varies depending on evaluation harnesses and precision configurations.
NVIDIA's BF16 model card evaluation table shows rival open-weight models ahead of Nemotron 3 Ultra on several coding and reasoning rows. On SWE-Bench Verified, Nemotron 3 Ultra registers 70.7, trailing GLM-5.1 at 76.2, Kimi-K2.6 at 75.7, and DeepSeek-V4-Pro at 74.5. On Terminal Bench 2.1, Nemotron 3 Ultra scores 56.4, finishing ahead of Qwen-3.5 (49.9) and DeepSeek-V4-Pro (49.2), but behind Kimi-K2.6 (67.2) and GLM-5.1 (59.3). On GPQA without tools, the model scores 87.0 compared to 91.0 for Kimi-K2.6.
Long-context retrieval and factuality are where the model card shows Nemotron 3 Ultra ahead. On the 1-million-token RULER benchmark, Nemotron 3 Ultra posts a 94.7 score, outperforming Qwen-3.5 at 90.1 and DeepSeek-V4-Pro at 94.2. In the OmniScience Non-Hallucination assessment, Nemotron 3 Ultra achieves 78.7, which represents the highest score in the official comparison table where DeepSeek-V4-Pro scored 5.7.
Independent tracking from Artificial Analysis records an Intelligence Index score of 23 for Nemotron 3 Ultra 550B A55B (Reasoning), ranking it 39th out of 120 evaluated models as of October 18, 2026. The independent evaluation measures an output generation speed of 143.5 tokens per second, a time to first token (TTFT) of 1.21 seconds, and a $0.58 cost to run the Intelligence Index, on which the model produced 110 million output tokens.
Published scores show minor variances between documentation sources. NVIDIA's developer blog reports SWE-bench Verified performance between 65.0 percent and 70.4 percent across different harness environments, compared to 70.7 on the model card. The blog notes Terminal-Bench 2.0 at 54 percent, compared to 56.4 on the model card for version 2.1. The NVFP4 model card reports 69.5 on SWE-Bench Verified, 53.9 on Terminal Bench 2.1, and 94.0 on RULER at 1 million tokens.
Coding Capabilities and Agent Workflows
Nemotron 3 Ultra targets complex agentic workflows, multi-turn tool use, and sub-agent delegation. NVIDIA's BF16 model card reports 70.7 on SWE-Bench Verified and 56.4 on Terminal Bench 2.1 for the BF16 model.
NVIDIA reports that the model leads open-weight options in agentic register-transfer level coding for semiconductor design. The model integrates with orchestration systems such as Hermes Agent and OpenClaw.
For workflows with lower hardware capacity, Nemotron 3 Super provides a smaller footprint at 120B total and 12B active parameters.
Position of Nemotron 3 Ultra in the Model Lineup
Selecting between Nemotron 3 Ultra and its siblings involves balancing parameter footprint against the specific modality and reasoning demands of your workflow.
Nemotron 3 Ultra is the largest model in the family, with 550 billion total parameters and 55 billion active parameters per token. Teams operating smaller hardware clusters can deploy Nemotron 3 Super, which packs 120 billion total parameters with 12 billion active parameters per token. For visual workflows, Nemotron 3.5 Super VL adds vision capabilities introduced in October 2026. General design principles across the broader family appear in our architectural review of Nemotron 3.
If you cannot get at least 4x B200 or 8x H100 GPUs for the NVFP4 checkpoint, use a hosted Nemotron 3 Ultra endpoint or self-host Nemotron 3 Super instead.
To determine whether your agent infrastructure justifies the hardware demands of frontier reasoning, benchmark your specific tool-use workflows against a hosted Nemotron 3 Ultra endpoint before investing in dedicated cluster hardware.
What you need to run Nemotron 3 Ultra yourself
Nemotron 3 Ultra is a frontier-scale Mixture-of-Experts model, so "running it yourself" is a real infrastructure decision — not something a single laptop or gaming GPU can do. Match the path below to how seriously you need to self-host. For most teams the API or rented GPUs are the right answer; buying hardware only pays off at steady, high volume or when your data can never leave your walls.
| Path | What it is | Best for | Get started |
|---|---|---|---|
| Call the hosted API | Use Nemotron 3 Ultra as a pay-per-token API — zero hardware | Most teams; evaluating before committing | OpenRouter |
| Rent GPUs by the hour | Spin up H100 / A100 nodes on demand, tear them down after | Self-hosting without capital outlay; bursty workloads | RunPod |
| Local on unified memory | A single workstation with enough unified memory to hold a 4-bit quant | One powerful on-prem box; privacy-first solo/SMB use | Apple Mac Studio (M3 Ultra, 512GB) |
| Local on workstation GPUs | Multiple 48GB professional cards for MoE offload / tensor parallelism | Power users and small clusters that want cards they own | NVIDIA RTX 6000 Ada (48GB) |
Once Nemotron 3 Ultra is running, the fastest way to put it to work day to day is inside Cursor — point it at the model through OpenRouter as a custom model. And if you would rather run a model on one affordable box, see Best mini PCs for local AI and Local AI hardware calculator.
Frequently Asked Questions
- Yes, NVIDIA distributes the model weights freely under the OpenMDW-1.1 license for both commercial and non-commercial use. However, running the model requires paying for hosted API provider tokens or operating high-end GPU hardware clusters.
- Download the model checkpoints from Hugging Face via repositories such as nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 or nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4. Deploy the files using inference containers like vLLM, SGLang, or TensorRT-LLM configured for 8-way tensor parallelism on 8x B200 for BF16. For NVFP4, NVIDIA's card gives single-node 4x B200 examples for vLLM, SGLang and TensorRT-LLM.
- Within the Nemotron line, Ultra is the larger coding option: NVIDIA's BF16 model card lists 70.7 on SWE-Bench Verified and 56.4 on Terminal Bench 2.1, though GLM-5.1 (76.2) and Kimi-K2.6 (75.7) score higher on SWE-Bench Verified. For smaller server footprints, Nemotron 3 Super (120B total / 12B active) offers an alternative option.
- Nemotron 3 Ultra holds 550 billion total parameters and activates 55 billion parameters per token. It uses a Latent Mixture-of-Experts architecture combined with Mamba-2 state-space layers and attention mechanisms.
- On NVIDIA's BF16 model card, Nemotron 3 Ultra scores 94.7 on RULER at 1 million tokens and 78.7 on OmniScience Non-Hallucination. In software engineering benchmarks like SWE-Bench Verified, however, rival open models including GLM-5.1 (76.2) and Kimi-K2.6 (75.7) record higher accuracy.
- Model weights carry no licensing charge under OpenMDW-1.1. Hosted API endpoints cost between $0.40 and $0.75 per 1 million input tokens and $1.76 to $3.30 per 1 million output tokens as of October 2026. Self-hosting requires a minimum cluster of 4x B200 or 8x H100 GPUs for the NVFP4 checkpoint (BF16 needs 8x B200 or 16x H100).
Planning an Open-Model Rollout?
Layer3Labs helps teams pick, host and wire open models like Nemotron into real workflows, with the guardrails and cost checks done up front.
Book a Consultation