What LLM Can I Run on My GPU or Mac?

To find out what large language model (LLM) fits your hardware, match your memory against the combined footprint of the model weights and key-value (KV) cache. On a 24 GB graphics processing unit (GPU) at an 8,192-token context length, 11 of the 13 supported models fit inside available memory. Phi-4-mini, Mistral 7B, Qwen2.5 7B, DeepSeek-R1-Distill-Llama 8B, and Qwen3 8B run at 16-bit precision, while larger models fit by dropping down to 8-bit or 4-bit weights.

Enter your memory into the Your memory (GB) field, choose your Memory type, and set your context length to evaluate your machine. The calculator updates the table automatically, sorting models from largest to smallest and showing the highest precision level that fits your hardware footprint.

Model weights remain fixed in video random-access memory (VRAM), while the KV cache expands with every additional token.

What LLM Can I Run

Reading the Fit Table

The output table shows which models fit your hardware as soon as you enter your memory and target context length. What LLM Can I Run lists compatible models first, ordered from largest parameter count to smallest, followed by models that exceed your memory.

The Model column lists each architecture alongside a direct link to its repository on Hugging Face. Best precision that fits shows the highest precision tier between 16-bit floating-point (FP16) or bfloat16 (BF16), 8-bit integer (INT8), and 4-bit (Q4) that fits your hardware. Memory needed combines the model weights at that precision with the KV cache at 16-bit precision, calculated using 1,024³ bytes per gigabyte.

Spare displays the leftover gigabytes, and Longest context that fits shows the maximum context length that leftover memory holds at that precision tier. If the leftover capacity exceeds the model architecture's native context window, the table displays (model limit) to show the architectural ceiling. When a model requires more memory than your hardware provides, the table marks it with ✗ Doesn't fit, shows the memory needed at 4-bit quantization, and lists how many gigabytes short your configuration is.

How Hardware Fit Is Worked Out

Total memory demand equals static model weights plus the dynamic KV cache required for active tokens. Model weight footprint equals total parameters multiplied by the precision bytes per weight: 2 bytes for FP16 or BF16, 1 byte for INT8, and 0.5 bytes for 4-bit.

According to the NVIDIA developer blog tutorial on inference optimization, KV cache bytes equal 2 × layers × KV heads × head dim × tokens × bytes per value. The initial factor of 2 accounts for the separate key and value matrices. Because modern transformer designs use grouped-query attention to share key and value projections across attention heads, the calculator uses the KV-head count from each model's configuration.

For Qwen3 8B, which has 36 layers, 8 KV heads, and a head dimension of 128, the 16-bit KV cache per token equals 2 × 36 × 8 × 128 × 2, or 147,456 bytes (144 KB). What LLM Can I Run tests precisions in descending order (FP16, INT8, then 4-bit) and selects the highest precision where weights plus KV cache fit within your memory limit, because weight memory drops with precision while 16-bit KV cache memory remains constant.

• Parameters: total parameter count from each model maker's Hugging Face repository.
• Layers: total transformer hidden layer depth extracted from config.json.
• KV heads: distinct key-value projection heads, lower than query heads in grouped-query attention models.
• Head dimension: size of each attention head, fixed at 128 across all 13 supported models.

Fitting Models to Common Memory Sizes

Hardware capacity determines which open-weights architectures you can deploy and what quantization levels are necessary. On a 12 GB GPU at an 8,192-token context length, 8 of the 13 models fit inside memory. Phi-4-mini Instruct fits at FP16 using 8.2 GB. The 7B to 8B models fit at INT8: Qwen2.5 7B Instruct needs 7.5 GB, Mistral 7B Instruct v0.3 needs 7.8 GB, DeepSeek-R1-Distill-Llama 8B needs 8.5 GB, and Qwen3 8B needs 8.8 GB. The 14B models fit at 4-bit: Phi-4 needs 8.1 GB, Qwen3 14B needs 8.1 GB, and Qwen2.5 14B Instruct needs 8.3 GB. Mistral Small 24B Instruct 2501 needs 12.4 GB at 4-bit, falling 0.4 GB short of fitting.

When you select Mac unified memory, What LLM Can I Run allocates 75% of your total capacity to the model and reserves 25% for macOS and background applications. On a 16 GB Mac, this reserve leaves 12 GB available for model execution, yielding the exact same 8 model fits at 8,192 tokens as a dedicated 12 GB GPU.

On a 24 GB GPU at an 8,192-token context, 11 of the 13 models fit into memory. Phi-4-mini needs 8.2 GB, Mistral 7B needs 14.5 GB, Qwen2.5 7B needs 14.6 GB, DeepSeek-R1-Distill-Llama 8B needs 16.0 GB, and Qwen3 8B needs 16.4 GB, all running at full FP16 precision. Moving up to INT8 enables Phi-4 at 14.6 GB, Qwen3 14B at 15.0 GB, Qwen2.5 14B at 15.2 GB, and Mistral Small 24B at 23.6 GB. At 4-bit quantization, Qwen2.5 32B Instruct takes 17.1 GB with 6.9 GB spare, while Qwen3 32B consumes 17.3 GB. DeepSeek-R1-Distill-Llama 70B requires 35.4 GB at 4-bit, and Qwen2.5 72B Instruct requires 36.4 GB, meaning neither fits on a single 24 GB GPU.

On an 80 GB enterprise card like the NVIDIA H100 Tensor Core GPU with a 32,768-token context, all 13 models fit in memory. Architectures up to Qwen3 32B run at FP16, with Qwen2.5 32B taking 68.5 GB. DeepSeek-R1-Distill-Llama 70B fits at INT8 using 75.7 GB with 4.3 GB spare, while Qwen2.5 72B fits at INT8 using 77.7 GB with 2.3 GB spare, both carrying a tight warning.

Context Length and Tight Fits

Doubling your context length doubles the memory consumed by the KV cache. While static weights remain constant, extended sequence lengths can exhaust leftover capacity and force a model down into a lower precision tier on the same graphics card.

On a 24 GB GPU at an 8,192-token context, Mistral Small 24B fits at INT8 with 23.6 GB of total memory, leaving 0.4 GB spare. Because the leftover capacity sits below 10% of total memory, What LLM Can I Run flags the row with a ⚠ (tight) indicator. That 0.4 GB of spare memory limits the longest context that fits to 10,802 tokens before memory is exhausted.

On that same 24 GB GPU, Qwen2.5 32B at 4-bit uses 17.1 GB of memory at 8,192 tokens, leaving 6.9 GB spare. That headroom allows context to expand up to 36,315 tokens before running out of memory. If your context requirements expand beyond that limit, the workload will fail to run unless you reduce context or deploy additional memory.

When the Fit Table Is Wrong for You

The fit table estimates memory using static model weights and the KV cache, excluding activations, runtime buffers, and framework memory overhead. Real video memory usage during live serving is always higher, which is why tight rows require caution.

Quantized file distributions like GGUF introduce metadata and mix quantization precisions across tensor layers, meaning actual file sizes on disk differ somewhat from raw parameter counts multiplied by precision bytes. Furthermore, runtimes that store the KV cache at lower precision need less memory than shown.

This calculator determines whether an architecture fits within physical memory, not how fast it processes tokens or the output quality of its responses. To evaluate generation latency on specific hardware, consult the LLM Tokens Per Second Calculator. If you are budgeting hardware purchases from scratch, use the Local AI Hardware Calculator. To isolate memory demand across extended contexts or multi-user batches, use the KV Cache Calculator. You can also review machine builds in Best Budget Mini PC for Local AI and top models in Best Local LLMs.