Run AI Offline on Laptop Hardware: Memory and Model Setup

To run AI offline on laptop hardware, you download an open-weight model to local storage while connected to the internet, then execute the model using runtime software such as Ollama or LM Studio. System memory determines which model your laptop can load: 8 to 16 gigabytes (GB) of random-access memory (RAM) supports compact models such as Phi-3 mini, a 4-bit quantized 8-billion or 9-billion parameter model requires approximately 6 GB of video random-access memory (VRAM), and 70-billion parameter models require roughly 40 GB of VRAM.

At Layer3Labs, we build custom agents, enterprise integrations, and automated workflows for small and mid-sized business teams across the United States. Once the initial model download completes, runtime engines operate completely disconnected from external networks, so all inference runs on your own machine.

Three Steps to Run AI Offline on Laptop Hardware

Executing an artificial intelligence (AI) model without an internet connection requires three basic setup steps: installing a runtime engine, downloading model weights, and launching the local interface.

• Install Ollama or LM Studio while connected to the internet. LM Studio has a graphical interface; both have libraries of popular models you can pull with no manual weight conversion.
• Pull the model you want. The download needs internet; after that the model files are local.
• Disconnect from the network and prompt the model in the application.

Memory Requirements Across Model Dimensions

System memory capacity is the primary constraint that dictates which open-weight models your laptop can successfully load. Running a large language model (LLM) requires loading its neural network parameters directly into working memory, either in system RAM or dedicated GPU VRAM.

Quantization stores model weights at lower numerical precision to cut overall memory consumption. Community runners rely heavily on GGUF to execute these compressed weights. A 4-bit quantization reduces the memory footprint of an 8-billion parameter model down to roughly 6 GB of VRAM. However, quantization can lower output quality, mainly on larger models and hard tasks.

Microsoft Phi models provide lightweight architectures: Phi-2 (2.7B) and Phi-3 mini (3.8B) run on a laptop CPU with 8 to 16 GB of RAM, no GPU needed. A 4-bit Phi-4 (14B) fits into roughly 8 GB of RAM, as outlined in our Phi local guide. Google Gemma models demand roughly 5 GB for Gemma 2B, 18 GB for Gemma 9B, and 54 GB for Gemma 27B at full precision. A 4-bit Gemma 9B fits in about 6 GB of VRAM, detailed in our Gemma local guide.

Open-weight architectures like Llama scale from consumer laptops to server tiers. Llama 8B at 4-bit quantization fits in roughly 6 GB of VRAM, while Llama 70B at 4-bit requires approximately 40 GB. Massive builds like Llama 405B or Llama 4 mixture-of-experts (MoE) models need the combined memory of multiple GPUs, per our Llama local guide. Download sizes also vary across families: Qwen3 on Ollama needs 523 MB for 0.6b, 1.4 GB for 1.7b, 2.5 GB for 4b, 5.2 GB for 8b, 9.3 GB for 14b, 19 GB for 30b, 20 GB for 32b, and 142 GB for 235b. These are download sizes, not running memory requirements, as noted in our Qwen local guide.

• Compact models (1B to 4B): Phi-2 and Phi-3 mini run on a laptop CPU with 8 to 16 GB of RAM; Gemma 1B to 4B runs acceptably on CPU only.
• Mid-Sized Models (8B to 14B): 4-bit versions of Llama 8B, Gemma 9B, and Phi-4 require roughly 6 to 8 GB of VRAM.
• Large Models (70B): Quantized 70B models require roughly 40 GB of VRAM or unified memory.
• Server-Scale Models (405B and MoE): Massive builds require the combined memory of multiple GPUs.

Commands to Try After the Download

Installing Ollama lets you pull and run a model with one command. For a desktop interface, LM Studio provides a graphical alternative that pulls models from its own library. Running ollama pull phi4 downloads the Phi-4 flagship model (14B) with no manual setup, following our guide to running Phi locally.

These figures reflect download sizes from the Ollama library page instead of working memory requirements. Because that library page does not state random-access memory (RAM) or video random-access memory (VRAM) needs, compare each download size with your free memory and with the memory table on this page. Check the Ollama library for newer releases that carry their own entry, and test a small harness of your own real prompts on each size.

• ollama run qwen3: Runs the default tag, which is the same as 8b, with a 5.2GB download.
• ollama run qwen3:1.7b: Downloads 1.4GB.
• ollama run qwen3:4b: Downloads 2.5GB.
• ollama run qwen3:8b: Downloads 5.2GB.
• ollama run qwen3:14b: Downloads 9.3GB.
• ollama run qwen3:30b: Downloads 19GB.
• ollama run qwen3:32b: Downloads 20GB.

Choosing a Model by Memory Tier

Pick the model by how much memory your laptop has. In our view, start with the smallest model that could accomplish the task, and keep a small collection of real-world test prompts to evaluate performance across tiers.

If your laptop possesses between 8 GB and 16 GB of standard system RAM and lacks a discrete graphics card, focus on compact architectures. Phi-3 mini (3.8B) and Gemma 2B run acceptably on CPU execution alone.

If your machine has a GPU with 6 GB or more of VRAM, a 4-bit 8B or 9B model fits; Phi-4 (14B) at 4-bit needs about 8 GB.

Test your own prompts on each size. Because quantization stores weights at lower precision, output quality can drop, mainly on larger models and hard tasks.

• 8 GB to 16 GB RAM (CPU-Only): Deploy compact models like Phi-2, Phi-3 mini, or Gemma 2B.
• GPU with about 6 GB of VRAM: 4-bit Llama 8B or Gemma 9B.
• GPU with 8 GB or more of VRAM, or CPU with 16 to 32 GB of RAM (slower): Phi-4 (14B).
• Unified-memory laptops (MacBook Pro, Ryzen AI Max+; up to 128 GB): 4-bit 70B models (roughly 40 GB).

Inference Speed Expectations for Central and Graphics Processors

Offline inference processing speed depends on your hardware and model size. CPU-only inference is slower than GPU inference.

No tokens-per-second figures are published for these models, so time your own prompts on your own machine. CPU-only runs acceptably for Gemma 1B to 4B and Phi-3 mini, while 14B models like Phi-4 run slower on CPU.

A dedicated graphics processor accelerates local inference compared to CPU-only execution. To estimate your machine's hardware capabilities before installing large models, consult our local AI hardware calculator.

Functional Boundaries of Local Offline Execution

Running a model offline restricts processing strictly to the data encoded within local weights and eliminates real-time internet integration. The model cannot browse web pages or access external networks.

Closed commercial platforms like OpenAI ChatGPT, Anthropic Claude, Google Gemini, and hosted Grok are cloud services whose underlying model weights cannot be downloaded to a personal computer. Only open-weight models can be stored and run locally. Grok's weights have a separate local-run guide in our walkthrough on how to run Grok locally.

• Zero Web Access: Offline models cannot browse live websites or fetch current external data.
• Cloud Exclusions: Gemini, Claude, and GPT are cloud services and cannot be downloaded locally.
• Scale constraints: Llama 405B and Llama 4 MoE models need the combined memory of multiple GPUs.

Laptop Hardware Specifications for Local Execution

As analyzed in our guide to the best AI laptops, memory differs by hardware type. A Windows laptop equipped with a discrete NVIDIA graphics card tops out at 24 GB of VRAM, which is the limit of the mobile GeForce RTX 5090.

In contrast, an Apple MacBook Pro or an AMD Ryzen AI Max+ mobile workstation uses a unified memory architecture that pools system RAM and graphics memory together. These unified architectures scale up to 128 GB of memory. That is more than the roughly 40 GB a 4-bit 70B model needs.

Neural processing units (NPUs) do not govern large language model capacity. An NPU is a compact, low-power coprocessor designed for continuous background tasks, including noise reduction and live audio captions.

We do not sell hardware.