How to Use Ollama: A Step-by-Step Guide

Using Ollama means installing it, pulling a model with one command, then chatting with that model from your terminal or your own app. The whole first-run loop takes about ten minutes on a normal laptop.

Ollama is a free tool that runs open-weight language models on your own computer. No cloud account, no per-token bill, no data leaving your machine.

This guide walks through the real first hour: install, pull, run, then wire the local API into a client so you can actually build something with it.

How Do You Install Ollama?

You install Ollama by downloading the app for your OS or running one install script, then it starts as a background service automatically.

On macOS and Windows, download the installer from ollama.com and run it like any other app. It adds a menu bar icon and starts a local server automatically.

On Linux, the vendor publishes a one-line install script: curl -fsSL https://ollama.com/install.sh | sh. Run that in a terminal and it installs the ollama binary plus a systemd service.

Verify the install by running ollama --version in a terminal. If that prints a version number, the background server is already running on port 11434.

• macOS/Windows: download the .dmg or .exe installer from ollama.com/download
• Linux: curl -fsSL https://ollama.com/install.sh | sh
• Docker: docker run -d -p 11434:11434 --name ollama ollama/ollama (add --gpus=all for NVIDIA GPUs)
• Check RAM before picking a model - 8GB handles small models, 16GB+ is more comfortable for 7-8B models

How Do You Pull Your First Model?

You pull a model with ollama pull followed by the model name, and Ollama downloads the weights straight from its library.

Run ollama pull llama3.2 for a small, fast general-purpose model, or ollama pull qwen2.5 for a strong coding-and-reasoning option. Both download in the background with a progress bar.

Model files are large, often several gigabytes, so the first pull depends on your connection speed. Once downloaded, the model is cached locally and future runs start instantly.

Check the model's page on the Ollama library site for exact file size and RAM needs before you pull it. Bigger parameter counts need more memory, and an undersized machine will fail to load the model or run painfully slow.

• ollama pull <model> downloads a model without running it
• ollama list shows every model already downloaded on your machine
• ollama rm <model> deletes a model you no longer need
• Model names support tags, like llama3.2:1b for the smallest variant

How Do You Run a Model and Chat With It?

You run a model with ollama run followed by its name, which opens an interactive chat session right in your terminal.

Type ollama run llama3.2 and Ollama loads the model into memory, then drops you into a prompt where you can type messages and get replies.

If the model was not pulled yet, ollama run downloads it automatically before starting the chat, so you can skip the separate pull step for a quick test.

Type /bye to exit the chat session. The model stays loaded in memory for a few minutes after you exit, so a second run command in that window responds faster than the first.

• ollama run <model> starts an interactive chat in the terminal
• ollama run <model> "your prompt" runs one prompt and prints the answer, useful in scripts
• ollama ps shows which models are currently loaded in memory
• ollama stop <model> unloads a model from memory to free up RAM

How Do You Expose the Local API?

Ollama exposes a local REST API on port 11434 automatically the moment it starts, no extra setup needed.

Every model you pull becomes callable at http://localhost:11434/api/generate for one-off completions or http://localhost:11434/api/chat for multi-turn conversations. Both accept a JSON body with the model name and prompt, and support streaming responses.

Ollama also ships an OpenAI-compatible layer at http://localhost:11434/v1/. Point any OpenAI SDK or tool at that base URL, swap the model name to your local model, and it works with no other code changes.

The API only listens on localhost by default. To reach it from another device on your network, set the OLLAMA_HOST environment variable to 0.0.0.0 before starting the server, and only do that on a trusted network since there is no built-in auth.

• POST /api/chat - conversation-style requests with message history
• POST /api/generate - single-prompt completions
• GET /api/tags - lists locally installed models, useful for a health check
• POST /v1/chat/completions - OpenAI-compatible endpoint for drop-in SDK use

How Do You Connect a Client or App to Ollama?

You connect a client to Ollama by pointing its API base URL at localhost:11434 and calling the same endpoints your app already knows.

With the official Python or JavaScript client libraries, install ollama (pip or npm) and call client.chat(model="llama3.2", messages=[...]) directly against the local server.

With an OpenAI SDK, set base_url to http://localhost:11434/v1 and use any placeholder string for the API key field, since Ollama does not check it locally.

Local model runners like Ollama and LM Studio are also common backends for agent frameworks like LangChain and CrewAI, which both ship built-in Ollama integrations so you can swap a cloud model for a local one in a single config line.

• Python: pip install ollama, then ollama.chat(model='llama3.2', messages=[...])
• JavaScript: npm install ollama, then the same chat() call pattern
• OpenAI SDK: base_url='http://localhost:11434/v1', api_key='ollama' (any string works)
• curl: curl http://localhost:11434/api/chat -d '{"model":"llama3.2","messages":[{"role":"user","content":"hi"}]}'

What Does a Realistic First-Hour Workflow Look Like?

A realistic first hour covers install, one small model pull, a terminal chat test, and one working API call from code.

Minute 0 to 10: install Ollama and confirm ollama --version works. Minute 10 to 20: pull a small model like llama3.2 and chat with it once in the terminal to confirm output quality. Minute 20 to 40: write a five-line script that calls http://localhost:11434/api/chat and prints the response. Minute 40 to 60: swap in the OpenAI-compatible endpoint if your target app already speaks that format, and confirm the same script works unchanged.

When we review client automation stacks that lean on local or self-hosted model runners, the workflows that hold up under real load are the ones that treat the local API exactly like a hosted one from day one, with retries and timeouts, rather than as a toy that never needs error handling.

Skip trying every model in the library on day one. Pick one small model, get the full loop working end to end, then swap models later once the pipeline is proven.

• Do not run ollama pull on a large model as your very first command - test the loop with a small one first
• Set a request timeout in your client code; a cold model load can take 10-30 seconds before the first token

What Goes Wrong When People Set Up Ollama?

The most common Ollama setup failures are an out-of-memory crash on a model that is too large, a port conflict from another local service on 11434, and forgetting the server needs to be running before any API call works.

A model that does not fit in available RAM will either fail to load or fall back to disk paging, which makes every response painfully slow. Check the model's listed RAM requirement against your machine before pulling anything above 8B parameters on a laptop.

If curl to localhost:11434 returns connection refused, the background service is not running. Restarting the app, or running ollama serve manually in a terminal, fixes it.

A quieter failure: leaving OLLAMA_HOST bound to 0.0.0.0 on a shared network with no authentication in front of it. Anyone on that network can hit your API and your GPU. Keep it on localhost unless you have put a reverse proxy with auth in front of it.

• Out-of-memory: pull a smaller model or a quantized variant (models tagged :q4 or similar use less RAM)
• Connection refused: the Ollama server is not running - restart the app or run 'ollama serve'
• Slow first response: normal - the model is loading into memory; subsequent requests are faster
• Never expose OLLAMA_HOST=0.0.0.0 on an untrusted network without adding auth in front of it

When Should You Use Ollama Instead of a Hosted API?

Use Ollama when data privacy, offline access, or per-token cost matter more than having the single best-performing model.

Local models are a good fit for prototyping agent logic without racking up API bills, for handling sensitive data that cannot leave your network, and for offline or edge deployments.

A hosted API from a frontier lab still wins on raw quality for the hardest reasoning and coding tasks, and it needs zero hardware planning. If your app needs the strongest available model and cost per call is not the constraint, a hosted model is usually the simpler choice.

Many teams use both: a local Ollama model for cheap, high-volume, low-risk tasks like classification or drafting, and a hosted model for the smaller share of requests that need top-tier reasoning.