Ollama on Mac: from install to your first model

Ollama is the shortest path from “I have a Mac” to “I am talking to a model running on my own laptop”. It wraps llama.cpp, uses Metal on Apple Silicon automatically, and needs no Python environment, no CUDA, and no model conversion.

Here is the whole path, including the two settings that decide whether it feels fast or sluggish.

1. Install

Two ways, both fine:

brew install ollama

Or download the app from ollama.com — the menu bar app starts the background service for you.

If you install through Homebrew, start the service once:

brew services start ollama

Confirm it is running:

ollama --version

2. Pick a model that fits

This is where most people go wrong. On a Mac the constraint is unified memory, and macOS itself wants a few gigabytes of it.

A rough budget:

Mac memory	Comfortable model size (4-bit)	Typical choice
8 GB	up to ~3 GB	4B models
16 GB	up to ~7 GB	8B models
24 GB	up to ~12 GB	14B models
32 GB	up to ~18 GB	14B–30B MoE
64 GB+	30 GB and beyond	27B–70B class

The number that matters is the download size on disk, not the parameter count. A 4-bit 8B model is roughly 5 GB; the same model at 8-bit is roughly 9 GB for barely any quality gain.

Start with something small:

ollama run qwen3:8b

First run downloads the weights (a few GB), then drops you into a prompt. Type /bye to leave.

3. Keep it loaded between prompts

By default Ollama unloads a model after five minutes of inactivity. The next prompt then pays the load cost again — on a laptop that feels like a stall.

export OLLAMA_KEEP_ALIVE=30m

Set it in your shell profile, or pass keep_alive in the API request. If you are working in a single session, keeping the model resident is the single biggest perceived-speed win.

4. Watch the memory ceiling

Two things grow after the model loads:

• KV cache — grows with context length. A 32k context on a 14B model can add several gigabytes.
• Other apps — browsers are the usual culprit.

If generation suddenly slows to a crawl or macOS starts swapping, you have crossed the line. Check with:

ollama ps

It shows which models are loaded and how much memory they hold.

5. A realistic expectation of speed

Tokens per second on Apple Silicon track memory bandwidth far more than GPU core count:

• 8 GB M1/M2 Air — a 4B model is usable for chat; 8B is borderline.
• 16 GB M4 — 8B models feel responsive; 14B works but is slower.
• M4 Pro / Max — 14B is comfortable, 30B-class MoE models become practical.

If you want exact expectations for your machine, use the Can my Mac run it? checker — it estimates throughput from your chip’s bandwidth and the model size.

6. Point your tools at it

Ollama exposes an OpenAI-compatible endpoint at http://localhost:11434. Anything that accepts a custom base URL can use it: chat clients, editor extensions, agent frameworks. No key required, and nothing leaves the machine.

That is the real payoff: a private model that every tool on your Mac can share.

What next

• Compare what different memory tiers can actually run: Best local LLM for a 16 GB Mac.
• For image generation, the equivalent starting point is ComfyUI on Apple Silicon.

FAQ

Does Ollama work on an M1 Mac with 8 GB of memory?

Yes, with small models. 8 GB leaves roughly 3 GB for weights after macOS and your apps, which fits 4B-parameter models at 4-bit quantisation. Expect a usable chat experience, not fast long-document work.

Does Ollama run on the GPU or the CPU?

On Apple Silicon it uses Metal, so inference runs on the GPU using the same unified memory as the CPU. You do not need to configure anything — if Metal is available, Ollama uses it.

Where are models stored?

Under ~/.ollama/models. A 4-bit 8B model is roughly 5 GB, so disk space adds up quickly if you keep several families installed. ollama list shows what you already have.

Can I use Ollama with apps that expect the OpenAI API?

Yes. Ollama exposes an OpenAI-compatible endpoint at http://localhost:11434. Anything that accepts a custom base URL — chat clients, editor extensions, agent frameworks — can use it with no API key.