How to use Ollama

Ollama is the fastest way to run open models from the command line. One command downloads a model and drops you into a chat; a local API lets your own apps talk to it. New to local AI in general? Read Getting started with local AI first.

Install

macOS / Windows — download the installer from ollama.com/download and run it. It installs the ollama command and a small background service.
Linux:

curl -fsSL https://ollama.com/install.sh | sh

Verify it's working:

ollama --version

Run your first model

This single command downloads the model (first time only) and starts a chat:

ollama run gemma3

Type your message, press Enter, and chat. Use /bye to exit. To run Gemma 4 12B once you have ~16 GB free:

ollama run gemma4:12b

Tip: check ollama.com/library for the exact model name/tag. Tags pick a size or quantization, e.g. gemma3:12b or llama3.1:8b.

Managing models

ollama pull gemma4:12b     # download without starting a chat
ollama list                # show installed models + sizes
ollama rm gemma3           # delete a model to free disk space
ollama ps                  # see what's currently loaded in memory

Use it from your own apps (the API)

Ollama serves a local API at http://localhost:11434. Plain generate:

curl http://localhost:11434/api/generate -d '{
  "model": "gemma4:12b",
  "prompt": "Explain I2C in one sentence."
}'

It also exposes an OpenAI-compatible endpoint, so most OpenAI SDKs work by pointing the base URL at Ollama:

from openai import OpenAI
client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama")
r = client.chat.completions.create(
    model="gemma4:12b",
    messages=[{"role": "user", "content": "Hello!"}],
)
print(r.choices[0].message.content)

Tips

GPU is used automatically when available (NVIDIA/AMD on Windows & Linux, Metal on Mac) — no config needed.
Out of memory? Pick a smaller model or a lower quantization (e.g. a q4 tag), or shorten the context.
Customise a model with a Modelfile to bake in a system prompt or parameters:

FROM gemma4:12b
SYSTEM "You are a terse embedded-systems assistant."
PARAMETER temperature 0.5

ollama create my-gemma -f Modelfile
ollama run my-gemma

That's the whole loop: install → run → chat or call the API. Prefer a graphical app instead? See Using models in LM Studio.