Skip to content
Recaply.
Sign inSubmit a tool

LearnTutorials

OllamaTutorialsOpen Ollama

Run Gemma 4 12B locally with Ollama

Run Google's Gemma 4 12B on your own computer with Ollama in 20 minutes, then chat, ask about images and connect your apps, with no API bills.

By Recaply EditorsIntermediate20 min5 stepsUpdated

A small cut gem sits on an open laptop's keyboard while a hand plugs a cable into the laptop

What you need

Ollama needs macOS 14 Sonoma or newer, and Intel Macs run models on the CPU only. On Windows, it needs Windows 10 22H2 or newer.

  1. Step 1: Install Ollama

    3 min

    Ollama downloads, runs and serves local models, and the commands are the same on every system.

    1. Go to ollama.com/download and pick your system.
    2. Mac: open the disk image, drag Ollama into Applications and open it. Windows: run OllamaSetup.exe.
    3. Linux: run the install command below in a terminal.
    4. Open a new terminal window and run ollama ls to check that the command works.
    Linux: run this
    curl -fsSL https://ollama.com/install.sh | sh

    You should see a header row with no models under it yet, and no error.

    The terminal says it can't find the ollama command?

    On a Mac, open the Ollama app once and let it link the command to /usr/local/bin. Then open a new terminal window and try again.

  2. Step 2: Download Gemma 4 12B

    8 min

    You download the model once. After that, every chat runs on your machine, even offline.

    1. Run the command below.
    2. Wait for the download to finish. It's about 8 GB, so it can take a few minutes.
    Run this
    ollama pull gemma4:12b

    You should see a progress bar that fills up and finishes without an error.

    Your main drive is running out of space?

    Models are saved in ~/.ollama/models on a Mac. Set the OLLAMA_MODELS variable to keep them on another drive, then pull the model again.

    Type the :12b tag. Plain gemma4 pulls the smaller E4B model instead.

  3. Step 3: Chat in the terminal

    3 min

    A short test shows the model runs before you add images or apps.

    1. Run the command below to open a chat.
    2. At the >>> prompt, type a short question and press Enter.
    3. In a second terminal window, run ollama ps to see where the model is loaded.
    Run this
    ollama run gemma4:12b

    You should see a reply in the terminal, and gemma4:12b listed by ollama ps.

    Replies come out very slowly?

    Check the PROCESSOR column in ollama ps. 100% CPU means the model didn't load onto your GPU. Close other apps, or try the smaller gemma4:e4b.

  4. Step 4: Ask about an image

    3 min

    Gemma 4 reads images, so it can describe screenshots, charts and photos without sending them anywhere.

    1. Write your question and put the image's full path at the end, inside the quotes.
    2. Run it as one command, as in the example below.
    Run this
    ollama run gemma4:12b "What does this chart show? [PATH TO YOUR IMAGE]"

    You should see a description of your image, written by the model on your own computer.

    It answers without looking at the image?

    Use the full path, such as /Users/you/Desktop/chart.png, and check the file name. Keep the path inside the same quotes as the question.

    Google says the 12B model also understands audio. Ollama lists this tag for text and image input only.

  5. Step 5: Connect an app to Gemma

    3 min

    Ollama serves an OpenAI-compatible API, so most apps with an OpenAI option can use Gemma.

    1. Keep the Ollama app open, or run ollama serve in a terminal.
    2. In your app's OpenAI or custom provider settings, set the base URL to http://localhost:11434/v1/.
    3. Set the model to gemma4:12b. If the app asks for an API key, type ollama.
    4. To test the connection, run the command below.
    Test it (Mac or Linux)
    curl http://localhost:11434/v1/chat/completions -H "Content-Type: application/json" -d '{"model": "gemma4:12b", "messages": [{"role": "user", "content": "Say hello in 5 words."}]}'

    You should see a JSON reply with Gemma's answer in the content field.

    The app can't connect?

    Check that Ollama is running and that the URL ends in /v1/. Ollama only listens on this computer by default, so the app must run here too.

    Ollama ignores the API key. Never paste a real OpenAI key into this field or into shared config files.

You get

Gemma 4 12B running on your own computer, with chat, image questions and a local API for your apps

Try next

Fine-tune an LLM to match a writing style with Together AI6 steps · 40 minUse Gemma 4 for free with Google AI Studio5 steps · 16 minRun Claude Code on a local model with Ollama6 steps · 24 min

Did this guide work for you?

Sources

0 of 5Next: Install Ollama