How to Deploy Ollama with Docker and Docker Compose

Running Ollama in Docker provides better isolation, easier updates and consistent deployment across servers. This guide covers Docker and Docker Compose deployment with GPU support.

Step 1 — CPU Only Deployment

docker run -d   -v ollama:/root/.ollama   -p 11434:11434   --name ollama   --restart always   ollama/ollama

Step 2 — GPU Deployment (NVIDIA)

# Install NVIDIA Container Toolkit first
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
sudo apt install nvidia-container-toolkit -y
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

# Run with GPU
docker run -d   --gpus all   -v ollama:/root/.ollama   -p 11434:11434   --name ollama   --restart always   ollama/ollama

Step 3 — Docker Compose with Open WebUI

nano docker-compose.yml

version: '3.8'
services:
  ollama:
    image: ollama/ollama
    volumes:
      - ollama_data:/root/.ollama
    ports:
      - "11434:11434"
    restart: always
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]

  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    volumes:
      - webui_data:/app/backend/data
    ports:
      - "3000:8080"
    environment:
      - OLLAMA_BASE_URL=http://ollama:11434
    depends_on:
      - ollama
    restart: always

volumes:
  ollama_data:
  webui_data:

docker compose up -d

Step 4 — Pre-Pull Models on Start

# Pull model after container starts
docker exec ollama ollama pull llama3
docker exec ollama ollama pull mistral

Step 5 — Monitor Container

docker stats ollama
docker logs ollama --tail 50 -f

Conclusion

Docker deployment makes Ollama portable and easy to update. Our team deploys containerised AI environments including Ollama with Open WebUI.