Install Ollama on a VPS and serve LLMs over HTTPS

A clean Ollama install on Ubuntu — model picking that respects your RAM, the API exposed behind Nginx with token auth, and a real throughput benchmark on a $30 VPS. Covers both CPU-only and GPU setups.

Disclosure: contains an affiliate link to Inferno VPS.

Realistic expectations

Ollama lets you run open-weight LLMs on your own server. What you get is real but bounded:

Prerequisites

Tested on Ubuntu 22.04, Ollama 0.4.x, on a 16 GB Inferno VPS in Frankfurt (CPU-only, AMD EPYC).

Step 1 — Install Ollama

curl -fsSL https://ollama.com/install.sh | sh

The installer creates a systemd service, an ollama system user, and starts the daemon on port 11434 bound to 127.0.0.1.

systemctl status ollama
ollama --version

Step 2 — Pick and pull a model that fits

Rough RAM requirements for the quantized (Q4_K_M) versions Ollama ships by default:

ModelRAM at idleNotes
llama3.2:3b~2.5 GBTiny, fast on any VPS. Limited reasoning.
llama3.1:8b~5 GBSweet spot for personal use. Capable enough for summarization and Q&A.
qwen2.5:14b~9 GBStronger reasoning. Slower per token.
mistral-small:24b~14 GBStrong for the size class. Needs 16 GB+ RAM.
llama3.1:70b~45 GBNeeds a beefy machine. Not for typical VPS.

Start with llama3.1:8b for general use:

ollama pull llama3.1:8b
ollama list

Step 3 — Test in the CLI

ollama run llama3.1:8b "Summarize the Linux kernel boot process in 200 words."

You'll see tokens stream out. Note the speed at the end (Ollama prints stats). On our 16 GB Inferno test box we measured 11.2 tokens/sec for this model with a typical prompt.

Step 4 — Expose the API (optional)

Skip this section if you only call Ollama from the same machine.

By default the API is on 127.0.0.1:11434 — local only. To allow remote calls, change the bind:

sudo systemctl edit ollama
# Add:
# [Service]
# Environment="OLLAMA_HOST=0.0.0.0:11434"

sudo systemctl restart ollama

Don't stop here. The bare Ollama API has no authentication. Anyone who finds port 11434 can use your model and your CPU. Always front it with Nginx and a token check.

Step 5 — Nginx with token auth

sudo apt install -y nginx certbot python3-certbot-nginx
sudo nano /etc/nginx/sites-available/ollama
map $http_authorization $auth_ok {
    default 0;
    "Bearer YOUR_LONG_RANDOM_TOKEN_HERE" 1;
}

server {
    listen 80;
    server_name ollama.example.com;

    location /.well-known/acme-challenge/ { root /var/www/html; }

    location / {
        if ($auth_ok = 0) { return 401 "unauthorized\n"; }

        proxy_pass http://127.0.0.1:11434;
        proxy_http_version 1.1;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_buffering off;
        proxy_read_timeout 600s;
        client_max_body_size 50M;
    }
}

Generate a strong token:

openssl rand -hex 32

Replace YOUR_LONG_RANDOM_TOKEN_HERE with the output. Enable and certify:

sudo ln -s /etc/nginx/sites-available/ollama /etc/nginx/sites-enabled/
sudo nginx -t && sudo systemctl reload nginx
sudo certbot --nginx -d ollama.example.com --redirect

Step 6 — Test the secured API

# Should fail with 401
curl https://ollama.example.com/api/tags

# Should succeed
curl https://ollama.example.com/api/tags \
  -H "Authorization: Bearer YOUR_TOKEN"

# Chat completion
curl https://ollama.example.com/api/chat \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -d '{"model":"llama3.1:8b","messages":[{"role":"user","content":"hello"}],"stream":false}'

OpenAI-compatible endpoint

Ollama also exposes an OpenAI-style API at /v1/. Drop it into any tool that supports an OpenAI-compatible base URL:

OPENAI_BASE_URL=https://ollama.example.com/v1
OPENAI_API_KEY=YOUR_TOKEN
MODEL=llama3.1:8b

Works with the OpenAI Python SDK, LangChain, LlamaIndex, n8n's "OpenAI" node, Continue.dev, Aider, and many others. See n8n AI agents self-hosted for the n8n integration.

GPU setup (optional)

If you provision a GPU VPS:

sudo apt install -y nvidia-driver-535
nvidia-smi              # confirm the GPU is visible

# Reinstall Ollama — installer detects the GPU and configures CUDA support
curl -fsSL https://ollama.com/install.sh | sh
ollama list

Re-run the benchmark; throughput should jump 5-10x.

Real benchmark numbers

From our 16 GB Inferno CPU-only test, identical prompt across models:

ModelTokens/secTime to first token
llama3.2:3b28.40.6 s
llama3.1:8b11.21.2 s
qwen2.5:14b5.82.4 s

For comparison, ChatGPT-class hosted models typically run at 50-100 tokens/sec on a fast connection.

Resource controls

Ollama unloads models from RAM after 5 minutes of inactivity by default. Tune via env vars:

OLLAMA_KEEP_ALIVE=30m     # how long to keep a model loaded
OLLAMA_MAX_LOADED_MODELS=2 # cap how many fit concurrently
OLLAMA_NUM_PARALLEL=2     # concurrent requests per model

Add to the systemd override created earlier.

Common Ollama mistakes

"Killed" or sudden OOM during the first prompt

The model is bigger than your RAM. free -h while Ollama loads to see how much it actually takes. Pick a smaller model or add swap (slow but functional).

"403 from /v1/ endpoint"

Some clients send Authorization differently. The map trick above only checks the exact header value; if your client uses lowercase or appends spaces, normalize the Nginx config.

"Very slow first token after idle"

The model was unloaded due to OLLAMA_KEEP_ALIVE. The first request after idle re-loads it (5-20 s on CPU). Either bump keep-alive or accept the warmup latency.

"Streaming response not streaming"

Nginx is buffering. proxy_buffering off; in the Nginx config — already set in our example. Some other middleware (Cloudflare proxying) may also buffer; bypass for the Ollama domain.

VPS for Ollama on CPU?

For 8B models the sweet spot is 16 GB RAM with fast CPU. Inferno's higher-tier plans (16 GB) handle this well, and the eight EU + Asia locations let you run inference close to where you call from.

View Inferno plans →

What to do next


Author: Ivo Pärnaste — DevOps, Tallinn. Last tested: 2026-05-22 on Ollama 0.4.6, Ubuntu 22.04, Inferno 16 GB AMD EPYC plan, CPU-only.