Install Ollama on a VPS and serve LLMs over HTTPS
A clean Ollama install on Ubuntu — model picking that respects your RAM, the API exposed behind Nginx with token auth, and a real throughput benchmark on a $30 VPS. Covers both CPU-only and GPU setups.
Realistic expectations
Ollama lets you run open-weight LLMs on your own server. What you get is real but bounded:
- CPU-only on a $20-40 VPS: 5-15 tokens/sec on quantized 7B-14B models. Usable for personal "ask Claude a question without sending data to Anthropic" workflows, AI-agent automations, batch summarization. Not interactive enough for a ChatGPT-style live conversation.
- Small GPU (consumer or cloud): 30-80 tokens/sec on the same models. Interactive. Cost rises sharply.
- The 70B and 405B class: requires multi-GPU setups. Out of scope for a single VPS.
Prerequisites
- Ubuntu 22.04 or 24.04 VPS.
- At least 8 GB RAM for a meaningful model. 4 GB will run a 3B at low quality; 16-32 GB lets you run quantized 14B comfortably.
- A non-root user with sudo.
- Optional: a domain pointed at the VPS if you want HTTPS API access from outside.
Tested on Ubuntu 22.04, Ollama 0.4.x, on a 16 GB Inferno VPS in Frankfurt (CPU-only, AMD EPYC).
Step 1 — Install Ollama
curl -fsSL https://ollama.com/install.sh | sh
The installer creates a systemd service, an ollama system user, and starts the daemon on port 11434 bound to 127.0.0.1.
systemctl status ollama
ollama --version
Step 2 — Pick and pull a model that fits
Rough RAM requirements for the quantized (Q4_K_M) versions Ollama ships by default:
| Model | RAM at idle | Notes |
|---|---|---|
llama3.2:3b | ~2.5 GB | Tiny, fast on any VPS. Limited reasoning. |
llama3.1:8b | ~5 GB | Sweet spot for personal use. Capable enough for summarization and Q&A. |
qwen2.5:14b | ~9 GB | Stronger reasoning. Slower per token. |
mistral-small:24b | ~14 GB | Strong for the size class. Needs 16 GB+ RAM. |
llama3.1:70b | ~45 GB | Needs a beefy machine. Not for typical VPS. |
Start with llama3.1:8b for general use:
ollama pull llama3.1:8b
ollama list
Step 3 — Test in the CLI
ollama run llama3.1:8b "Summarize the Linux kernel boot process in 200 words."
You'll see tokens stream out. Note the speed at the end (Ollama prints stats). On our 16 GB Inferno test box we measured 11.2 tokens/sec for this model with a typical prompt.
Step 4 — Expose the API (optional)
Skip this section if you only call Ollama from the same machine.
By default the API is on 127.0.0.1:11434 — local only. To allow remote calls, change the bind:
sudo systemctl edit ollama
# Add:
# [Service]
# Environment="OLLAMA_HOST=0.0.0.0:11434"
sudo systemctl restart ollama
Don't stop here. The bare Ollama API has no authentication. Anyone who finds port 11434 can use your model and your CPU. Always front it with Nginx and a token check.
Step 5 — Nginx with token auth
sudo apt install -y nginx certbot python3-certbot-nginx
sudo nano /etc/nginx/sites-available/ollama
map $http_authorization $auth_ok {
default 0;
"Bearer YOUR_LONG_RANDOM_TOKEN_HERE" 1;
}
server {
listen 80;
server_name ollama.example.com;
location /.well-known/acme-challenge/ { root /var/www/html; }
location / {
if ($auth_ok = 0) { return 401 "unauthorized\n"; }
proxy_pass http://127.0.0.1:11434;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_buffering off;
proxy_read_timeout 600s;
client_max_body_size 50M;
}
}
Generate a strong token:
openssl rand -hex 32
Replace YOUR_LONG_RANDOM_TOKEN_HERE with the output. Enable and certify:
sudo ln -s /etc/nginx/sites-available/ollama /etc/nginx/sites-enabled/
sudo nginx -t && sudo systemctl reload nginx
sudo certbot --nginx -d ollama.example.com --redirect
Step 6 — Test the secured API
# Should fail with 401
curl https://ollama.example.com/api/tags
# Should succeed
curl https://ollama.example.com/api/tags \
-H "Authorization: Bearer YOUR_TOKEN"
# Chat completion
curl https://ollama.example.com/api/chat \
-H "Authorization: Bearer YOUR_TOKEN" \
-d '{"model":"llama3.1:8b","messages":[{"role":"user","content":"hello"}],"stream":false}'
OpenAI-compatible endpoint
Ollama also exposes an OpenAI-style API at /v1/. Drop it into any tool that supports an OpenAI-compatible base URL:
OPENAI_BASE_URL=https://ollama.example.com/v1
OPENAI_API_KEY=YOUR_TOKEN
MODEL=llama3.1:8b
Works with the OpenAI Python SDK, LangChain, LlamaIndex, n8n's "OpenAI" node, Continue.dev, Aider, and many others. See n8n AI agents self-hosted for the n8n integration.
GPU setup (optional)
If you provision a GPU VPS:
sudo apt install -y nvidia-driver-535
nvidia-smi # confirm the GPU is visible
# Reinstall Ollama — installer detects the GPU and configures CUDA support
curl -fsSL https://ollama.com/install.sh | sh
ollama list
Re-run the benchmark; throughput should jump 5-10x.
Real benchmark numbers
From our 16 GB Inferno CPU-only test, identical prompt across models:
| Model | Tokens/sec | Time to first token |
|---|---|---|
| llama3.2:3b | 28.4 | 0.6 s |
| llama3.1:8b | 11.2 | 1.2 s |
| qwen2.5:14b | 5.8 | 2.4 s |
For comparison, ChatGPT-class hosted models typically run at 50-100 tokens/sec on a fast connection.
Resource controls
Ollama unloads models from RAM after 5 minutes of inactivity by default. Tune via env vars:
OLLAMA_KEEP_ALIVE=30m # how long to keep a model loaded
OLLAMA_MAX_LOADED_MODELS=2 # cap how many fit concurrently
OLLAMA_NUM_PARALLEL=2 # concurrent requests per model
Add to the systemd override created earlier.
Common Ollama mistakes
"Killed" or sudden OOM during the first prompt
The model is bigger than your RAM. free -h while Ollama loads to see how much it actually takes. Pick a smaller model or add swap (slow but functional).
"403 from /v1/ endpoint"
Some clients send Authorization differently. The map trick above only checks the exact header value; if your client uses lowercase or appends spaces, normalize the Nginx config.
"Very slow first token after idle"
The model was unloaded due to OLLAMA_KEEP_ALIVE. The first request after idle re-loads it (5-20 s on CPU). Either bump keep-alive or accept the warmup latency.
"Streaming response not streaming"
Nginx is buffering. proxy_buffering off; in the Nginx config — already set in our example. Some other middleware (Cloudflare proxying) may also buffer; bypass for the Ollama domain.
VPS for Ollama on CPU?
For 8B models the sweet spot is 16 GB RAM with fast CPU. Inferno's higher-tier plans (16 GB) handle this well, and the eight EU + Asia locations let you run inference close to where you call from.
What to do next
- Open WebUI — a ChatGPT-style interface for your Ollama instance.
- n8n AI agents with local Ollama as the LLM.
- LocalAI — alternative API layer with broader model support.
- Best VPS for Ollama.
Author: Ivo Pärnaste — DevOps, Tallinn. Last tested: 2026-05-22 on Ollama 0.4.6, Ubuntu 22.04, Inferno 16 GB AMD EPYC plan, CPU-only.