host models - locally on a MacBook or PC

Host models locally on a MacBook or PC with Ollama

The Simplest Intro to Hosting Your Own Model (vLLM / SGLang)

If you want a “one command” way to run open-source LLMs on your personal computer, Ollama is a very simple option. After installation, it works in your terminal and also exposes a local REST API at http://localhost:11434 that your apps can call. The first run pulls and caches a model. After that you can use it offline.

# 1) Install
# macOS / Windows: download the installer from the website
# Linux:
curl -fsSL https://ollama.com/install.sh | sh

# 2) First run: pull a model and chat
ollama run llama3

# 3) Optional: keep a background service running; the API listens on 11434
ollama serve

# 4) Call the HTTP API
curl http://localhost:11434/api/generate -d '{
  "model": "llama3",
  "prompt": "Reply in under 50 words."
}'

# Handy model management
# ollama list            # list downloaded models
# ollama stop llama3     # stop a loaded model

Ollama has a quick start and an API reference. There is also an official Windows app, so you can avoid the terminal if you prefer a GUI.

---

Hosting models on a laptop without Ollama

If you want a friendlier GUI, an OpenAI compatible endpoint, or a different backend, try these options. All of them can run locally and offline.

• LM Studio: Desktop all in one app with a built in model catalog and chat UI. It can use Metal, CUDA, or Vulkan for GPU acceleration and is newcomer friendly, especially on Windows laptops with only integrated graphics.

• Text Generation WebUI (oobabooga): Feature rich web dashboard that supports GGUF and GPTQ and multiple backends. Portable builds are available, just unzip and run.

• llama.cpp / llama-cpp-python: Lightweight, native inference stack. You can run it from the command line or expose an OpenAI compatible local server for programmatic use.

• LocalAI: Go based, OpenAI API compatible local service. Good for Docker setups where you want to self host an OpenAI like endpoint.

• GPT4All: Cross platform desktop app with one click model downloads and offline chat. Ideal for non technical users.

• Open WebUI: Clean web interface that sits in front of Ollama or any OpenAI compatible backend, making model management and chatting easier.

• MLC LLM: Deployment engine for Metal, CUDA, and Vulkan. It compiles models for efficient native execution and suits readers who want to tune for maximum performance.

• vLLM: High throughput inference and serving framework. Consider it if your laptop has a strong NVIDIA GPU and you need concurrency or long context windows.

• KoboldCpp / Jan: KoboldCpp is a lightweight GUI for writing and story tasks, built on llama.cpp. Jan is an offline ChatGPT style desktop app.

Bottom line: Treat Ollama as simple, Docker like local infrastructure with a built in API. Use the tools above when you want stronger GUIs (LM Studio, Open WebUI, GPT4All), more control and performance (llama.cpp, MLC LLM, vLLM), or specialized writing and offline chat experiences (KoboldCpp, Jan).