Module 9: Installing your first local AI · Lesson 9.5 · about 20 min
A chat window that runs on your machine
Give your local model a ChatGPT-style window with history, file uploads and several models, all running on your own computer.
Module 9 · about 20 minutes · read first: 9.2. You’ll be able to use your local model in a full chat window with saved history and documents.
1. Why this matters
The terminal is fine for testing, but for daily use you want what the cloud sites give you: a list of past chats, a model picker, file uploads, and a page your family or team can open. You can have all of that on your own machine.
2. Three options, from simplest
- The app you already have. LM Studio has a chat window built in, and the Ollama desktop app has a simple one too.
- llama.cpp’s own page.
llama-serverserves a clean chat page athttp://localhost:8080(Lesson 9.3). - Open WebUI, a free, open-source interface that looks and feels like the cloud chat sites, with chat history, several models, document upload and user accounts for a team. It talks to Ollama, or to any server that speaks the OpenAI-style API (Lesson 11.1).
3. Install Open WebUI
It is a Python program. It needs a recent Python (its install page names the version; at the time of writing it asks for Python 3.11). With Ollama already running:
python -m venv webui-env
# Windows: webui-env\Scripts\activate Mac/Linux: source webui-env/bin/activate
pip install open-webui
open-webui serve
Open http://localhost:8080, create the first account (it stays on your machine and becomes the admin, the account that manages the others), and your Ollama models appear in the model picker. If something else already uses port 8080, start it with open-webui serve --port 8081.
Its pages also explain a Docker install (Docker runs a program in a sealed box with everything it needs), which many people prefer for keeping it updated; either way works.
Watch it happen: one message, four hops, no internet
You type. In the browser you open http://localhost:8080 and ask "Plan a week of school lunches". The browser sends it to Open WebUI on port 8080, on this same computer.
The window passes it on. Open WebUI adds the earlier messages of this chat and sends the lot to Ollama on port 11434, also on this computer.
The engine writes. Ollama runs Gemma 4 12B, which writes the answer at, say, 20 tokens per second. A 400-token answer takes 20 seconds.
Back to you, and saved. The words stream back through Open WebUI to your browser, and the chat is saved in a database file on your disk. At no point did anything leave the house.
Maths and principle
Principle: the interface and the engine are separate programs that talk over a local address, so either can be swapped.
Equation: what you see = interface(engine(model, your messages)).
Analogy: a TV and a set-top box. Change the box and keep the TV, or the other way round, as long as both use the same cable.
Try it: Open WebUI on port 8080 and Ollama on 11434. Which address does your browser open? (8080. Open WebUI calls 11434 itself.)
- Interface
- the window you use: chat list, model picker, uploads (Open WebUI, LM Studio's chat, llama-server's page)
- Engine
- the program that actually runs the model (Ollama, llama.cpp, MLX)
- Port
- the number that picks which program answers: 8080 for the window, 11434 for Ollama
- Answers per hour
- how many replies one engine can give, one after another, at its writing speed
- Physical meaning
- A shared chat window is like one cook serving a family: everyone gets their own plate and their own history, but the cook makes one dish at a time. Writing speed sets how many people can share it comfortably.
“A chat window that looks like ChatGPT must be sending my chats to the cloud.” Looks say nothing about where the data goes. Run locally and connected only to Ollama, Open WebUI keeps every chat in a file on your own disk. Test it: pull the cable and it keeps working (Lesson 9.6).
4. Worked example
Made-up numbers. A family shares one desktop with a 16 GB graphics card. Dad installs Ollama and Open WebUI, pulls gemma4:12b, and makes accounts for three people. Each sees only their own chats. Homework help, recipes and letter drafts all stay in the house.
5. Practice
- Open your app’s chat window or
llama-server’s page and save one useful chat. - Install Open WebUI and pick your Gemma 4 model from its list.
- Upload a short text file and ask a question about it.
Answers
- Any chat that saves and reopens is the goal.
- If no models appear, check Ollama is running (
ollama listin a terminal). - Open WebUI reads the file and adds the relevant parts to your question. Module 12 explains how this document search works and how to make it better.
New words: interface (the window you use), admin (the first account, which manages the others), Docker (a way to run a program in a sealed box with everything it needs).
- Three ways to get a chat window: your app’s own, llama-server’s page, or Open WebUI.
- Open WebUI is the window; Ollama (or another engine) runs the model; they talk on local ports (8080 and 11434).
- Chats and accounts are stored on your own machine.
- One program per port: if 8080 is taken, use —port 8081.
- Answers per hour = tokens per second × 3,600 ÷ tokens per answer.
Check yourself
Sources
- Open WebUI (open source chat interface for local models), https://github.com/open-webui/open-webui
- Open WebUI documentation: installing with pip or Docker and connecting to Ollama (official page), https://docs.openwebui.com
- Ollama desktop app, https://ollama.com
- llama.cpp server and its built-in chat page, project documentation (official page), https://github.com/ggml-org/llama.cpp/tree/master/tools/server
Your ticks stay in this browser. Save them as a file on the course home.
Updated 2026-11-22. Prices and model names change fast: check the official source before you buy.