All lessons
Offline AI: run AI models on your own computer
Download AI models and run them on your own laptop, desktop or office server, with no internet and no monthly limit. From what a token is to building private document search, with prices in India, Canada, the USA and the UK.
What you will have at the end
- A private assistant that runs on your own laptop or desktop with the internet switched off.
- Search across your own documents: ask questions of your PDFs and notes and get answers with the page they came from.
- Your own workflows: small Python programs that send work to the model on your machine, such as summarising a folder of files every night.
- A cost sheet comparing your setup with Claude, ChatGPT and Google plans, in rupees, Canadian dollars, US dollars or pounds.
Want to know how the model works inside, with the maths? The free course From a straight line to an LLM builds up from linear regression to attention and transformers, with a simulation in every lesson.
0 of 119 lessons done
Without the tick, your progress lasts until you close this tab. Either way it stays in this browser: there is no sign-up and nothing is sent to us.
All modules
Part A: Foundations
1 Welcome, and why run AI yourself 5 lessons
2 How a language model works 7 lessons
- 2.1 Text in, text out: predicting the next token
- 2.2 Parameters: what "7B" and "120B" mean
- 2.3 Training and inference: learning versus using
- 2.4 Transformers and attention, explained with a classroom
- 2.5 Dense models and Mixture-of-Experts
- 2.6 Base, instruct, reasoning and multimodal models
- 2.7 Why models make things up
3 Tokens: the unit of everything 10 lessons
- 3.1 What a token is
- 3.2 Tokenizers differ: languages, code and counts
- 3.3 The context window: a model's working memory
- 3.4 Input, output and thinking tokens
- 3.5 Tokens per second: reading speed and writing speed
- 3.6 Saving tokens 1: short, clear prompts
- 3.7 Saving tokens 2: trimming and summarising context
- 3.8 Saving tokens 3: caching, limits and structured output
- 3.9 Why your own machine makes tokens "unlimited"
- 3.10 Measure tokens and speed on your own machine
Part B: Hardware
4 The parts that matter for AI 6 lessons
5 Buying guide with prices in four countries 7 lessons
6 Power, heat and running costs 4 lessons
Part C: Software and setup
7 Getting and choosing models 6 lessons
8 Quantisation: shrinking models to fit 5 lessons
9 Installing your first local AI 7 lessons
10 Settings and prompting 5 lessons
Part D: Building offline pipelines
11 The local API 4 lessons
12 Private document search (RAG), fully offline 7 lessons
- 12.1 Why the model cannot read your files by itself
- 12.2 Embeddings: turning meaning into coordinates
- 12.3 Cutting documents into chunks
- 12.4 Vector databases you can run locally
- 12.5 Finding, re-ranking and citing sources
- 12.6 Reading PDFs, scans, spreadsheets and email
- 12.7 Testing answers: did it find the right page?
13 Agents and automation offline 8 lessons
- 13.1 What an agent is
- 13.2 Letting a local model use tools safely
- 13.3 A coding assistant that runs on your machine
- 13.4 Offline speech to text and text to speech
- 13.5 Reading images and screenshots locally
- 13.6 Making images offline
- 13.7 Scheduled jobs: summaries and reports overnight
- 13.8 Guard rails: stopping an agent doing damage
14 Customising models 5 lessons
Part E: Scale, money, limits and careers
15 Scaling up: from one laptop to a team server 7 lessons
- 15.1 Many users, one model: batching
- 15.2 Serving engines built for many users
- 15.3 Splitting a model across graphics cards or machines
- 15.4 Running it on your office network
- 15.5 Watching speed, memory and errors
- 15.6 Hybrid: local first, cloud for the hardest jobs
- 15.7 Renting graphics cards by the hour
16 Cost: local AI against subscriptions 6 lessons
17 Pros, cons and limits 8 lessons
- 17.1 The quality gap: where cloud models still lead
- 17.2 Speed limits and where they come from
- 17.3 Memory limits: model, context and cache
- 17.4 Limits in theory: knowledge cut-off, reasoning, long context
- 17.5 Limits in practice: setup, updates, heat and noise
- 17.6 Security and privacy: what local protects and what it does not
- 17.7 Licences, law and sector rules in four countries
- 17.8 Every cloud feature and its local equivalent
18 Who uses this: sectors and careers 12 lessons
- 18.1 Healthcare and clinics
- 18.2 Law and accounting
- 18.3 Finance and banking
- 18.4 Teachers, students and offline classrooms
- 18.5 Software and data teams
- 18.6 Research and science labs
- 18.7 Government and air-gapped sites
- 18.8 Small business, retail, hotels and logistics
- 18.9 Field work with no internet
- 18.10 Writers, creators and translators
- 18.11 Local AI skills in job ads
- 18.12 Capstone: an offline AI setup for one sector