Home / Learn / Data & AI / Offline AI / Lesson 16.5
All lessons

Module 16: Cost: local AI against subscriptions · Lesson 16.5 · about 20 min

When the cloud wins and when local wins

A fair scorecard. The cloud wins on the strongest models, bursts and zero upfront cost; local wins on privacy, volume, offline use and no limits. A decision tree to choose for each job.

Key points, meanings and formulas on one page: choose "Save as PDF" in the print box.

Module 16 · about 20 minutes · read first: 16.4. You’ll be able to decide, job by job, whether the cloud or your own machine is the better home.

Neither side wins everything: choose the home for each job by asking whether the data may leave, whether a local model is good enough, and how much of the job there is.

1. Why this matters

This course is about running AI yourself, but it is not against the cloud. Choosing well means knowing where each side is truly better, so you neither overspend on hardware nor send private data out when you did not need to.

2. The scorecard

Need Cloud Local
Strongest model on hard tasks Wins (frontier models) Close on many tasks, behind on the hardest
Money up front Wins (none) Hardware first
Short bursts, occasional use Wins Hardware sits idle
Web search, video, phone apps, syncing Wins out of the box Possible with work (Lesson 17.8)
Privacy and data rules Depends on contract and settings Wins: nothing leaves
Works with no internet No Wins
High steady volume Bills grow with use Wins after break-even
No usage limits or rate limits Windows, caps, queues at busy times Wins: only your speed
Same model for years, no surprise changes Models retire and change Wins: your files stay the same
Upkeep Wins (none) Updates, drivers, heat

A frontier model is one of the strongest models available at the time, run only by the company that made it.

3. A decision tree

  1. Is the data allowed to leave? No: local (or a rented machine in an approved region, Lesson 15.7). Yes: go on.
  2. Do you need to work offline? Yes: local.
  3. Does a local model do the job well enough? Test it on 10 real examples (Lesson 12.7 shows how). Yes: go on. No: cloud for this job.
  4. Is the volume high and steady? Yes: local, after checking break-even. No: cloud is fine.
  5. Still unsure? Use both (Lesson 16.6).

Four jobs, one tree. An engineering consultancy lists its AI jobs: drafting emails, summarising client drawings, checking long public technical standards, and a yearly market report.

Question 1: may the data leave? Client drawings and reports: no. That job goes local straight away.

Question 3: is local good enough? Emails: yes, in a 10-example test. Long standards: no, they need a very long context and the best reasoning, so that job goes to the cloud.

Question 4: volume. Emails are daily and steady, so local. The market report is once a year and needs web search: cloud.

Result. Two jobs local, two cloud. The firm buys one office machine and keeps two cloud seats instead of twelve.

4. The volume question in numbers

For a job that may go either way, the cloud cost rises with every job while the machine costs about the same each month. So there is a volume where they meet.

J∗=Lc
L
what the local machine costs per month in all: hardware share (Lesson 16.3) plus power
c
what one job costs on the API: tokens in × input price + tokens out × output price
J*
the jobs per month where the two cost the same
Physical meaning
Below J* the machine would sit idle most of the month and the cloud is cheaper; above it, every extra job on the machine is almost free. It is the same choice as renting a car for the odd trip or owning one you drive every day.
Maths and principle

Principle: choose by the job, not by the tool.

Equation: best home = the cheapest option that (a) may see the data and (b) passes your quality test.

Analogy: printing at home against a print shop. Daily letters at home; the wedding invitations at the shop.

Try it: a translator with 200 public web pages a month and 20 confidential contracts. Where does each go? (Pages: either, choose by cost; contracts: local.)

“Local is always cheaper, because the tokens are free.” The tokens are free; the machine is not. For small or bursty work the hardware sits idle, and a cheap API model can cost a few dollars a month. Local wins on money only above the break-even volume, and wins on privacy at any volume.

5. Worked example

Made-up firm. An engineering consultancy lists its AI jobs: drafting emails (local, fine quality), summarising client drawings and reports (local, data rule), checking long technical standards that are public (cloud, needs the long context and best reasoning), and a yearly market report (cloud, needs web search). Half the jobs, and nearly all the volume, go local, so they buy one office machine and keep two cloud seats instead of twelve.

6. Practice

  1. Put your five most common AI jobs through the decision tree.
  2. Which row of the scorecard matters most in your work?
  3. Why does “models retire and change” matter for a business process?
Answers
  1. Your answer; write down why for each.
  2. For many people it is privacy or limits; for some it is the strongest model.
  3. A process tested on one model can behave differently when the provider swaps it; a local file stays identical until you change it.

New words: frontier model, decision tree, rate limit.

  • The cloud wins on the strongest models, zero upfront cost, short bursts and built-in extras.
  • Local wins on privacy, offline use, steady high volume, no usage limits and a model that never changes under you.
  • Decide per job: may the data leave? must it work offline? is local good enough? is the volume high?
  • The break-even volume is the local monthly cost ÷ the API cost of one job.
  • Most people end up using both (Lesson 16.6).

Check yourself

1A student needs AI for two weeks of exam revision, then rarely. Which wins?
2A clinic must summarise patient notes every day. Which wins?
3What is the strongest reason the cloud still wins for some experts?

Sources

Your ticks stay in this browser. Save them as a file on the course home.

Updated 2026-11-22. Prices and model names change fast: check the official source before you buy.