Home / Learn / Data & AI / Offline AI / Lesson 4.4
All lessons

Module 4: The parts that matter for AI · Lesson 4.4 · about 15 min

Storage, RAM, power supply and cooling

The supporting parts of a local AI machine. How much disk space models take, how much main memory you need, how big a power supply, and why cooling matters.

Key points, meanings and formulas on one page: choose "Save as PDF" in the print box.

Module 4 · about 15 minutes · read first: 4.1. You’ll be able to spec the rest of the machine around the memory.

Memory decides what fits and bandwidth how fast it writes; the disk, main memory, power supply and cooling decide whether the machine is pleasant and stays up under load.

1. Why this matters

The other parts decide whether the machine is nice to use: models that load in seconds, a PC that does not shut down under load, and fans you can live with. Think of a good chef. A fast knife (the graphics card) matters, but so do a big fridge (disk), a clear counter (main memory) and a strong gas line (power).

2. Storage

  • Model files are big: Gemma 4 E2B about 4.6 GB, Gemma 4 12B about 7.7 GB, Qwen3.8-27B about 18 GB, gpt-oss-120b about 65 GB.
  • You will keep several. Plan 100 to 500 GB free on an NVMe SSD (a fast solid-state disk) for models.
  • A fast SSD loads models in seconds. A hard disk works but takes minutes. Once loaded, the disk does not affect writing speed.
  • Macs charge a lot for bigger internal storage; a fast external USB-C or Thunderbolt SSD is fine for model files.

Load time is simply size ÷ read speed:

load time (s)=file size (GB)disk read speed (GB/s)

“A faster SSD makes the model write faster.” The disk is only used to load the model into memory at the start. After that, every token is read from memory, not from disk. A faster SSD shortens the wait before the first answer, nothing more.

3. Main memory (RAM) on a PC with a graphics card

Even when the model runs on the card, it is loaded through main memory, and your other apps need room too. Rule of thumb: main memory at least equal to graphics memory, ideally double. A 16 GB card wants 32 GB of main memory.

4. Power supply

A graphics card’s rated power (board power) is roughly what it draws at full load: about 180 W for an RTX 5060 Ti, 350 W for an RTX 3090 and 575 W for an RTX 5090. Add 100 to 200 W for the rest of the PC, then headroom (spare capacity, so the supply is never at its limit). Use the card maker’s recommended power supply size and the right power cables. Module 6 turns watts into money.

Here is how to size one, for a used RTX 3090:

Find the card's board power. The RTX 3090 is rated at about 350 W.
Add the rest of the PC. Processor, disks, fans and memory draw up to about 150 W. Peak: 350 + 150 = 500 W.
Add headroom. Graphics cards draw short spikes above their rating, and supplies run cooler and quieter below their limit. 500 × 1.3 = 650 W.
Round up and check. Supplies come in steps (550, 650, 750, 850 W). Choose 750 W, and check that it meets the card maker's own recommendation and has the right power plugs.
power supply≥(card+rest of PC)×1.3
Board power
The most the graphics card is designed to draw, in watts.
Rest of PC
Processor, memory, disks and fans together, usually 100 to 200 W.
× 1.3
Headroom: at least 30% spare, for spikes and a cooler, quieter supply.
Physical meaning
The power supply is the pipe that feeds the whole machine. If it is too narrow, the moment the model starts writing and the card pulls hard, the PC can simply switch off.

5. Cooling

Language models load the graphics chip in bursts: hard while writing, idle while you read. Long jobs (batch summaries, embeddings) load it for hours. A case with good airflow keeps the chip from slowing itself down (Lesson 6.4) and keeps fans quieter.

Maths and principle

Principle: size the supporting parts for the model, with headroom: disk for several models, main memory at least the card’s memory, power supply above the peak.

Equation: power supply ≥ (card power + rest of PC) × 1.3 or more.

Analogy: a good chef needs a big fridge (disk), a clear counter (main memory) and a strong gas line (power), not only a fast knife.

Try it: a 350 W card in a PC whose other parts draw 150 W? ((350 + 150) × 1.3 = 650 W, so a 750 W supply is a comfortable choice.)

6. Worked example

Made-up numbers. Omar adds an RTX 5060 Ti 16 GB to an office PC with a 300 W supply and 16 GB of main memory. The card’s maker asks for 550 W. He budgets for a 650 W supply and a second 16 GB memory stick. Without the supply upgrade the PC switches off when the model starts writing.

7. Practice

  1. How much free SSD space do you have now?
  2. Your PC has 16 GB main memory and you buy a 24 GB card. What should you add?
  3. Does a faster SSD make tokens come out faster?
Answers
  1. If it is under 100 GB, plan an external SSD for models.
  2. At least another 16 GB of main memory (to 32 GB or more), and check the power supply.
  3. No. It only shortens loading time.

New words: NVMe SSD (fast solid-state disk), board power (a card’s rated draw), headroom.

  • Keep 100 to 500 GB of fast SSD space for models; the disk only affects loading time, not writing speed.
  • Main memory should be at least as large as the graphics card’s memory, ideally double.
  • Power supply ≥ (card + rest of PC) × 1.3, rounded up to the next size, and at least the card maker’s recommendation.
  • Good airflow stops the chip slowing itself down and keeps fans quieter.

Check yourself

1You plan to keep five models of 5 to 20 GB each. What disk space should you set aside?
2Why does a fast SSD help?
3A graphics card is rated at 180 W. Roughly what power supply should the PC have?

Sources

Your ticks stay in this browser. Save them as a file on the course home.

Updated 2026-11-22. Prices and model names change fast: check the official source before you buy.