Home / Learn / Data & AI / Offline AI / Lesson 9.4
All lessons

Module 9: Installing your first local AI · Lesson 9.4 · about 20 min

MLX on a Mac

Apple's MLX runs models fast on Apple Silicon Macs. Install it with one command, run a model, and start a local server.

Key points, meanings and formulas on one page: choose "Save as PDF" in the print box.

Module 9 · about 20 minutes · read first: 9.2. Mac only; on a PC, skip to 9.5. You’ll be able to run a model with MLX and serve it to other programs.

On an Apple Silicon Mac the processor and graphics share one big memory, so about three quarters of it can hold a model, and the memory's read speed sets how fast that model writes.

1. Why this matters

On an Apple Silicon Mac (M1 or later), MLX often runs models faster than other engines because it is written for Apple’s chips. Desktop apps can use it for you; using it directly lets you script it and pick from thousands of ready-converted models.

2. Unified memory in one paragraph

On a PC, the graphics card has its own memory (often 8 to 16 GB) separate from the main memory. On an Apple Silicon Mac, processor and graphics share one pool: unified memory. So a 32 GB Mac can put a 19 GB model where the graphics side reaches it at full speed. The catch: macOS keeps some for itself, so plan on using about three quarters of the total.

Watch it happen: will it fit, and how fast?

The Mac. 32 GB of unified memory that can be read at 170 GB per second (Apple lists this "memory bandwidth" for each chip).

Room for models. About three quarters: 0.75 × 32 = 24 GB. macOS and your apps keep the rest.

Pick a model. A 16 GB 4-bit model fits, with 8 GB left for the conversation. A 30 GB one does not.

Top speed. Each token reads the whole model once: 170 ÷ 16 ≈ 10.6 tokens per second at best. Real speed is often 60% to 80% of that.

Smaller is faster. A 7.7 GB model on the same Mac: 170 ÷ 7.7 ≈ 22 tokens per second. Half the size, about twice the speed.

3. Install and run

You need Python 3 (macOS offers to install it the first time you type python3). A virtual environment is a private folder of Python packages for one project, so nothing clashes with the rest of your Mac. Then:

python3 -m venv ~/mlx-env
source ~/mlx-env/bin/activate
pip install mlx-lm

Run a model (it downloads on first use, then works offline):

mlx_lm.generate --model mlx-community/<model-name> --prompt "Explain unified memory in two sentences."

The mlx-community group on Hugging Face publishes converted models; search there for “gemma 4” and pick a 4-bit version that fits, copying its exact name. For a back-and-forth chat use mlx_lm.chat --model ..., and for a local server that programs can call (Module 11):

mlx_lm.server --model mlx-community/<model-name> --port 8080
Maths and principle

Principle: what a Mac can run is set by about three quarters of its memory; how fast is set by its memory bandwidth.

Equation: largest model ≈ 0.75 × memory; top speed ≈ bandwidth ÷ model size.

Smax⁡≈0.75×M,top speed≈BS

Analogy: a shared kitchen. Everyone cooks from one big fridge, so nobody waits for food to be carried from another room, but the house still keeps a shelf for itself.

Try it: a 32 GB Mac with 170 GB/s and a 16 GB model? (Fits: 0.75 × 32 = 24 GB. Top speed about 170 ÷ 16 ≈ 10 tokens per second, before MoE speed-ups.)

M
the Mac's unified memory in GB
Smax
the largest total (model plus conversation) you should plan for, about three quarters of M
S
the model file's size in GB
B
memory bandwidth: how many GB per second the chip can read
Physical meaning
Memory size decides which models a Mac can run; bandwidth decides how fast. Two Macs with the same memory can differ two or three times in speed if their chips read memory at different rates.

“A Mac with 64 GB can run a 64 GB model.” macOS needs memory for itself and your apps, and the conversation needs its cache. Plan on three quarters (48 GB), minus room for the context.

4. Worked example

Made-up numbers. Lena has a 24 GB MacBook. Three quarters is 18 GB, so a 19 GB model is out and a 7.7 GB 12B model is easy. With MLX it writes about 20 tokens per second; she starts mlx_lm.server each morning and points her writing scripts at it.

5. Practice

  1. Make the virtual environment and install mlx-lm.
  2. Run one 4-bit model with mlx_lm.generate, then again with Wi-Fi off.
  3. What is the largest model file a 48 GB Mac should run?
Answers
  1. If pip is not found, make sure the environment is active (your prompt shows (mlx-env)).
  2. The second run works offline because the model is already downloaded.
  3. About 0.75 × 48 = 36 GB, minus room for the conversation: a file of about 30 GB.

New words: unified memory (one memory pool shared by processor and graphics), virtual environment (a private folder of Python packages for one project), Apple Silicon (Apple’s M-series chips), memory bandwidth (how fast memory can be read).

  • MLX is Apple’s engine for M-series Macs; pip install mlx-lm brings generate, chat and server commands.
  • Unified memory lets the graphics side use most of the Mac’s memory; plan on about three quarters.
  • Top speed ≈ memory bandwidth ÷ model size: half the size, about twice the speed.
  • mlx-community on Hugging Face has ready-converted 4-bit models; once downloaded they run offline.

Check yourself

1Why do Macs run local models well for their price?
2Which command installs the MLX language-model tools?
3Does MLX run on a Windows PC?

Sources

Your ticks stay in this browser. Save them as a file on the course home.

Updated 2026-11-22. Prices and model names change fast: check the official source before you buy.