Module 2: How a language model works · Lesson 2.2 · about 20 min
Parameters: what "7B" and "120B" mean
A model's parameters are the numbers it learned. What the B in 12B means, how parameters turn into gigabytes, and why more is usually smarter but not always.
Module 2 · about 20 minutes · read first: 2.1. You’ll be able to read a model’s size, turn it into gigabytes and judge what it says about ability.
1. Why this matters
The number in a model’s name is the first thing to read. It tells you roughly three things: how capable the model is, how much memory it needs and how fast it will run.
2. What a parameter is
A parameter (also called a weight) is one number the model learned during training. The scores in Lesson 2.1 come from multiplying your tokens by these numbers, layer after layer. A model is, quite literally, a very large file of numbers plus the recipe for using them.
Picture a mixing desk in a recording studio with billions of tiny knobs. Training turns each knob until the music (the predictions) sounds right. Each knob’s final position is one parameter.
- B = billion. E2B and E4B in Gemma 4 mean “effectively” 2 or 4 billion working at a time, with some extra stored parts.
- For Mixture-of-Experts models the name may give two numbers: 26B A4B means 26 billion in total, about 4 billion active per token (Lesson 2.5).
3. From parameters to gigabytes
Bytes per parameter depend on precision: 16 bits = 2 bytes, 8 bits = 1 byte, 4 bits = half a byte. So:
| Model | 16-bit | 8-bit | 4-bit |
|---|---|---|---|
| 4B | 8 GB | 4 GB | 2 GB |
| 12B | 24 GB | 12 GB | 6 GB |
| 31B | 62 GB | 31 GB | about 16 GB |
| 120B | 240 GB | 120 GB | 60 GB |
Real files are a little bigger, because some parts are kept at higher precision.
Watch the 12B row being worked out.
Count the knobs. A 12B model has 12 billion parameters.
16 bits each. 16 bits = 2 bytes. 12 billion × 2 bytes = 24 GB. Too big for a 16 GB laptop.
8 bits each. 8 bits = 1 byte. 12 × 1 = 12 GB. Fits, but leaves little room.
4 bits each. Half a byte. 12 × 0.5 = 6 GB. Comfortable on a 16 GB laptop. Storing each number less precisely is called quantisation (Module 8); quality drops only a little.
4. Size and ability
More parameters can store more patterns, so bigger models usually reason better, know more facts and handle more languages. But data quality and training method matter as much. Small models improve every year, and a new 12B model often beats a 70B model from a few years ago. Test on your own tasks (Lesson 7.4).
“A 70B model is always smarter than a 12B one.” Size is only one ingredient. How much text the model was trained on, how clean it was and how it was taught to follow instructions matter just as much. Research on training (the “Chinchilla” paper in the sources) showed many big models were under-trained for their size. A newer, well-trained small model often wins, and it runs faster too.
Maths and principle
Principle: memory for weights is parameters times bytes per parameter.
Equation: GB = parameters (billions) × bits ÷ 8.
Analogy: a library’s size is books × pages per book. Parameters are the books; precision is how many pages each takes up.
Try it: a 27B model at 8 bits? (27 GB.)
- P
- the number of parameters, in billions (the "B" in the name)
- bits
- how many bits store each parameter: 16, 8 or 4 are common
- ÷ 8
- turns bits into bytes, since 8 bits make 1 byte
- GB = P × bits ÷ 8
- the size of the weights in gigabytes
- Physical meaning
- Every parameter has to sit in memory while the model works, so this number is the floor on how much memory you need. Halve the bits and you halve the memory, which is how a model that needs a data-centre card at 16 bits ends up running on a laptop at 4 bits.
5. Worked example
Made-up numbers. Two models score the same on Pooja’s five real tasks: a 12B and a 27B. The 12B needs 7.7 GB and writes at 18 tokens per second on her machine; the 27B needs 18 GB and writes at 7. She keeps the 12B.
6. Practice
- How many GB is a 20B model at 4 bits?
- What does “60B A6B” tell you?
- Why might you choose a 12B over a 27B?
Answers
- 20 × 4 ÷ 8 = 10 GB.
- 60 billion parameters in total, 6 billion active per token: memory like a 60B, speed like a 6B.
- It fits more easily, runs faster, and may be good enough for your tasks.
New words: parameter or weight (one learned number), B (billion), active parameters (those used per token in an MoE model), precision (bits per number).
- A parameter (weight) is one number the model learned; B in a model’s name means billion parameters.
- Weights in GB = parameters (billions) × bits per parameter ÷ 8: a 12B model is 24 GB at 16 bits, 6 GB at 4 bits.
- “26B A4B” means 26 billion in total (sets memory) and about 4 billion active per token (sets speed).
- Bigger usually means more capable, but data and training matter as much; newer small models often beat older big ones.
- Pick the smallest model that does your tasks well: it fits more easily and runs faster.
Check yourself
Sources
- Kaplan, J. et al. (2020), Scaling Laws for Neural Language Models https://arxiv.org/abs/2001.08361
- Hoffmann, J. et al. (2022), Training Compute-Optimal Large Language Models (the "Chinchilla" paper: data matters as much as size) https://arxiv.org/abs/2203.15556
- Hugging Face Hub documentation, model cards (where parameter counts and file sizes are listed) https://huggingface.co/docs/hub/model-cards
Your ticks stay in this browser. Save them as a file on the course home.
Updated 2026-11-22. Prices and model names change fast: check the official source before you buy.