Module 1: Data, models and the maths you need · Lesson 1.4 · about 20 min
Vectors and the dot product
A vector is a list of numbers, such as one row of a table. The dot product multiplies two lists item by item and adds the results. It powers linear models, neurons and the attention inside language models.
Module 1 · about 20 minutes · read first: 1.3. You’ll be able to add and scale vectors, compute a dot product two ways, measure the length of a vector, and write a linear model in one line.
1. Why this matters
A row of a data table is a list of numbers. Machine learning treats that list as one object, a vector, and the most important thing it does with vectors is the dot product. A linear model is a dot product plus a number. A neuron is a dot product passed through a bend. Attention in a language model is a table of dot products between words. Learn this one operation well and you hold the main tool of the whole subject.
2. A vector is a list of numbers
A home with floor area 95 m², 3 bedrooms and age 40 years is the vector
Each number is a component (or dimension), and vectors are written in bold. With 2 components you can draw a vector as an arrow from (0, 0) to the point, like directions on a map: “3 blocks east, 4 blocks north”. With 3 you need a 3-D picture, and with 300 you can’t draw it at all, but the arithmetic is exactly the same. That is the beauty of vectors: picture ideas in 2-D, then trust them in any number of dimensions.
Two simple operations:
- Add vectors component by component: . As arrows, walk the first, then the second.
- Scale by a number: . The arrow keeps its direction and doubles in length.
3. The dot product
Multiply matching components, then add:
The answer is one number, not a vector. You already do this at the shop till:
Now drag the arrows:
4. What the dot product means
The same number can be found from lengths and the angle between the arrows:
Here is the length of and (theta) is the angle between the arrows. Since is 1 at 0°, 0 at 90° and −1 at 180°:
- Large positive: the arrows point the same way. They “agree”.
- Zero: at right angles. They share nothing.
- Negative: they point apart. They “disagree”.
This is why the dot product measures similarity. In Lesson 10.2, words become vectors, and words with similar meanings have arrows pointing the same way, so their dot product is large.
5. Length and the angle
By Pythagoras, the length of a vector is
Rearranging the formula of section 4 gives the cosine similarity, a score from −1 to 1 that ignores length and keeps only direction:
“A bigger dot product always means more alike.” Only if the lengths are equal. A huge basket and a small basket can have a big dot product just because the huge one has big numbers. Cosine similarity divides out the lengths, so a family’s weekly shop and a single person’s can still score close to 1 if they buy things in the same proportions.
6. A linear model is a dot product
The many-input line from Lesson 1.3, , is just
The weight vector says how much each feature counts. Training finds and . Seen as arrows, the prediction is large when the example points in the direction the weights “like”.
- x
- The features of one example, as a list: (floor area, bedrooms, age).
- w
- The weights: how much each feature is "worth" in the prediction. A negative weight pulls the prediction down.
- w · x
- Multiply each feature by its weight and add: one number.
- b
- The bias: a fixed amount added at the end.
- cos θ
- How much two lists point the same way, from −1 (opposite) to 1 (same direction), ignoring size.
- Physical meaning
- The dot product is a weighted vote. Each feature casts a vote of its size times its importance, and the votes are added. A till adding up a bill, a judge totalling marks with different weights per round, and a neuron deciding whether to fire all do exactly this.
Maths and principle
Principle: the dot product turns two lists into one number that measures how much they point the same way, scaled by their lengths.
Equation: .
Analogy: a shopping bill. The vector of quantities (2 loaves, 3 litres of milk) dotted with the vector of prices ($3 a loaf, $2 a litre) gives the total: 2 × 3 + 3 × 2 = $12. A linear model is a bill where the weights are the prices of each feature.
Try it: , . Dot product? Lengths? Angle? (24; both 5; , so about 16°.)
7. Worked example
Made-up numbers. A simple house-price model has weights for (floor area in m², bedrooms, age in years) and bias , with prices in $000. For the home :
So the model predicts about $354,500. The negative weight on age says older homes sell for less, other things being equal. Computers do this for millions of rows at once by stacking rows into a matrix (Lesson 3.4).
8. Practice
- Compute .
- What is the length of ?
- Two word vectors have cosine similarity 0.95, and another pair has −0.1. Which pair is more alike in meaning?
Answers
- 4 + 0 − 3 = 1.
- .
- The first: 0.95 means the arrows point almost the same way.
Try it on your own data: the embedding visualiser places text as vectors and compares them with cosine similarity.
New words: vector, component or dimension, dot product, length or norm, cosine similarity, weight vector.
- A vector is a list of numbers, such as one row of a data table.
- Dot product: multiply matching items and add; the answer is one number.
- It also equals length × length × cos(angle), so it measures how much two vectors agree.
- Cosine similarity divides out the lengths and keeps only direction, from −1 to 1.
- A linear model is ŷ = w · x + b: a weighted vote of the features plus a fixed amount.
Check yourself
Sources
- Strang, G. (2016), Introduction to Linear Algebra, 5th ed., Wellesley-Cambridge Press, Chapter 1 (vectors, lengths and dot products)
- Goodfellow, I., Bengio, Y. and Courville, A. (2016), Deep Learning, MIT Press, Chapter 2 (linear algebra) https://www.deeplearningbook.org/
- Manning, C. D., Raghavan, P. and Schütze, H. (2008), Introduction to Information Retrieval, Cambridge University Press, Chapter 6 (cosine similarity)
Your ticks stay in this browser. Save them as a file on the course home.
Updated 2026-11-22. Numbers in the examples are made up to show the idea unless a source is given. Simulations run in your browser; nothing you type is sent anywhere.