Module 4: Classification and logistic regression · Lesson 4.2 · about 25 min
The sigmoid and odds: from a line to a probability
Logistic regression keeps the straight-line score of a linear model and squeezes it through the S-shaped sigmoid into a probability between 0 and 1. Shape the curve with two sliders and learn to read weights as changes in odds.
Module 4 · about 25 minutes · read first: 1.5, 3.1, 4.1. You’ll be able to compute the sigmoid, convert between probability, odds and log-odds, and interpret a logistic regression weight.
1. Why this matters
Logistic regression is the everyday workhorse of yes/no prediction in medicine, finance and marketing. It is also the simplest neural network: one “neuron” with an S-shaped curve. Its output is a real probability, its weights can be explained in plain words, and the same S-curve comes back in Lessons 9.2 and 10.4.
2. The model
Start with the straight-line score from Module 3. Each input gets a weight, and a bias is added:
Here is the list of inputs, their weights (how much each counts), the bias (the starting point), and the score. can be any number at all, from very negative to very positive. To turn it into a probability, squeeze it through the sigmoid (also called the logistic function):
is the base of natural logarithms. You don’t need to love it: just watch what the formula does.
- Big positive : is tiny, so is close to 1.
- : , so .
- Big negative : is huge, so is close to 0.
The weight sets how steep the S is: a big weight means a small change in x swings the probability fast. The bias slides the S sideways. The 50 % point, where , sits at .
Inputs to a score. A pass/fail model uses hours studied: z = 0.7 × hours − 4.2. A student who studied 8 hours gets z = 5.6 − 4.2 = 1.4.
Flip the sign and raise e to it. e−1.4 ≈ 0.247.
Add 1 and take one over it. 1 ÷ (1 + 0.247) ≈ 0.80. So the model gives this student an 80 % chance of passing.
Read it as odds. 0.80 ÷ 0.20 = 4: "4 to 1 on". That is just ez = e1.4 ≈ 4.06, rounded.
Two more hours. z rises by 1.4 to 2.8, so the odds are multiplied by e1.4 ≈ 4 again: about 16 to 1, a 94 % chance.
3. Odds and log-odds
Gamblers use odds: the chance something happens divided by the chance it doesn’t.
is odds 1 (“evens”). is odds 4 (“4 to 1 on”). is odds 0.25 (“4 to 1 against”). Take the natural log (ln) of the odds and you get the log-odds, also called the logit. It runs from minus infinity to plus infinity, just like the score . That is no accident. Solving for gives
- z
- The score: the weighted sum of the evidence. It is also the log-odds of a "yes".
- σ(z)
- The sigmoid: turns any score into a probability between 0 and 1.
- p ÷ (1 − p)
- The odds: how many times more likely "yes" is than "no".
- ew
- The odds ratio: what one more unit of an input multiplies the odds by.
- Physical meaning
- Evidence adds up on the log-odds scale, like weights on a balance. The sigmoid is the pointer that shows how far the balance has tipped, and it can never go past fully "yes" or fully "no".
So logistic regression is simply a straight-line model for the log-odds. That gives its weights a clear reading: “one more unit of adds to the log-odds”, which is the same as “multiplies the odds by ”, whatever the other inputs are. That number is the odds ratio, the way doctors and economists report these models.
“A weight of 0.7 means each extra hour adds 0.7, or 70 %, to the chance of passing.” No. The weight adds 0.7 to the log-odds, which multiplies the odds by about 2. How much the probability moves depends on where you start: from 50 % it jumps to about 67 %, but from 95 % it can only creep up to about 97 %.
Maths and principle
Principle: the sigmoid maps any score to a probability, and its inverse, the logit, maps a probability back to a score. The model is a straight line on the logit scale.
Equations:
The last one is the sigmoid’s slope (how fast it rises). It is largest, 0.25, at and tiny far out. That matters in Lesson 4.3 and again for neural networks (Lesson 9.2).
Also : the curve is symmetric about the 50 % point.
Analogy: a dimmer switch with soft ends. In the middle, a small turn changes the light a lot; near fully on or fully off, turning further barely matters.
Try it: , . Probability at ? At ? (, so 0.5. , so .)
4. Worked example
Made-up numbers. A model of whether a student passes, from hours studied : .
| hours | z | odds | p |
|---|---|---|---|
| 2 | −2.8 | 0.061 | 0.06 |
| 4 | −1.4 | 0.247 | 0.20 |
| 6 | 0 | 1 | 0.50 |
| 8 | 1.4 | 4.06 | 0.80 |
| 10 | 2.8 | 16.4 | 0.94 |
Every 2 extra hours adds 1.4 to and multiplies the odds by . But the probability gain is uneven: from 4 to 6 hours it goes up 30 points, from 8 to 10 only 14. Effects are steady on the odds scale, not on the probability scale.
5. Practice
- Convert to odds and log-odds.
- . What is ? (.)
- A medical model has an odds ratio of 1.5 for smokers. What does that mean?
Answers
- Odds 0.25 / 0.75 = 1/3; log-odds ln(1/3) ≈ −1.10.
- .
- With everything else the same, smokers’ odds of the outcome are 1.5 times non-smokers’ odds (not 1.5 times the probability).
- Logistic regression computes a straight-line score z = w·x + b, then p = σ(z) = 1 / (1 + e−z).
- σ(0) = 0.5; big positive scores give p near 1, big negative scores p near 0.
- Odds are p ÷ (1 − p); the score z is the log of the odds.
- Each unit of an input multiplies the odds by ew, the odds ratio; the change in probability depends on the starting point.
Try it on your own data: the regression workbench fits logistic regression with odds ratios and their confidence intervals.
New words: logistic regression, sigmoid, odds, log-odds or logit, odds ratio.
Check yourself
Sources
- James, Witten, Hastie and Tibshirani (2021), An Introduction to Statistical Learning, 2nd edition, Springer, Section 4.3 (Logistic regression) https://www.statlearning.com/
- Bishop (2006), Pattern Recognition and Machine Learning, Springer, Section 4.3.2 (Logistic regression)
- scikit-learn user guide: Logistic regression https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
Your ticks stay in this browser. Save them as a file on the course home.
Updated 2026-11-22. Numbers in the examples are made up to show the idea unless a source is given. Simulations run in your browser; nothing you type is sent anywhere.