Module 3: Tokens: the unit of everything · Lesson 3.4 · about 20 min
Input, output and thinking tokens
Tokens you send, tokens the model writes, and the hidden "thinking" tokens reasoning models add. Why they are counted separately and what each costs in time and money.
Module 3 · about 20 minutes · read first: 3.1 and 3.3. You’ll be able to tell input, output and thinking tokens apart and estimate the time each adds.
1. Why this matters
When a local job feels slow, or a cloud plan runs out early, the cause is usually one kind of token: long outputs, or a reasoning model thinking at length. Knowing which lets you fix the right thing.
2. Three kinds
- Input tokens: everything you send, the system prompt, history, documents and your question. Read in one fast batch.
- Output tokens: the answer, written one token at a time.
- Thinking (reasoning) tokens: some models write out their working first, often hidden or folded in the interface, then the final answer. They count as output: they take writing time and, in the cloud, are usually billed as output.
Think of a letter. Reading the letter you received takes a minute; writing your reply by hand takes much longer; and any rough draft you throw away still took writing time.
3. Why the difference matters
On your machine, input is cheap in time and output is expensive. In the cloud, output tokens usually cost several times more than input tokens, and plan limits drain fastest with long answers and heavy thinking. Either way, the biggest savings come from shorter outputs and thinking only when needed (Lessons 3.6 and 10.4).
“Thinking tokens are free, because I never see them.” They are hidden or folded away, but the model still writes every one at normal writing speed, and cloud APIs usually bill them as output. A reasoning model that thinks for 1,500 tokens before a 300-token answer has written six times as much as you read.
Maths and principle
Principle: total time is reading time plus writing time, and writing is usually the bigger part.
Equation: time ≈ input ÷ reading speed + (thinking + output) ÷ writing speed.
Analogy: a student in an exam: reading the question is quick, writing the essay is slow, and rough working in the margin adds writing time even though it is not marked.
Try it: 2,000 input tokens at 500 per second, 1,500 thinking and 300 answer tokens at 15 per second? (4 s + 120 s = about 2 minutes. Without thinking: 4 + 20 = 24 s.)
Here is that request, step by step.
Read the input. 2,000 tokens (instructions, a document and the question), all at once, at 500 per second: 2,000 ÷ 500 = 4 seconds.
Think. The model writes 1,500 tokens of working, one at a time, at 15 per second: 100 seconds. You may see only "Thinking…".
Answer. 300 tokens at 15 per second: 20 seconds.
Total. 4 + 100 + 20 = 124 seconds. Writing took 97% of the time.
Thinking off. 4 + 20 = 24 seconds, five times quicker. Worth it for easy questions; keep thinking on for hard ones.
- input
- tokens you send: system prompt, history, documents and the question
- reading speed
- how many input tokens per second the machine can process in one batch (prefill, Lesson 3.5)
- thinking + output
- every token the model writes, working and answer together
- writing speed
- how many tokens per second it writes, one at a time (decode)
- Physical meaning
- Writing speed is usually 10 to 100 times slower than reading speed, so the length of what the model writes decides how long you wait. To speed a job up, shorten the answer or switch thinking off before you shorten the input.
4. Worked example
Made-up numbers. A reasoning model takes 90 seconds to answer “Is 17 a prime number?” because it writes 1,200 thinking tokens first. Faisal switches thinking off for simple questions and on for maths problems. Simple questions now take 3 seconds.
5. Practice
- Ask your model for a 50-word and then a 500-word answer. Time both.
- Using the box, how long do 1,000 output tokens take at 20 per second?
- You hit a cloud plan’s limit every afternoon. Name two changes that save tokens.
Answers
- The 500-word answer takes about ten times as long; reading time hardly changes.
- 50 seconds.
- Ask for shorter answers; switch thinking off for easy questions; start new chats instead of very long ones; send only the relevant part of documents.
New words: input tokens, output tokens, thinking or reasoning tokens (working written before the final answer).
- Input tokens are everything you send; they are read in one fast batch.
- Output tokens are written one at a time, so they take most of the time and usually cost more in the cloud.
- Thinking tokens are working a reasoning model writes before its answer; they count as output.
- Time ≈ input ÷ reading speed + (thinking + output) ÷ writing speed.
- The biggest savings: shorter answers and thinking only when the question needs it.
Check yourself
Sources
- Wei, J. et al. (2022), Chain-of-Thought Prompting Elicits Reasoning in Large Language Models https://arxiv.org/abs/2201.11903
- Ollama API documentation (prompt_eval_count and eval_count: input and output tokens per request) https://github.com/ollama/ollama/blob/main/docs/api.md
- Jurafsky, D. and Martin, J. H., Speech and Language Processing, 3rd edition draft, chapter on large language models https://web.stanford.edu/~jurafsky/slp3/
Your ticks stay in this browser. Save them as a file on the course home.
Updated 2026-11-22. Prices and model names change fast: check the official source before you buy.