All lessons

Module 2: Mathematical methods · Lesson 2.3 · about 50 min

Random variables and probability distributions

Random variables, mean, variance and moments; the binomial, Poisson, Gaussian, uniform, exponential and Cauchy distributions and the limits that connect them; counting statistics, random walks and the central limit theorem; changing variables, characteristic functions and cumulants; Bayes' theorem and covariance.

Key points, meanings and formulas on one page: choose "Save as PDF" in the print box.

Module 2 · about 50 minutes. By the end you will be able to compute means, variances and probabilities for the standard distributions, know when one turns into another, handle counting statistics and random walks, change variables in a probability density, and use characteristic functions and Bayes’ theorem. NET puts one or two of these in Part C and many more in Part A.

Physics is full of randomness that follows strict rules: decays, photon counts, thermal motion, measurement errors. A handful of distributions, and the two limits that join them (Poisson and Gaussian), describe nearly all of it.

1. Random variables, mean and variance

A random variable X takes values with probabilities P(xi) (discrete) or a density p(x) (continuous), normalised to 1. Averages are weighted sums:

⟨f(X)⟩=∑if(xi)P(xi)or∫f(x)p(x)dx,σ2=⟨X2⟩−⟨X⟩2

Rules that save time: ⟨aX+b⟩=a⟨X⟩+b; Var(aX+b)=a2VarX; for independent variables ⟨XY⟩=⟨X⟩⟨Y⟩ and variances add, for sums and differences alike. The covariance ⟨XY⟩−⟨X⟩⟨Y⟩ is zero for independent variables, but zero covariance does not prove independence (take Y=X2 with X symmetric about 0).

Worked example. Uniform on [0,a]: ⟨x⟩=a/2, ⟨x2⟩=a2/3, so σ2=a2/12.

2. The standard distributions

Distribution Formula Mean Variance Where it appears
binomial (nk)pk(1−p)n−k np np(1−p) n yes/no trials
Poisson λke−λ/k! λ λ counts of rare independent events
Gaussian 12πσe−(x−μ)2/2σ2 μ σ2 sums of many small effects
exponential λe−λt 1/λ 1/λ2 waiting times, lifetimes
uniform 1/(b−a) (a+b)/2 (b−a)2/12 random phases
Cauchy γ/πx2+γ2 undefined infinite natural line shapes
λ
the mean number of events in the interval (Poisson), or the rate (exponential)
σ
the standard deviation: the typical spread about the mean
√N
the counting uncertainty: N counts carry an uncertainty √N, a fraction 1/√N
FWHM
full width at half maximum: 2√(2 ln 2) σ ≈ 2.355σ for a Gaussian, 2γ for a Lorentzian
Physical meaning
The Poisson law is what you get when many independent nuclei or photons each have a tiny chance to register. Its spread equals its mean, which sets the shot noise of every counting experiment.

The two limits. For large n and small p with np=λ fixed, the binomial becomes Poisson. For large mean (np(1−p)≳10, or λ≳10), both become Gaussian with the same mean and variance. The Gaussian gives the familiar 68%, 95.5% and 99.7% within 1σ, 2σ, 3σ.

Bars: the binomial distribution. Dots: the Poisson distribution with the same mean. Curve: the Gaussian with the same mean and variance. Make p small and n large to see the Poisson limit; make np(1 − p) large to see everything turn Gaussian.

Worked example. 1000 nuclei, each decaying in a minute with probability 0.002. With λ=2: P(0)=e−2=0.135, P(1)=2e−2=0.271, P(≥2)=1−3e−2=0.594. The exact binomial differs only in the fourth decimal place.

3. Counting statistics and the random walk

The waiting time between events of a Poisson process is exponential, and it has no memory: P(T>s+t|T>s)=e−λt. That is why a radioactive nucleus has a half-life independent of its age.

A random walk of N independent steps of length l, each equally likely forward or back, has

⟨x⟩=0,⟨x2⟩=Nl2,xrms=lN

With a bias (probability p forward, q back) the mean drifts as N(p−q)l and the variance is 4Npql2. Since N∝t, ⟨x2⟩∝t: this is diffusion. The probability of ending at x=m (in units of l) is binomial, (N(N+m)/2)2−N for the unbiased walk.

The central limit theorem. The sum of N independent, identically distributed variables with finite variance σ2 tends to a Gaussian with variance Nσ2; their mean has standard deviation σ/N. It fails for the Cauchy distribution (no finite variance).

Worked example. Single readings scatter with σ=2 mm. To get the mean to ±0.2 mm you need N=(2/0.2)2=100 readings.

4. Changing variables, moments and characteristic functions

If y=g(x) is one-to-one, probability is conserved in matching intervals: pY(y)|dy|=pX(x)|dx|, so

pY(y)=pX(x)|dxdy|

summed over every x that gives the same y. Example: X uniform on (0,1) and Y=−ln⁡X gives pY=e−y. For y=x2 with x Gaussian, both roots ±y count, giving the χ2 density with one degree of freedom.

The characteristic function ϕ(k)=⟨eikX⟩ is the Fourier transform of the density. Its Taylor coefficients are the moments, and those of ln⁡ϕ are the cumulants κn (κ1 = mean, κ2 = variance). For independent variables, cumulants add.

Distribution ϕ(k) Cumulants
Gaussian eikμ−σ2k2/2 only κ1=μ, κ2=σ2
Poisson exp⁡[λ(eik−1)] all κn=λ
Cauchy e−γ|k| none defined

Gaussian moments are worth memorising: ⟨x2n⟩=(2n−1)!!σ2n for zero mean, so ⟨x4⟩=3σ4, and ⟨|x|⟩=σ2/π.

5. Conditional probability and Bayes’ theorem

P(A|B)=P(B|A)P(A)P(B),P(B)=P(B|A)P(A)+P(B|A‾)P(A‾)

Worked example. A detector flags a rare event type (1% of events) with probability 0.99, and wrongly flags 1% of ordinary events. Then P(true|flag)=0.99×0.010.99×0.01+0.01×0.99=0.5. Half the flags are false, because the background is 99 times more common.

“Variances subtract for a difference.” They never do: Var(X−Y)=VarX+VarY for independent X, Y, which is why background subtraction makes a signal noisier. A second classic slip: the most probable value, the mean and the median are different numbers for skewed distributions (Maxwell speeds: vp<⟨v⟩<vrms), and a Poisson count of mean 3 has only a 22% chance of being exactly 3.

  • σ2=⟨x2⟩−⟨x⟩2; Var(aX+b)=a2σ2; independent variances add for sums and differences.
  • Binomial np, np(1−p); Poisson λ, λ; exponential 1/λ, 1/λ2; uniform (b−a)2/12; Cauchy has no variance.
  • Binomial → Poisson (small p, np fixed) → Gaussian (large mean). 68 / 95.5 / 99.7% within 1, 2, 3 σ; FWHM = 2.355σ.
  • Counting: uncertainty N. Random walk: xrms=lN. Mean of N readings: σ/N.
  • Change of variables: pY=pX|dx/dy|, summed over all roots.
  • ϕ(k)=⟨eikx⟩; Gaussian cumulants stop at 2; Poisson cumulants all equal λ.
  • Bayes: posterior ∝ likelihood × prior; rare events need very low false-alarm rates.

Practise: exam questions on this lesson

They start easy and climb to top-rank level. More on the practice page.

Before this: lesson 2.2, group theory. Next: lesson 2.4, numerical methods. The same distributions return in statistical physics: see the GATE lesson on ensembles and the kinetic theory lesson for the Maxwell distribution.

Check yourself

1A counter records 900 counts in a fixed time. The fractional uncertainty of this count is about
2A random walker takes 400 independent unit steps, left or right with equal probability. Its rms distance from the start is
3Which distribution has no finite variance, so that the central limit theorem does not apply to it?

Sources

  • M. L. Boas, Mathematical Methods in the Physical Sciences, 3rd ed. (2006), chapter 15
  • F. Reif, Fundamentals of Statistical and Thermal Physics (1965), chapter 1
  • P. R. Bevington and D. K. Robinson, Data Reduction and Error Analysis for the Physical Sciences, 3rd ed. (2003), chapters 2 and 3
  • CSIR-UGC NET Physical Sciences syllabus, Mathematical Methods of Physics (probability and random variables)

Your ticks stay in this browser. Save them as a file on the course home.

Updated 2026-11-22. Not affiliated with CSIR, UGC or the National Testing Agency. Check the official CSIR NET website for the current pattern. All questions are original.