All lessons

Module 2: Mathematical methods · Lesson 2.4 · about 50 min

Numerical methods: roots, integration and ODEs

Bisection, Newton-Raphson, secant and fixed-point iteration with their orders of convergence and failure modes; interpolation and finite differences; trapezoidal, Simpson and Gauss quadrature with error laws and Richardson extrapolation; Euler, Runge-Kutta and symplectic methods for ODEs and their stability; solving linear systems.

Key points, meanings and formulas on one page: choose "Save as PDF" in the print box.

Module 2 · about 50 minutes. By the end you will be able to carry out one or two steps of each standard method by hand (which is what NET asks), state how fast each converges, predict how the error changes with the step size, and recognise when a method will fail or go unstable.

Every numerical method replaces the problem by one you can solve exactly: a curve by its tangent, an area by trapezoids or parabolas, a smooth motion by small straight steps. How well the replacement matches the Taylor series decides how fast the error shrinks.

1. Finding roots

Method Step Order Needs
bisection halve the bracket [a,b] with f(a)f(b)<0 1 (factor 1/2 per step) a sign change
fixed point xn+1=g(xn) 1 (factor |g′(x∗)|) |g′(x∗)|<1
secant xn+1=xn−fnxn−xn−1fn−fn−1 1.618 two starting points
Newton-Raphson xn+1=xn−f(xn)f′(xn) 2 f′

The order p means |en+1|≈C|en|p. For Newton, expanding about the root r:

en+1≈f′′(r)2f′(r)en2

so the number of correct digits roughly doubles each step. Bisection needs n>log2⁡(L/ϵ) steps to shrink a bracket of length L below ϵ: about 3.3 steps per decimal digit, but it never fails.

Worked example. 2 from f=x2−2, x0=1: Newton’s step is xn+1=12(xn+2/xn), giving 1.5, 1.41667, 1.4142157. Errors: 0.086, 0.0025, 2×10−6: squared each time.

When Newton fails. At a double root f′(r)=0 and convergence drops to linear (for f=x2, xn+1=xn/2). A bad start can cycle: f=x3−2x+2 from x0=0 goes 0,1,0,1,…. For f=x1/3, xn+1=−2xn: it runs away.

Root finding: the error of bisection, the secant method and Newton-Raphson for cos x = x on a log scale; Newton's curve plunges, bisection's falls in a straight line. Integration: the error of the trapezoidal and Simpson rules against the number of sub-intervals, with slopes −2 and −4 on log-log axes.

2. Interpolation and differences

Through n+1 points with distinct x there is exactly one polynomial of degree at most n. With equally spaced points, Newton’s forward-difference form is quickest by hand: p(x0+sh)=y0+sΔy0+s(s−1)2!Δ2y0+….

Finite differences approximate derivatives, with errors that follow from Taylor series:

f′(x)≈f(x+h)−f(x)h+O(h),f′(x)≈f(x+h)−f(x−h)2h+O(h2),f′′(x)≈f(x+h)−2f(x)+f(x−h)h2+O(h2)

A smaller h is not always better: round-off error grows like ϵ/h, so the best step for a central difference is about (3ϵ)1/3∼10−5 in double precision.

3. Numerical integration

With n sub-intervals of width h=(b−a)/n:

T=h2[f0+2(f1+⋯+fn−1)+fn],S=h3[f0+4f1+2f2+4f3+⋯+4fn−1+fn]
Trapezoidal error
−(b − a)h²f″/12: exact for straight lines, error ∝ h²
Simpson error
−(b − a)h⁴f⁽⁴⁾/180: exact up to cubics, error ∝ h⁴, needs even n
Gauss, n points
exact for polynomials up to degree 2n − 1 (two points ±1/√3 handle cubics)
Richardson
for error ∝ hp: I ≈ [2pA(h/2) − A(h)]/(2p − 1)
Physical meaning
The power of h tells you the payoff of more work: doubling the points cuts the trapezoidal error 4 times and Simpson's 16 times. For smooth functions, a higher-order rule is almost always the cheaper way to accuracy.

Worked example. ∫02x4dx=6.4. Simpson with h=1: 13(0+4+16)=6.667, error −0.267, exactly −(2)(1)(24)/180. With h=0.5 the error falls to 0.0167, sixteen times smaller.

Worked example: Richardson. Trapezoid for ∫01x2dx: T(0.5)=0.375, T(0.25)=0.34375. Then [4(0.34375)−0.375]/3=1/3, exact. Repeating this idea is Romberg integration.

4. Ordinary differential equations

For y′=f(x,y) with step h:

Method Step Local error Global error
explicit Euler yn+1=yn+hf(xn,yn) O(h2) O(h)
Heun / RK2 k1=f(xn,yn), k2=f(xn+h,yn+hk1), yn+1=yn+h2(k1+k2) O(h3) O(h2)
classical RK4 four slopes, weights 1, 2, 2, 1 over 6 O(h5) O(h4)

Worked example. y′=y, y(0)=1, one step h=0.1. Euler: 1.1. Heun: 1+0.05(1+1.1)=1.105. RK4: 1.1051708. Exact: e0.1=1.1051709.

Stability. For the test equation y′=−λy, Euler multiplies y by 1−λh each step, so it blows up unless h<2/λ, even though the true solution decays. A stiff system (one fast mode, slow interesting modes) forces tiny steps on explicit methods. Implicit (backward) Euler, yn+1=yn/(1+λh), is stable for every h but needs an equation solve per step for nonlinear f.

Long-time mechanics. Euler applied to the oscillator x˙=v, v˙=−x multiplies the energy by 1+h2 every step: orbits spiral outwards. Symplectic methods such as leapfrog (velocity Verlet) keep phase-space area exactly, so their energy error stays bounded for millions of orbits. That is why they are used for planetary and molecular dynamics.

Velocity Verlet, half kick. v ← v + (h/2) a(x).

Drift. x ← x + h v.

Half kick again. v ← v + (h/2) a(x), with the new x.

Why it works. Each sub-step is an exact canonical map (a shear of phase space), so their product keeps area, like the true flow. Global error is O(h²), but there is no energy drift.

5. Linear systems

Gaussian elimination on a dense n×n system costs about n3/3 multiplications (doubling n costs 8 times as much); partial pivoting keeps round-off under control. Iterative methods (Jacobi, Gauss-Seidel) are cheaper for large sparse systems and are guaranteed to converge when the matrix is strictly diagonally dominant, |aii|>∑j≠i|aij|. A least-squares straight-line fit has slope m=∑(xi−x‾)(yi−y‾)/∑(xi−x‾)2.

“Euler’s method has error O(h²).” That is its error per step. Over a fixed interval you take 1/h steps, so the global error is O(h): Euler is first order. The same bookkeeping makes RK4 fourth order globally (fifth locally). A second myth: “a smaller step always helps”. For explicit methods on stiff problems the step is limited by stability, not accuracy; for numerical derivatives it is limited by round-off.

  • Orders: bisection 1, secant 1.618, Newton 2 (1 at a multiple root); fixed point converges if |g′(x∗)|<1.
  • Bisection steps: n>log2⁡(L/ϵ).
  • Forward difference O(h); central differences O(h2).
  • Trapezoid O(h2), exact for lines; Simpson O(h4), exact for cubics, even n; n-point Gauss exact to degree 2n−1.
  • Richardson: [2pA(h/2)−A(h)]/(2p−1).
  • Euler O(h), Heun O(h2), RK4 O(h4) globally. Euler stable for y′=−λy only if h<2/λ; backward Euler always.
  • Symplectic integrators keep phase-space area and bounded energy error.

Practise: exam questions on this lesson

They start easy and climb to top-rank level. More on the practice page.

Before this: lesson 2.3, probability distributions. Next: module 3 begins with lesson 3.2, canonical transformations and Hamilton-Jacobi theory. Background on series and ODEs: Taylor series and differential equations.

Check yourself

1Newton-Raphson for f(x) = x² − 3 starts at x₀ = 2. What is x₁?
2Halving the step size in the composite trapezoidal rule reduces its error by about
3Explicit Euler applied to dy/dt = −100y is stable only if the step h is below

Sources

  • W. H. Press, S. A. Teukolsky, W. T. Vetterling and B. P. Flannery, Numerical Recipes, 3rd ed. (2007), chapters 3, 4, 9 and 17
  • R. L. Burden and J. D. Faires, Numerical Analysis, 9th ed. (2010), chapters 2 to 5
  • S. S. Sastry, Introductory Methods of Numerical Analysis, 5th ed. (2012)
  • CSIR-UGC NET Physical Sciences syllabus, Mathematical Methods of Physics (elementary computational techniques)

Your ticks stay in this browser. Save them as a file on the course home.

Updated 2026-11-22. Not affiliated with CSIR, UGC or the National Testing Agency. Check the official CSIR NET website for the current pattern. All questions are original.