Burden & Faires, 9th Ed. · Fausett, 2nd Ed. · Complete Notes

Numerical Analysis

A full undergraduate course in numerical methods - root-finding, interpolation, differentiation and integration, differential equations, and numerical linear algebra - with error analysis built in from the first chapter and carried through every method that follows. Every chapter includes interactive visualizations and worked examples with real, checkable numbers.

14 Chapters
Complete

These notes assume a solid foundation in Calculus and Linear Algebra and build a complete toolkit for approximating what can't be solved exactly: roots of nonlinear equations, values of functions between known data points, derivatives and integrals of functions given only samples, solutions of differential equations, and solutions of large linear systems. Every method comes with an honest account of its error - where it comes from, how fast it shrinks, and where it breaks down - because in numerical computing, a method without an error bound is just a guess.

Follow the chapters in order the first time through - later chapters lean directly on earlier ones (Taylor's Theorem from Chapter 1 reappears as the error term in Chapters 5, 8, and 9; the Fixed-Point Theorem from Chapter 3 reappears, generalized to matrices, in Chapter 14) - but each post also stands alone as a reference once you've seen the whole arc.

Every chapter ends with a Practice Problem that goes beyond its worked examples - see all fourteen at once on the Practice Problems index.

Unit 1 - Foundations & Root-Finding
01
Chapter 1
Error Analysis
Taylor's Theorem, absolute & relative error, floating-point arithmetic and catastrophic cancellation, algorithm stability, and rates of convergence - the vocabulary every later chapter depends on.
Taylor's Theorem Machine Epsilon Loss of Significance Big-O Notation Order of Convergence
4 sections3 interactive tools
Read →
02
Chapter 2
The Bisection Method
Root-finding by repeated halving. Guaranteed convergence, an error bound known before you start, and the pitfalls of even-multiplicity roots.
Intermediate Value Theorem Error Bound Linear Convergence
3 sections2 interactive tools
Read →
03
Chapter 3
Fixed-Point Iteration
Rewriting $f(x)=0$ as $x=g(x)$ - and why most rearrangements diverge while a careful few converge fast. Cobweb diagrams make the difference visible.
Fixed-Point Theorem Cobweb Diagrams Contraction Mapping
3 sections1 interactive tool
Read →
04
Chapter 4
Newton's Method
Quadratic convergence from a tangent line, the Secant and False Position methods as derivative-free alternatives, multiple roots, and exactly how and why Newton's Method can fail.
Quadratic Convergence Secant Method Multiple Roots Failure Modes
4 sections3 interactive tools
Read →
Unit 2 - Interpolation & Approximation
05
Chapter 5
Lagrange Interpolation
Existence & uniqueness of the interpolating polynomial, the Lagrange basis construction, an exact error formula - and Runge's phenomenon, where more data makes things worse.
Lagrange Basis Interpolation Error Neville's Method Runge's Phenomenon
3 sections3 interactive tools
Read →
06
Chapter 6
Divided Differences
Newton's incremental polynomial form, forward/backward difference formulas for equally spaced data, and Hermite interpolation - matching derivatives as well as values.
Newton's Form Forward & Backward Differences Hermite Interpolation
3 sections2 interactive tools
Read →
07
Chapter 7
Cubic Splines
Piecewise cubics that fix Runge's phenomenon for good - natural and clamped boundary conditions, the tridiagonal system that solves them, and parametric curves.
Natural Splines Clamped Splines Tridiagonal Systems Parametric Curves
4 sections1 interactive tool (draggable)
Read →
// Fundamental Results - Chapters 1–7
Taylor's Theorem remainder
$R_n(x) = \dfrac{f^{(n+1)}(\xi)}{(n+1)!}(x-x_0)^{n+1}$
Order of convergence
$|e_{n+1}| \approx \lambda |e_n|^{\alpha}$
Bisection error bound
$|p_n-p| \le (b-a)/2^n$
Newton's Method
$p_n = p_{n-1} - f(p_{n-1})/f'(p_{n-1})$
Interpolation error
$f(x)-P_n(x) = \dfrac{f^{(n+1)}(\xi)}{(n+1)!}\prod(x-x_i)$
Divided differences
$f[x_0,\ldots,x_k] = \dfrac{f[x_1,\ldots,x_k]-f[x_0,\ldots,x_{k-1}]}{x_k-x_0}$
Unit 3 - Differentiation & Integration
08
Chapter 8
Numerical Differentiation
Forward, backward, central, and five-point difference formulas; the truncation-vs-round-off tradeoff and optimal step size; Richardson extrapolation for near-free extra accuracy.
Finite Differences Optimal Step Size Richardson Extrapolation
3 sections1 live-precision interactive tool
Read →
09
Chapter 9
Numerical Integration
The Trapezoidal Rule and Simpson's 1/3 & 3/8 Rules, derived by integrating interpolating polynomials, with composite forms and a head-to-head accuracy comparison.
Trapezoidal Rule Simpson's Rule Composite Rules
3 sections3 interactive tools
Read →
10
Chapter 10
Adaptive & Gaussian Quadrature
Romberg integration (Richardson extrapolation applied to the trapezoidal rule), adaptive Simpson's Rule that focuses effort where it's needed, and Gaussian quadrature's optimal node placement.
Romberg Integration Adaptive Quadrature Gauss-Legendre
3 sections1 recursive interactive tool
Read →
Unit 4 - Differential Equations
11
Chapter 11
Euler's Method
Initial value problems, well-posedness and the Lipschitz condition, Euler's Method's tangent-line stepping, and higher-order Taylor methods.
Initial Value Problems Local vs. Global Error Taylor Methods
3 sections2 interactive tools
Read →
12
Chapter 12
Runge-Kutta Methods
Heun's and midpoint methods, the classical fourth-order Runge-Kutta method (RK4), systems of ODEs, and a first look at stability.
RK2 / Heun's Method RK4 Systems of ODEs Stability
3 sections1 interactive tool
Read →
Unit 5 - Numerical Linear Algebra
13
Chapter 13
Norms & Eigenvalues
Vector and matrix norms, the condition number and why some systems resist accurate solution no matter the method, spectral radius, and the Power Method for dominant eigenvalues.
Vector & Matrix Norms Condition Number Power Method
4 sections1 interactive tool
Read →
14
Chapter 14
Iterative Methods for Linear Systems
Jacobi's Method, Gauss-Seidel, Successive Over-Relaxation (SOR), and the spectral-radius convergence criterion - plus the diagonal-dominance shortcut that guarantees it.
Jacobi's Method Gauss-Seidel SOR Diagonal Dominance
4 sections1 interactive tool
Read →
What you'll need
  • A full year of Calculus (limits, derivatives, Taylor series, integrals)
  • Linear Algebra (matrices, eigenvalues, vector spaces) - see the Linear Algebra notes
  • Comfort reading and writing basic MATLAB or Python is assumed for the accompanying assignments
Textbooks

Numerical Analysis
Richard L. Burden & J. Douglas Faires - 9th Edition.
Applied Numerical Analysis Using MATLAB
Laurene V. Fausett - 2nd Edition.

How error is treated

Every method in these notes comes with its error term derived, not just quoted - truncation error from Taylor's Theorem, round-off error from floating-point arithmetic, and where the two trade off against each other.

// Fundamental Results - Chapters 8–14
Central difference
$f'(x_0) \approx \dfrac{f(x_0+h)-f(x_0-h)}{2h}$
Simpson's Rule
$\displaystyle\int_{x_0}^{x_2}\!f\,dx \approx \dfrac{h}{3}[f_0+4f_1+f_2]$
Euler's Method
$w_{i+1}=w_i+hf(t_i,w_i)$
RK4
$w_{i+1}=w_i+\tfrac h6(k_1+2k_2+2k_3+k_4)$
Condition number
$\kappa(A)=\|A\|\cdot\|A^{-1}\|$
Iterative convergence
converges $\iff \rho(T) < 1$
Numerical Analysis · Complete Notes Burden & Faires, Numerical Analysis, 9th Ed. · Fausett, Applied Numerical Analysis Using MATLAB, 2nd Ed.
Scroll to Top