Numerical Methods for Least Squares Problems

The method of least squares was discovered by Gauss in 1795. It has since become the principal tool to reduce the influence of errors when fitting models to given observations. Today, applications of least squares arise in a great number of scientific areas, such as statistics, geodetics, signal processing, and control.

In the last 20 years there has been a great increase in the capacity for automatic data capturing and computing. Least squares problems of large size are now routinely solved. Tremendous progress has been made in numerical methods for least squares problems, in particular for generalized and modified least squares problems and direct and iterative methods for sparse problems. Until now there has not been a monograph that covers the full spectrum of relevant problems and methods in least squares.

This volume gives an in-depth treatment of topics such as methods for sparse least squares problems, iterative methods, modified least squares, weighted problems, and constrained and regularized problems. The 860 references provide a comprehensive survey of the available literature on the subject.

Special Features

A solid understanding of numerical linear algebra is needed for the more advanced sections. However, many of the chapters are more elementary and because basic facts and theorems are given in an introductory chapter, the book is partly self-contained.

Mathematicians working in numerical linear algebra, computational scientists and engineers, statisticians, and electrical engineers. The book can also be used in upper-level undergraduate and beginning graduate courses in scientific computing and applied sciences.

Chapter 1: Mathematical and Statistical Properties of Least Squares Solutions....1
Introduction; The Singular Value Decomposition; The QR Decomposition; Sensitivity of Least Squares Solutions;

Chapter 2: Basic Numerical Methods. ...37
Basics of Floating Point Computation; The Method of Normal Equations; Elementary Orthogonal Transformations;
Methods Based on the QR decomposition; Methods Based on Gaussian Elimination; Computing the SVD;
ank Deficient and Ill-Conditioned Problems; Estimating Condition Numbers and Errors; Iterative Refinement;

Chapter 3: Modified Least Squares Problems. ...127
Introduction; Modifying the Full QR Decomposition; Downdating the Cholesky Factorization;
Modifying the Singular Value Decomposition; Modifying Rank Revealing QR Decompositions;

Chapter 4: Generalized Least Squares Problems. ...153
Generalized QR Decompositions; The Generalized SVD; General Linear Models and Generalized Least Squares;
Weighted Least Squares Problems; Minimizing the l_p Norm; Total Least Squares;

Chapter 5: Constrained Least Squares Problems....187
Linear Equality Constraints; Linear Inequality Constraints; Quadratic Constraints;

Chapter 6: Direct Methods for Sparse Least Squares Problems....215
Introduction; Banded Least Squares Problems; Block Angular Least Squares Problems; Tools for General Sparse Problems;
Fill Minimizing Column Orderings; The Numerical Cholesky and QR Decompositions; Special Topics;
Sparse Constrained Problems; Software and Test Results

Chapter 7: Iterative Methods for Least Squares Problems....269
Introduction; Basic Iterative Methods; Block Iterative Methods; Conjugate Gradient Methods; Incomplete Factorization Preconditioners;
Methods Based on Lanczos Bidiagonalization; Methods for Constrained Problems;

Chapter 8: Least Squares Problems with Special Bases. ...317
Least Squares Approximations and Orthogonal Systems; Polynomial Approximation; Discrete Fourier Analysis;
Toeplitz Least Squares Problems; Kronecker Product Problems;

Chapter 9: Nonlinear Least Squares Problems. ...339
The Nonlinear Least Squares Problem; Gauss-Newton Type Methods; Newton-Type Methods; Separable and Constrained Problems;


1996 / xviii+ 408 pages / Softcover
ISBN 0-89871-360-9 / List Price $74.50 / SIAM Member Price $52.15 / Order Code OT51

To order the book from SIAM click here: SIAM Homepage