Robust Regression

Least squares regression is based on the minimization of the sum of squared residuals. Outliers heavily affect this sum, since the residuals of outliers are large numbers, the square of which is even larger. Thus, a single outlier can substantially affect a least squares regression (leverage effect).

One way to reduce the effect of outliers is to choose a function of the sum of squared residuals which puts less weight on large residuals. For a more detailed discussion of robust regression techniques, see Draper 1981, or Huber 1981.