Curvilinear Regression

Simple linear regression has been developed to fit straight lines to data points. However, sometimes the relationship between two variables may be represented by a curve instead of a straight line. Such "non-linear" relationships need not be non-linear in a mathematical sense. For example, a parabolic relationship may be well-modeled by a (modified) linear regression, since a parabola is a linear equation, as far as its parameters are concerned. Sometimes, such relationships are called "curvilinear".
 

Please note that the term "non-linear" has a double meaning: first, people use the term when they think of curves which are not straight lines, and secondly, a non-linear relationship in its mathematical sense is a function which relates the x and y variable(s) by one or more non-linear functions (such as a cosine). More details can be found here.

There are several ways to fit a curve other than a line (or, generally speaking, an n-dimensional hyperplane) to the data:
 

• deriving the proper regression formula, which may be cumbersome, and requires some calculus knowledge,
• using linear regression after transforming the data into a linear problem,
• using optimization algorithms to minimize the error surface, and
• using non-linear modeling techniques, such as neural networks.

The first two approaches require the type of functional relationship to be known. In many standard cases, the second approach may be appropriate:
 

• Transform the curvilinear model to a linear model, by applying a proper transformation to both the independent and the dependent variable. For the univariate case, you may visually check the linearity after the transformation, by plotting the transformed variables against each other.
• Calculate the regression parameters for the linearized model.
• Transform the regression parameters of the linearized model back to the original (curvilinear) case.

Below is a table of the transformations for linearizing some common relationships.