The method of least squares, or even simply maximum likelihood is one of the more powerful tools available to a statistician. It is powerful because its simplicity means it can be used in a variety of regression problems.

Regression simply means line fitting, and lines are just a graphical way to represent a model, which is the mathematical way to describe the relationship between an independent variable and one or more dependent variables. There is a lot of text on linear straight line fitting, so I’m not going to go into too much detail. I will however briefly discuss the principle behind least squares.