Question about residuals, Least Squares Line, Root Mean Square Error?

Question

How do residuals, Least Squares line, and root mean square error relate and how do they determine a line of fit?

Answer 1

- residuals are the distance between each point and the line of best fit in the y-axis direction.
- choosing a line of some particular slope minimizes the sum of all the squares of distances of points to the line. This is termed "least squares line".
RMSE = A measure of total error defined as the square root of the sum of the variance. The variance is the average of the square of the distance of each data point from the mean. In other words root mean square error is calculated by first squaring all the errors, then taking the square root of the arithmetic average of the squared errors.

Answer 2

Wikipedia says residuals are ordinal errors (ei defined below)

A Least Squares line is that line y = a + bx which comes closest to the points in a group of points {(Fi, xi)}.

Defining ei = Fi-yi, any number of lines can be constructed through this set so that Σei = Σ(Fi - yi) = 0, but there will only be one line (if it exists) such that Σei^2 = Σ(Fi-yi)^2 has a minimum value.

The line can be found by trial and error, as was taught in elementary statistics courses, or it can be found using the equation set:

na + bΣx = ΣF
aΣx + bΣx^2 = ΣxF

The derivation of these equations is either explained or given as an exercise in partial derivatives in calculus. The use of y = a + bx instead of y = mx +b is used to make it easier to extend to generalized trend curves expressed as power series. e.g. y = a + bx + cx^2, etc.

The root mean square error is the square root of the mean of the sum squares of the errors between the line and the points.
e(rms) = sqrt(Σ(ei^2)/n = sqrt(Σ(Fi-yi)^2)/n, where n = the number of points.