Lines of Best Fit

Least Squares Criterion: The most common method for finding the line of best fit is the 'Least Squares' method. It works by minimizing the sum of the squares of the vertical distances (residuals) between each data point and the line.

Residuals: A residual is defined as the difference between the observed value ($y$) and the predicted value ($\hat{y}$) provided by the line. Squaring these values ensures that positive and negative deviations do not cancel each other out and penalizes larger outliers more heavily.

Centroid Property: Mathematically, the line of best fit for any dataset will always pass through the point $(\bar{x}, \bar{y})$, which represents the mean of all $x$-values and the mean of all $y$-values. This point acts as the 'balance point' of the entire distribution.

The Linear Equation: The line is expressed in the form $\hat{y} = mx + b$, where $m$ is the slope and $b$ is the y-intercept. The slope indicates the average change in the dependent variable for every one-unit increase in the independent variable.

Calculating Slope ($m$): The slope is determined by the correlation between the variables and their respective standard deviations. It can be calculated using the formula $m = r \cdot \frac{s_y}{s_x}$, where $r$ is the correlation coefficient.

Determining the Intercept ($b$): Once the slope is known, the intercept is found by substituting the means of the data into the equation: $b = \bar{y} - m\bar{x}$. This ensures the line is anchored correctly relative to the average values of the dataset.

Correlation vs. Causation: A line of best fit shows that two variables are related, but it does not prove that one causes the other. External 'lurking variables' might be influencing both, or the relationship might be purely coincidental.

Positive vs. Negative Correlation: A positive slope indicates that as $x$ increases, $y$ tends to increase. Conversely, a negative slope indicates that as $x$ increases, $y$ tends to decrease.

Verify the Correlation Coefficient ($r$): Always check if the sign of $r$ matches the slope of your line. If $r$ is positive, your line must have an upward slope; if $r$ is negative, the line must slope downward.

Assess the Scatter: Before calculating, look at the scatter plot to see if a linear model is even appropriate. If the data points form a curve (like a parabola), a straight line of best fit will provide misleading predictions.

Identify Outliers: Be aware that a single point far from the rest of the data can 'pull' the line toward it, significantly altering the slope and intercept. In exams, you may be asked to describe how removing such a point would change the model's accuracy.

Forcing Through the Origin: A common mistake is assuming the line must pass through $(0,0)$. Unless the data specifically dictates it, the y-intercept ($b$) should be calculated freely based on the data's distribution.

Over-reliance on $r$: A high correlation coefficient ($r$) does not guarantee that a linear model is the best fit. Always check a residual plot; if the residuals show a clear pattern (like a U-shape), the original data is likely non-linear despite a high $r$ value.

Confusing $x$ and $y$: The line of best fit for $y$ on $x$ is different from the line of $x$ on $y$. Always ensure you are minimizing the vertical distances (errors in $y$) unless instructed otherwise.