Why this chapter matters
Prediction and comparison tasks often require relationship models, and ANOVA extends mean comparison to more than two groups.
What you will learn
- Interpret correlation direction and strength without overstating causation.
- Fit and interpret a simple linear regression model including slope and residuals.
- Explain ANOVA as variance-based comparison of multiple group means.
Understand the core ideas
Correlation and regression are tools for quantitative relationships, but they answer different questions from multi-group mean comparisons. Correlation summarizes direction and strength of linear association, while regression gives an equation for expected response change as predictor changes. The slope carries units and supports prediction inside the observed range. Residuals show prediction errors and help diagnose model fit. These tools can support explanation and forecasting, but causal claims require design features beyond the regression line itself.
ANOVA targets a separate problem: comparing means across three or more groups. It evaluates whether between-group variability is large relative to within-group variability through an F statistic. A significant ANOVA result indicates at least one group mean differs, not that all pairs differ. Follow-up comparisons are needed for specific contrasts, and those comparisons should account for multiple testing. Keeping these method roles distinct helps avoid incorrect claims, especially when users mix association language, prediction language, and causal language in one report.
Key terms
- Correlation coefficient
- Number from -1 to 1 describing direction and strength of linear association.
- Least squares regression line
- Line that minimizes sum of squared residuals for prediction.
- Residual
- Observed response minus predicted response for a data point.
- ANOVA F statistic
- Ratio comparing between-group variance estimate to within-group variance estimate.
Compute a one-way ANOVA F statistic
Three groups have observations A: 1, 2; B: 3, 4; C: 5, 6. Compare variation between group means with variation inside groups.
- Compute group means 1.5, 3.5, and 5.5, and the grand mean 3.5.
- Between-group sum of squares is . With 2 degrees of freedom, MS between = 8.
- Within-group sum of squares is 0.5 + 0.5 + 0.5 = 1.5. With 3 degrees of freedom, MS within = 0.5.
- Compute F = MS between / MS within = 8 / 0.5 = 16.
A common misconception
Claim: High correlation means one variable causes the other.
Correction: Correlation and regression quantify association patterns; causal claims need appropriate design and confounding control.
Lessons in this chapter
- Correlation and causation boundariesSeparate association evidence from causal claims. Read the full guide →
- Simple linear regression basicsInterpret slope, intercept, prediction, and residual error. Read the full guide →
- Diagnostics and model cautionCheck linearity, outliers, and extrapolation risk.
- ANOVA first lookDescribe null and alternative ideas for comparing three or more means.
Study task
Chapter checkpoint
What does the slope of a simple linear regression line represent?
It is the expected change in the response variable for a one-unit increase in the predictor, on average, within the modeled range.