Research Methods
ANOVA F-Ratio Visualiser
F is a fraction. Pull the group means apart and the top grows; make the groups noisier and the bottom grows. Nothing about the test makes sense until you can see both halves of it at once.
Simulated — generated scores from a documented seed
Learning objective
By the end you should be able to read F as a ratio of two variances, and say what question the omnibus test actually answers.
About 25 minutes. Nothing you do here is saved or sent anywhere.
What the omnibus test asks
One-way ANOVA asks a single question: are all the population means equal? It is not three t-tests wearing a coat, and a small p does not identify which groups differ, in which direction, or by how much. Those are separate questions with separate answers, and the second experiment on this page is built to make that unavoidable.
- Answer the prediction question to open Experiment 1.
- Separate the group means and watch F climb.
- Now raise the within-group spread and watch it collapse again.
- In Experiment 2, decide which pattern of means produced a given F.
First, a prediction
Three fictional groups of 30 people each sit the same 100-point test after a different revision method. In the first study the population means are 44, 52 and 60, and the within-group standard deviation is 6. A second study has exactly the same three population means, but the scores within each group are twice as spread out.
Experiment 1 — the two halves of the ratio
Three groups, labelled A, B and C so that nothing about the story does the explaining. Each dot is one simulated score, each heavy bar is a group mean, and the dashed line is the grand mean.
Key terms
- Between-groups mean square
- How far the group means sit from the overall mean, turned into a variance estimate.
- Within-groups mean square
- How far observations sit from their own group's mean, turned into a variance estimate.
- F
- The between-groups mean square divided by the within-groups one. If nothing is going on, the two are estimating the same quantity.
- Omnibus test
- One test asking whether the means differ at all, which does not say which of them differ from which.
Signal over noise
The distance of each bar from the dashed line is the between-groups part. The scatter of dots around their own bar is the within-groups part. F is one divided by the other, after each is turned into a mean square.
Three simulated groups
| Group | n | Mean | SD |
|---|
Reading this correctly
The full source table
| Source | SS | df | MS |
|---|
Experiment 2 — what F cannot tell you
A fictional study of three revision methods reports F(2, 87) = 7.50, p = .001, with 30 people per group and a within-group standard deviation of 6. Below are three patterns of group means. Look at each one, then say which of them produced that F.
Three shapes, one number
Browse the patterns freely. The F each one would produce stays hidden until you commit to an answer.
Pattern A
| Group | Mean | Distance from grand mean |
|---|
All three patterns, with their model-implied F
| Pattern | Means | SS between | F(2, 87) |
|---|
Challenge — what F entitles you to say
A fictional study of three revision methods, 30 people per group, reports F(2, 87) = 7.50, p = .001.
What this demonstrates
F is a comparison of two estimates of the same thing
If the population means really are equal, the scatter of the group means around the grand mean is just sampling noise. Both mean squares then estimate the same population variance. Their ratio then wanders around 1. When the population means are not equal, the numerator picks up something extra that the denominator never sees, and the ratio climbs. That is the entire logic, and it is why a ratio is used rather than a difference.
Three things move F, and only one of them is the effect
Separating the means raises it. Increasing the within-group spread lowers it, and does so as the square. Double the within-group standard deviation and the amount by which F exceeds 1 divides by four. Increasing n raises it, because between-groups sum of squares grows with n while the within-groups mean square does not. A large F is therefore not a statement about how big the difference between the methods is.
The omnibus test answers one question, and it is a modest one
A significant F says the data sit awkwardly with the hypothesis that all the population means are equal. It does not say which means differ, in which direction, or by how much. Experiment 2 makes this concrete: three genuinely different patterns of means — one evenly spaced, one with a high outlier, one with a low outlier — produce exactly the same F. Anything you want to say about particular groups needs a planned contrast or a post-hoc comparison, and those carry their own multiplicity problem.
ANOVA is not a bundle of t-tests
Running every pairwise t-test would answer three separate questions and would inflate the chance of at least one false positive, which is the reason the omnibus test exists. But the relationship runs the other way too: with only two groups, F is exactly the square of the independent-samples t. They are the same test written twice, and the generalisation to three groups is where they part company.
For teaching elsewhere: take this activity as one self-contained block of HTML, on the clipboard or as a file. Either way it is styled so that it will not disturb the page you put it into.