Research Methods
Factorial ANOVA Interaction Detective
Four cell means are all a 2×2 design has. Move them and the marginal means, the two main effects and the interaction all follow — and so does a picture that is easy to over-read.
Simulated — invented cell means, editable at will
Learning objective
By the end you should be able to define an interaction as a difference of differences, and say why a shape on a plot is not yet a finding.
About 25 minutes. Nothing you do here is saved or sent anywhere.
Two things a plot will not tell you
Non-parallel lines are the definition of an interaction in the sample, not evidence of one in the population. Two things decide whether the picture means anything: the scale it is drawn on — a zoomed axis makes half a point look like a chasm — and the uncertainty around each cell mean. Both are controls in this tool, and both are missing from most published interaction plots.
- Answer the three prediction questions to open the detective.
- Load a pattern, or drag the four cell means yourself.
- Read the interaction as a difference of differences, not as a shape.
- Switch the axis to the full scale and add uncertainty before believing anything.
First, three predictions
A fictional 2×2 study. People do a simple or a complex task and receive feedback either immediately or after a delay. The outcome is a score out of 100. These are the four cell means.
| Feedback | Simple task | Complex task |
|---|---|---|
| Immediate | 70 | 50 |
| Delayed | 50 | 70 |
The detective
The four sliders are the whole dataset. Everything else on this screen — the marginal means, the two main effects, the interaction and the plot — is arithmetic performed on them.
Key terms
- Cell mean
- The average for one combination of the two factors. There are four of them here.
- Main effect
- One factor's effect averaged over the levels of the other. It lives in the row and column margins.
- Interaction
- A difference of differences: how much one factor's effect changes across the levels of the other.
- Ordinal and disordinal
- Whether the two lines keep the same order across the plot, or cross over. Both are interactions.
Four numbers, one picture
An interaction is a difference of differences: how much the effect of feedback changes when you move from the simple task to the complex one.
Interaction plot
| Feedback | Simple | Complex | Row mean |
|---|
Say it in words
The same interaction, read the other way round
| Simple effect | Size |
|---|
Challenge — describe it in a sentence
"There was a significant interaction" is not a description of anything. Build the sentence that actually says what happened in the crossover pattern: immediate feedback gives 70 on the simple task and 50 on the complex one; delayed feedback gives 50 on the simple task and 70 on the complex one.
What this demonstrates
Main effects live in the margins
A main effect is a comparison of marginal means — averages taken across the levels of the other factor. That averaging is exactly why a main effect can vanish while something large is going on. In the crossover pattern, feedback helps by 20 points in one task and hurts by 20 in the other, and the two cancel to nothing. Reporting "no main effect of feedback" as though it meant "feedback did not matter" is wrong in a way that a table of marginal means cannot show you.
An interaction is a difference of differences
Take the effect of feedback within the simple task; take it again within the complex task; subtract. That single number is the interaction, and it does not care which factor you put on the horizontal axis, because subtracting the other way round gives the same value. The picture changes, the term does not.
A shape is not a finding
Non-parallel lines in a sample are guaranteed: four means drawn from anything will almost never line up exactly. Whether the non-parallelism means anything depends on how big it is relative to the scale of the outcome and relative to the uncertainty in each cell. The tool lets you take the same four numbers from "obviously a strong interaction" to "obviously nothing" without editing a single value, using the axis and the error bars alone.
Ordinal, disordinal, and what to say about main effects
When the lines do not cross within the range studied, the interaction is ordinal. The effect is bigger in one condition than the other but points the same way, so a main effect can usually be described sensibly alongside it. When they cross, it is disordinal, and describing the main effect on its own becomes misleading. The rule of thumb that follows is not "always interpret the interaction first" but "say which simple effects you mean".
For teaching elsewhere: take this activity as one self-contained block of HTML, on the clipboard or as a file. Either way it is styled so that it will not disturb the page you put it into.