Research Methods · Simplified
Factorial Interaction Detective
An interaction is a difference of differences. Two other things decide whether the one you can see means anything.
The design
Four cell means, and nothing else
A 2 by 2 study: feedback given immediately or after a delay, on a simple or a complex task. Everything below is arithmetic on these four numbers.
Key terms
- Main effect
- The difference between the two levels of one factor, averaged over the levels of the other.
- Interaction
- A difference of differences: how much the effect of one factor changes across the levels of the other. It is the same number whichever factor you put on the horizontal axis.
- Standard error
- How much a cell mean would move if the study were repeated. Four cell means each with standard error s give an interaction with standard error 2s.
- Crossover
- Lines that cross. A crossover in the point estimates is not the same as a crossover in the evidence.
One line per feedback timing across the two tasks. Parallel lines mean no interaction; the more they diverge, the larger the difference of differences.
What this shows
The number, the picture and the evidence
The interaction is fixed by the four means. How large it looks is fixed by the axis. Whether it is worth interpreting is fixed by the uncertainty, and none of the three settles the other two.
Key idea: An interaction is the difference of differences, and it is symmetric: how much the effect of feedback timing changes across tasks is exactly how much the effect of task changes across timings. Which factor goes on the horizontal axis is a presentation decision and does not change the number. What does change what you see is the vertical scale, and what decides whether the pattern is worth a sentence is the uncertainty in the four means.
There is no F, no p and no effect size here, because there is no dataset: only four means. Putting a test result on screen would invite reading one off numbers that contain no information about within-cell variation or sample size. The standard errors offered are a menu rather than an estimate from data. And a large main effect in the presence of a crossover interaction is usually not worth reporting on its own, because the average of two opposite effects describes nobody.
The longer version adds pattern presets, an additive-fit control that removes the interaction, a read-it-the-other-way-round disclosure and a fill-in-the-sentence challenge. It is at Factorial ANOVA Interaction Detective in the main collection.