Research Methods

Confound Detective

Three fictional studies, each with a confident causal claim. Sort the variables that travel with the manipulation from the ones that only add noise — then repair the design and watch the estimate move back towards the truth.

Simulated — fictional studies and a transparent bias model

Learning objective

By the end you should be able to state the two conditions a variable must meet to confound an effect, and test a candidate against both.

About 30 minutes. Nothing you do here is saved or sent anywhere.

The numbers are a model, not a finding

Each study has a true effect that only the tool knows, plus a bias contributed by each unresolved confound. Those figures are chosen to make the argument visible. They are not estimates of how large real confounding is in any real study, and no real trial, ward, module or dataset is described.

Answer the opening question to unlock the detective.
Read the study card, then classify each of its five variables.
Check your classification and read the note on each variable.
Move to the repair bench and tick fixes. Watch the two bars.

This part needs JavaScript

The detective builds its diagrams and estimates in your browser, so it cannot run with JavaScript switched off. The rest of this page is readable without it.

Key terms

Confound: A variable that both differs between the conditions and affects the outcome. It has to do both.
Noise: Variation in the outcome unrelated to the conditions. It widens an estimate without shifting it.
Bias: A systematic shift in what the study estimates. More participants will not reduce it.
Randomisation: Allocating by chance. It makes systematic differences between the groups unlikely, not impossible.
Statistical adjustment: Removing a measured variable's contribution arithmetically. It can only act on what was measured, and measured well.

Study 1 of 3

The causal picture

Each confound, whether it is still open, and the bias it is contributing to the estimate
Confound	Status	Bias it adds

Where the design stands

What each repair you ticked actually does

Which of these objections identifies something that could bias the comparison? Select all that apply. The two groups differ slightly in mean age The exercise group knew they were receiving something and the waiting list knew they were not More people dropped out of the exercise group, and those who dropped out were doing worst All 200 were volunteers, so the sample is unrepresentative The researcher who scored the symptom measure knew which group each person was in

A confound needs two properties, not one

It must differ systematically between the conditions, and it must plausibly touch the outcome. A variable with only the first is irrelevant; a variable with only the second adds noise and widens your interval without moving your estimate. Students who learn to list "extra variables" find confounds everywhere and cannot say which ones matter.

Noise and bias are different problems with different cures

Noise makes an estimate imprecise, and more data fixes it. Bias makes an estimate wrong, and more data makes the wrong answer more confident. The repair bench contains a sample-size button precisely so that it can be seen doing nothing.

Randomisation and adjustment are not substitutes

Randomising breaks the link between the condition and everything else, measured or not, known or not. Adjusting for a covariate handles that one variable, provided it was measured, measured well, and entered into the model in roughly the right form. That is a much longer list of things to be right about, which is why the two are not interchangeable however similar the output looks.

Some variables are the treatment

The spaced-repetition schedule in study 1 is not a confound; it is what the app is. Removing it does not clean up the comparison, it deletes the manipulation. Deciding which features are the active ingredient is a theoretical question, and answering it needs a condition designed to isolate it rather than a statistical control.

What this does not demonstrate

The bias figures are invented. Each study's true effect and each confound's contribution were chosen to make the teaching point legible. They are not estimates of real confounding anywhere.
Bias adds up neatly here, and it does not in life. Real confounds interact, point in opposite directions and sometimes cancel. They are additive here so the arithmetic can be followed.
A repaired design is not a good study. Blocking every confound in the list leaves the measures, the sample, the power and the question exactly as they were.
Directed diagrams encode assumptions. The arrows shown are assertions about the world. Drawing a different diagram is a substantive claim about the world, not a formatting choice.
No real studies. The module, the ward and the typography experiment are all invented, as are the numbers.

Confound Detective

Learning objective

The numbers are a model, not a finding

First, a judgement

The detective

One study at a time

Study 1 of 3

What the study would estimate

Variable by variable