Social & Critical Psychology · Simplified
The Method Comes Last
Researchers do not choose a method and then find something to use it on. What they take the thing to be has usually decided the method already.
The case
Meadowbank Academy
A fictional secondary school of about 900 pupils. Referrals for low-level disruption rose from 260 in the autumn term to 412 in the spring, and the governors have asked for research.
One further fact sits in the file. In January the behaviour policy changed. Referral had been discretionary; it became the expected response to a second warning. So nobody yet knows whether pupils are behaving differently.
Meadowbank is invented. No real school, policy, dataset or document is described, and every figure was written for teaching.
Step 1 of 3
First, a judgement
Three researchers are given this file and produce three completely different studies. What most decides which method each of them ends up using?
Step 2 of 3
Declare a starting point, then build
Pick what you take disruption at Meadowbank to be. Then make two decisions. None of the options below is labelled with the position it belongs to, because in a real research meeting nobody announces that either.
Key terms
- Research object
- What you take the thing under study to be. It is a decision, it is usually made before anything is collected, and it is rarely written down.
- In tension
- A choice that does not follow from the position you declared. It is not forbidden, and good mixed designs do it deliberately and say how they hold the two together.
- Coherence
- Whether the decisions follow from the starting point. It is the minimum a design has to clear, not a measure of quality.
Step 3 of 3
The same file, three studies
Here is what each of the three starting points would have produced from the file you have just read. Not one of them is wrong, and not one of them can be swapped for another without changing the question.
| Starting point | Takes disruption to be | Could say | Could not say | Would change |
|---|
What this shows
Every design has already answered a question nobody asked it
Key idea: The governors asked whether behaviour is getting worse. Only one of the three starting points can answer that question, and it is the one that has to treat the referral log as a measure at exactly the moment the rule for making referrals changed. The other two can say things the first cannot: what would have had to be true for the pattern to appear, or how an interaction becomes an attributable act by a named pupil and who is authorised to make that conversion. None of this makes one position better than the others. What it means is that the decision that does the most work in a study is the one taken first, usually in a corridor, and almost never written down. The useful habit is to be able to say what you took the thing to be, what that let you see, and what it put out of reach.
Three positions is a simplification of a contested landscape and this page does not rank them. Being in tension never means being forbidden: good mixed designs combine positions on purpose and explain how they hold them together, and a design that is perfectly coherent can still be a bad study. Coherence is the minimum to clear, not a measure of quality. The fourth option running through every decision belongs to none of the three, and it is included because it is the design a school is most likely to fund rather than because it is a trap.
The longer version adds two further decisions, what counts as evidence and what claim the study could support, and lets you run the same case again from a second starting point and compare the two studies you built. It is at Epistemology Lens Switch in the main collection.