My results aren't what I expected
Rule out errors first, then take surprising or null results seriously and write them up honestly.
In short
Start here, today. Before interpreting anything, check the pipeline: confirm the number of cases at each step, check how your main variables are coded, and rerun the key analysis from the raw data. For qualitative or archival work, reread the material behind your main finding and make sure you've read it fairly. This takes about 30 minutes.
After that, work through these steps. Choose one to jump to it.
Step by step
Tick each step as you finish it. Open a step for the how.
0 of 6 steps done
Check for errors first
Look for the dull explanations before the exciting ones: a reversed scale, a bad merge, mislabelled samples, an instrument out of calibration, a coding or transcription mistake. Ask someone else to read your code or protocol, because fresh eyes catch what yours skip over. If you find a mistake, fix it, log it and rerun everything that depended on it.
Take the result seriously
Once you're confident it isn't an error, treat it as information. Ask what would have to be true for this result to make sense: a different mechanism, a context unlike earlier studies, a measure that captures something else, a sample too small to detect a modest effect, a theory that doesn't hold here. Write each possibility down with the evidence that would support or rule it out.
Discuss it with your supervisor before doing more
Bring the result, the checks you've done and your list of explanations. Agree together which further analyses are justified and why before you run them, and record that in your Decision Log. Your supervisor has almost certainly seen surprising results before and can help you judge which explanations are worth pursuing.
Don't chase significance
Running more models, subgroups, outcomes or exclusions until a p-value crosses a threshold, and reporting only that one, is often called p-hacking. It makes false positives much more likely and is widely recognised as a questionable research practice, whatever the intention behind it. The qualitative version is choosing only the quotes or sources that fit the story you hoped to tell. If you run extra analyses, label them exploratory and report what you tried, and if you preregistered, report the preregistered analysis first.
Write it up honestly
State what you found plainly, then discuss what it might mean, how it fits or challenges earlier work, and its limits. Keep your original hypotheses as they were and say clearly that the results didn't support them; rewriting hypotheses after seeing the data is another integrity problem. A null result with a clear account of sample size, measurement and context is useful to other researchers, and some journals explicitly welcome null results, replications or registered reports.
Prepare to discuss it in your defence
Examiners will ask about unexpected results, and that's a chance to show judgement. Add the likely questions to your Q&A Practice Bank: why you think it happened, what you checked, what you'd do differently, and what the result means for the field.
Words you can use
Is this you? Signs to look for
- Your main effect is null, reversed or much smaller than earlier studies suggested.
- Your interviews or sources point somewhere quite different from your proposal.
- You're tempted to try one more model, subgroup or outcome until something comes out significant.
- You're dreading telling your supervisor because it feels like the project has failed.
Why this happens
Proposals are written with hopeful predictions, and real data often disagrees. Effects come out smaller than in earlier studies, partly because studies with large effects are more likely to get published. Participants say things the literature didn't anticipate. Archives tell a different story from the secondary sources. That is research doing its job, even when it feels like a setback.
There are two risks. The first is that the surprise is really an error, which is why you check before anything else. The second is that disappointment pushes you to run analysis after analysis until something looks significant, or to quietly drop what didn't fit. That is a research integrity problem, not a matter of style: it makes false findings more likely and can mislead everyone who builds on them. An honest, well-explained unexpected result is a contribution, and good examiners know it.
Worth a look (2)
- Sage Research MethodsBooks, case studies, videos and practice data sets on research methods, from interviews to regression. Your library usually provides access.
- PRISMAThe reporting guideline for systematic reviews and meta-analyses, with the checklist, flow diagram templates and extensions for scoping and other reviews. Many journals expect you to follow it.
When to bring someone in
Talk to your supervisor before running further analyses, and to a statistics consulting service if you're unsure whether a null result reflects sample size or measurement. If anyone pressures you to present results in a way you believe is misleading, you can get confidential advice from your graduate coordinator, your ombudsperson or your institution's research integrity office.
Find the right person, or talk to Ify