Research Methods & Ethics · 2005
Ioannidis, "Why Most Published Research Findings Are False"
By the early 2000s, clinicians and methodologists had long harbored private doubts about the reliability of published biomedical literature. Underpowered trials, outcome switching, and the near-universal preference for positive results had accumulated into a literature that many suspected was systematically skewed, but no one had formally quantified how skewed. John Ioannidis, then at the University of Ioannina before moving to Stanford, decided to work out the mathematics.
In a sole-authored theoretical paper in PLoS Medicine in 2005, Ioannidis used a Bayesian framework to calculate the positive predictive value of a research finding under varying assumptions. His model incorporated prior probability of a true effect, study power, significance threshold, and what he called the bias factor. He showed that smaller studies, smaller effect sizes, greater analytic flexibility, lower pre-study prior probability, and competitive fields with many groups pursuing the same hypothesis each independently reduced the probability that a positive finding reflected a true effect. Under plausible combinations of those parameters, more than half of published claims could be false positives.
The paper presented no new clinical data. Its influence came from formalizing something researchers sensed but had not stated rigorously. It became one of the most downloaded and cited articles in the history of PLoS Medicine. Critics objected that Ioannidis's assumptions were not universally applicable and that his title overstated the claim, but the core mathematical argument was not effectively refuted. The paper arrived at a moment when several high-profile drug findings were failing replication in confirmatory trials, lending it immediate relevance.
Over the following decade, the structural consequences were concrete. Funding agencies including the NIH and major European bodies began requiring preregistration of primary outcomes before enrollment. Journals adopted the CONSORT and STROBE reporting checklists. The reproducibility crisis in social psychology, which emerged around 2011 to 2015 with systematic replication failures, drew direct citations to Ioannidis's framework as its theoretical underpinning. Preclinical biomedical research showed similar patterns when replication studies were organized at scale.
Ioannidis continued producing methodological critiques of biomedical research practices throughout the 2010s, examining issues from surrogate endpoints to bias in nutrition research. The 2005 paper is still assigned in epidemiology and evidence-based medicine courses as a primary text. It did not create the problems it described; it gave researchers a common vocabulary and mathematical framework for discussing problems that had existed for decades.
Key People
- John Ioannidis — Stanford epidemiologist; sole author of the 2005 paper
- David Moher — CONSORT group leader; advancing reporting standards in parallel
- Douglas Altman — UK statistician; longtime critic of poor research methods in biomedicine
PLoS Medicine, 2005
Related landmarks
- 1996 · CONSORT Statement (Research Methods & Ethics)
- 1993 · Founding of the Cochrane Collaboration (Research Methods & Ethics)
- 1992 · Evidence-based medicine: a new approach to teaching the practice of medicine (Research Methods & Ethics)
- 1976 · Large simple trials and trial overviews (Peto and colleagues) (Research Methods & Ethics)