Psychology●●●●●Difficulty 4 of 5

Why do so many famous psychology findings vanish when scientists repeat them?

In 2015, researchers re-ran 100 published psychology studies. Only about a third of the positive results came out the same way twice.

▶ Start the story

Many famous findings vanish on a second try because the way science was done made it easy to publish flukes. Journals preferred positive, striking results and tucked failures into a "file drawer". Studies were often too small to detect real effects reliably. And researchers, usually with no intent to cheat, had lots of freedom in how to analyse their data, which can push the rate of false alarms above 60 percent. The result is what's now called the replication crisis.

The moment it became impossible to ignore came in 2011. Daryl Bem, a respected psychologist, published nine experiments in a top journal that seemed to show people could sense the future. Almost nobody believed it, and repeat experiments found nothing. What shook the field was that Bem had used the same methods everyone else used. If standard methods could "prove" precognition, what else had they "proved"? Bem later told a journalist he was all for rigour but preferred other people do it.

Portrait photograph of psychologist Daryl Bem, an older man with grey hair and glasses.
Daryl Bem, whose 2011 paper claiming evidence for precognition showed that standard methods could 'prove' the impossible.Photo: Robert Sheaffer · CC BY-SA 3.0

So psychologists checked. In 2015 the Reproducibility Project re-ran 100 published studies. Of 97 with positive results, only 36 percent worked again, and the effects were on average half as big. Famous ideas such as social priming have struggled since.

The response has been encouraging: many journals now accept studies based on their plan before any data exist, so results can't be bent after the fact. Some call it a credibility revolution.

Quiz me

0/3

  1. 1.Why did Bem's 'Feeling the Future' paper alarm psychologists so much, beyond its strange claim?
  2. 2.How does publication bias distort the scientific record?
  3. 3.How do registered reports help fix the problem?

Recap

A single striking study is a clue, not a fact: look for replications, big samples and plans registered before the data.

Surprising fact · In the 2015 Reproducibility Project, only 36% of 97 positive psychology results replicated, at about half the original effect size.

Sources (3)

No source, no claim. Every fact in this lesson (35 claims) cites at least one of these.

  1. [1]Replication crisis · Wikipedia
  2. [2]Daryl Bem · Wikipedia
  3. [3]Cognitive dissonance · Wikipedia
More lessons in 🧠 Psychology (3) See all psychology lessons →

One more light on your map.

Get one lesson like this every day, about the things you love. Free, in two or five minutes.

Get the share card for this lesson ↗