fMRI of dead salmon: how not to do science

Uncorrected randomness leads to false positives

Uncorrected randomness leads to false positives

Craig Bennett at Prefontal blog had highlighted the problem of cherry-picking data to collect false positives. He did this in a memorable way by presenting a poster of fMRI scans of a dead fish: “Dead salmon responds to pictures of people.”

A dead salmon has become a scientific celebrity after its brain supposedly lit up when shown pictures of humans during a brain scan.

…the study was done to show that data from an fMRI brain scan can lead to false positives — misleading results — if not carefully analyzed.

Yes, the salmon was dead — bought in a lifeless state at a fish market and scanned an hour later. No, the results are not shocking or miraculous. Like many scientific studies, the study and its results, presented earlier this year in a poster at a conference, are technical and rather bland:

“The goal of the salmon poster was to encourage the minority of researchers who report uncorrected statistics to move forward and begin using basic multiple comparisons correction in their research,” says study leader Craig Bennett, a postdoctoral researcher in the Department of Psychology at the University of California, Santa Barbara.

In a nutshell, the data reported by Bennett and colleagues in no way suggests the salmon’s brain was functioning, but rather reveal anomalies that can be misleading if you’re not careful.

That, of course, is the statistical problem that plagues studies of everything from drug efficacy to ESP: it skews your data to keep positive results and throw away negative ones. Whatever corrections, processing, or reporting standards you use must be decided on in advance and applied to all results.

Follow-up: read “The story behind the Atlantic salmon.”

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: