Craig Bennett at Prefontal blog had highlighted the problem of cherry-picking data to collect false positives. He did this in a memorable way by presenting a poster of fMRI scans of a dead fish: “Dead salmon responds to pictures of people.”
A dead salmon has become a scientific celebrity after its brain supposedly lit up when shown pictures of humans during a brain scan.
…the study was done to show that data from an fMRI brain scan can lead to false positives — misleading results — if not carefully analyzed.
Yes, the salmon was dead — bought in a lifeless state at a fish market and scanned an hour later. No, the results are not shocking or miraculous. Like many scientific studies, the study and its results, presented earlier this year in a poster at a conference, are technical and rather bland:
“The goal of the salmon poster was to encourage the minority of researchers who report uncorrected statistics to move forward and begin using basic multiple comparisons correction in their research,” says study leader Craig Bennett, a postdoctoral researcher in the Department of Psychology at the University of California, Santa Barbara.
In a nutshell, the data reported by Bennett and colleagues in no way suggests the salmon’s brain was functioning, but rather reveal anomalies that can be misleading if you’re not careful.
That, of course, is the statistical problem that plagues studies of everything from drug efficacy to ESP: it skews your data to keep positive results and throw away negative ones. Whatever corrections, processing, or reporting standards you use must be decided on in advance and applied to all results.
Follow-up: read “The story behind the Atlantic salmon.”