According to a forthcoming article in /Perspectives on Psychological Science,/ a lot of "social neuroscience" research contains impossibly high correlations between measures of brain activation obtained using fMRI, on the one hand, and behavioral or self-report measures on the other. (Not "impossible" in the pejorative, "I-can't-believe-it" sense but, rather, in the *mathematically*-you-can't-get-a-correlation-that-high-between-two-measures-with-reliabilities-like-that sense).
And, it turns out that the method used for creating such high correlations is essentially massages the data (by dropping voxels form the analysis that aren't likely to produce high correlations) and that this procedure massively (and artifactually) inflates the correlations. You can find a report on the article, and a link to a pre-pub pdf here: http://www.mindhacks.com/blog/2008/12/voodoo_correlations_.html I've also copied the abstract below (if you need an extra "push" in order to click on the link). I wonder how much fMRI research in recent years has fallen into this hole. As the /MindHacks/ reports says, it could be a bombshell. Regards, Chris -- Christopher D. Green Department of Psychology York University Toronto, ON M3J 1P3 Canada 416-736-2100 ex. 66164 [email protected] http://www.yorku.ca/christo/ ========================== Voodoo Correlations in Social Neuroscience Edward Vul1, Christine Harris2, Piotr Winkielman2, & Harold Pashler2 * Abstract The newly emerging field of Social Neuroscience has drawn much attention in recent years, with high-profile studies frequently reporting extremely high (e.g., >.8) correlations between behavioral and self-report measures of personality or emotion and measures of brain activation obtained using fMRI. We show that these correlations often exceed what is statistically possible assuming the (evidently rather limited) reliability of both fMRI and personality/emotion measures. The implausibly high correlations are all the more puzzling because social-neuroscience method sections rarely contain sufficient detail to ascertain how these correlations were obtained. We surveyed authors of 54 articles that reported findings of this kind to determine the details of their analyses. More than half acknowledged using a strategy that computes separate correlations for individual voxels, and reports means of just the subset of voxels exceeding chosen thresholds. We show how this non-independent analysis grossly inflates correlations, while yielding reassuring-looking scattergrams. This analysis technique was used to obtain the vast majority of the implausibly high correlations in our survey sample. In addition, we argue that other analysis problems likely created entirely spurious correlations in some cases. We outline how the data from these studies could be reanalyzed with unbiased methods to provide the field with accurate estimates of the correlations in question. We urge authors to perform such reanalyses and to correct the scientific record. --- To make changes to your subscription contact: Bill Southerly ([email protected])
