The article Vul et el did was on studies of functional MRI (fMRI) scans which use voxels and apply some sort of gaussian smoothing to determine which ones to use. Same exact method Firth et al used.
http://www.edvul.com/pdf/VulHarrisWinkielmanPashler-PPS-2009.pdf These basic steps common to most fMRI data analyses yield matrices consisting of tens or hundreds of thousands of numbers indicating activation levels in different voxels. These can be (and indeed generally are) displayed as images. However, to obtain quantitative summaries of these results and do further statistics on them (such as correlating them with behavioral measuresâthe topic of the present article), an investigator must somehow select a subset of voxels and aggregate measurements across them. This can be done in various ways. A subset of voxels in the whole brain image may be selected based on purely anatomical constraints (e.g., all voxels in a region generally agreed to represent the amygdala, or all voxels within a certain radius of some a priori speciï¬ed brain coordinates). Alternatively, regions can be selected based on ââfunctional constraints,ââ meaning that voxels are selected based on their activity pattern in functional scans. For example, one could select all the voxels for a particular subject that responded more to reading than to nonlinguistic stimuli. Finally, voxels could be chosen based on some combination of anatomy and functional response. In the articles we are focusing on here, the ï¬nal result, as we have seen, was always a correlation valueâa correlation between each personâs score on some behavioral measure and some summary statistic of their brain activation. The latter summary statistic reï¬ects the activation or activation contrast within a certain set of voxels. In either case, the critical question is, ââHow was this set of voxels selected?ââ As we have seen, voxels may be selected based on anatomical criteria, functional criteria, or both. Within these broad options, there are a number of additional more ï¬ne-grained choices. It is hardly surprising, then, that brief method sections rarely sufï¬ce to describe how the analyses were done in adequate detail to really understand what choices were being made. We display the raw results from our survey as the proportion of studies that investigators described with a particular answer to each question (see Fig. 2). As some questions only applied to a subset of participants, we display only the proportion of the relevant subset of studies. The raw answers to our survey do not by themselves explain how respondents arrived at the (implausibly high, or so we have argued) correlations. The key, we believe, lies in the 53% of respondents who said that ââregression across subjectsââ was the functional constraint used to select voxels, indicating that voxels were selected because they correlated highly with the behavioral measure of interest. Figure 3 shows very concretely the sequence of steps that these respondents reported following when analyzing their data. A separate correlation across subjects was performed for each voxel within a speciï¬ed brain region. Each correlation relates some measure of brain activity in that voxel (which might be a difference between responses in two tasks or in two conditions) with the behavioral measure for that individual. Thus, thenumber of correlations computed was equal to the number of voxels, meaning that thousands of correlations were computed in many cases. At the next stage, researchers selected the set of voxels for which this correlation exceeded a certain threshold, and reported the correlation within this set of voxels. What are the implications of selecting voxels in this fashion? Such an analysis will inï¬ate observed across-subject correlations and can even produce signiï¬cant measures out of pure noise. The problem is illustrated in the simple simulation displayed in Figure 4. First, the investigator computes a separate correlation of the behavioral measure of interest with each of the voxels (Fig. 4a). Then, he or she selects those voxels that exhibited a sufï¬ciently high correlation (by passing a statistical threshold; Fig. 4b). Finally, an ostensible measure of the ââtrueââ correlation is aggregated from the voxels that showed high correlations (e.g., by taking the mean of the voxels over the threshold). With enough voxels, such a biased analysis is guaranteed to produce high correlations even if none are truly present (Fig. 4). Moreover, this analysis will produce visually pleasing scattergrams (e.g., Fig. 4c) that will provide (quite meaningless) reassurance to the viewer that s/he is looking at a result that is solid, is âânot driven by outliers,ââ and so on. . On Sun, Feb 19, 2012 at 3:09 PM, Larry C. Lyons <[email protected]> wrote: > > I've been stying out of this discussion mainly because I've been reading > the article that Sam provided a link to.  I'm still going through the > article, but first and foremost it has nothing to do with the topic at > hand. Vui et al were looking at the unexpectedly high correlations between > personality traits and vartious systems within the brain. Last I checked > the psychological constructs known as personality traits are not political > orientation. Even someone who has taken an intro to psychology course at > the high school level should have picked up that one. Did ytou actually > read the article Sam? > > Second the main thrust of the criticism was how these personality traits > were correlated with problematic measures of brain activation. The study I > mentioned looked at the differences between specific groups. In  other > words a very different form of statistical analysis was used. This > meta-analysis you cited does not address that. > > There are other criticisms I could make about the methodological approach > the authors take. However everyone else would find it completely boring. I > also think that this study is important for entirely different reasons than > Sam gave and those reasons are not at all relevant to this discussion, > mainly involving recent research in the neuophysiology of personality. > > In a nutshell Sam you need to actually read and attempt to understand the > material you present in cases like these. That way you do not come across > with so much egg on your face. > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-community/message.cfm/messageid:347121 Subscription: http://www.houseoffusion.com/groups/cf-community/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-community/unsubscribe.cfm
