I have been doing simulations similar to #1 (Eklund) using surface-based analysis on both thickness and fMRI. I'll prepare a report of the results, but the early indications are that the same effect is in play, though it does not look like the effects are as bad as in Eklund.
For thickness analysis using applied smoothing of 5 or 10 mm FWHM, for a voxel-wise threshold of .001, the false positives are appropriate (ie, 5%). For a voxel-wise threshold of .01, the false positives is only a little off (about 7%); for a voxel-wise threshold of .05, the FPR is about 13%. If the data are not smoothed at all, then the false positive rates go way up. The reason appears to be the same as found in Eklund (ie, the autocorrelation function has a heavier-than-Gaussian tail). I did the analysis by randomly selecting 40 subjects from a homogeneous data set of 809 subjects aged 18-25. I then made two groups of 20 subjects each and ran a two-group test, then found clusters significant based on our Monte Carlo (Gaussian) simulations. I repeated this several thousand times. Any significant clusters were interpreted as false positives. These results are much better than Eklund, but Eklund was analyzing fMRI data. I'm still working on the fMRI data. It is much more complicated because the results depend on the assumed stimulus schedule (eg, 10 sec blocks vs 30 sec blocks) and whether a one-group or two-group anaysis is done; nuisance variables also play a role. At very low cluster-forming thresholds (ie, .05), the FPR is roughly 20-30%. At a threshold of .01, the FPR is about 3-13%. At a threshold of .001 are about 1-6%. This is all for an applied smoothing level of 5mm. All of these results are preliminary, so don't take them as true and established yet. As a reminder, you can always do a permutation test using mri_glmfit-sim. Eklund found that permutation did pretty well in most cases. doug On 8/2/16 12:43 AM, Ajay Kurani wrote: > Hello Freesurfer Experts, > Recently there were two article published regarding clusterwise > simulations for volumetric fmri analyses and potential errors for > underestimating clusterwise extent thresholds. > > 1) http://www.pnas.org/content/113/28/7900.full.pdf?with-ds=yes > 2) biorxiv.org/content/early/2016/07/26/065862 > <http://biorxiv.org/content/early/2016/07/26/065862> > > One issue pointed out from these articles seems software specific, > however the second issue is determining the proper clustersize. The > heavy-tail nature of spatial smoothness seems to be ignored and a > gaussian shape is generally assumed, leading to an underestimation of > the spatial smoothness which can affect cluster size calculations. > The issues are highlighted in the second article above. > > I created my own monte carlo simulation in Freesurfer for a specific > brain template and I wanted to find out if these concerns also apply > to my surface based simulations? I am not sure if it does since the > monte carlo tool is a GRF simulation as opposed to an analytic > equation, however given that these articles were highlighted very > recently, I wanted to ensure I am running things appropriately for > surface based cortical thickness/dti analyses. > > Thanks, > Ajay > > > _______________________________________________ > Freesurfer mailing list > Freesurfer@nmr.mgh.harvard.edu > https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer _______________________________________________ Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance HelpLine at http://www.partners.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.
