Hi Casper, Yes: With Caret, the same test/command that creates the *real* t- or f-map also creates the randomized ones. The most typical permutation method is on the group, but in the case of a paired t-test, for example, it's the sign of some randomly determined subjects/columns that gets sign-flipped.
Whatever test/statistic you compute for the real t- or f-map gets computed for the randomly split groups. For a cluster test, the user-specified threshold is applied (both positive and negative, usually) to both the real and randomized t- or f-maps. The size of the largest cluster in each randomized map is stored and a distribution of max clusters generated. The 95th percentile is typically the significance cut-off. Note, too, that you can get around the need for group or other permutation by using something like False Discovery Rate (FDR), which I'd guess AFNI/SUMA can compute for 1D files. FDR is different from FWER (family-wise error rate, which is what the cluster method gives you), and in my very limited experience it has not proven more sensitive (possibly because high-res surface data is smoother, which makes FDR more conservative?). But I thought it worth mentioning, since it has the benefit of not needing permuations. ;-) Donna On 11/19/2009 09:19 AM, Oers, C.A.M. van wrote: > Hi Donna, > > Thanks for the explanation. > > So you are saying that Caret cannot create these permuted f-maps, since > the statistical model needs to be applied to create the f-map, but also > to create the permuted f-maps. I would have to create them myself using > the statistical model, right? > How are the permuted f-maps created? What permutation is used? > > Thanks again, > Casper > > > -----Oorspronkelijk bericht----- > Van: [email protected] > [mailto:[email protected]] Namens Donna Dierker > Verzonden: maandag 16 november 2009 17:00 > Aan: Caret, SureFit, and SuMS software users > Onderwerp: Re: [caret-users] multiple comparison correction on SUMA data > > On 11/16/2009 05:27 AM, Oers, C.A.M. van wrote: > >> Hi, >> >> Although I saw a post by Donna Dierker regarding the subject, I am >> afraid I still have a question. >> >> I have created SUMA files using a mixed model analysis. The resulting >> files have an individual statistical threshold (and coefficient). I >> would like to use Caret to do a multiple comparison correction with a >> cluster size threshold. >> > I'm not sure how straightforward this will be, due to the randomization > component. The caret_command utility has these features: > > caret_command -metric-statistics-anova-one-way > caret_command -metric-statistics-anova-two-way > caret_command -metric-statistics-coordinate-difference > caret_command -metric-statistics-interhemispheric-clusters > caret_command -metric-statistics-kruskal-wallis > caret_command -metric-statistics-levene-map > caret_command -metric-statistics-normalization > caret_command -metric-statistics-one-sample-t-test > caret_command -metric-statistics-paired-t-test > caret_command -metric-statistics-shuffled-cross-correlation-map > caret_command -metric-statistics-shuffled-t-map > caret_command -metric-statistics-subtract-group-average > caret_command -metric-statistics-t-map > caret_command -metric-statistics-two-sample-t-test > caret_command -metric-statistics-z-map > > Enter the command for the help for that feature, or enter "caret_command > -help-ful > caret_command.txt" to get a text file with all the help. > > Let's choose the case of a two-sample-t-test. In this case, your > caret_command line will look like so: > > DISTCOL=1 > ITERATIONS=2500 > THRESHNEG=-2.68 > THRESHPOS=2.68 > P_VALUE=0.05 > VAR_SMOOTH_ITERATIONS=0 > VAR_SMOOTH_STRENGTH=0.0 > DO_TMAP_DOF=true > DO_TMAP_PVALUE=true > THREADS=4 > > FIDUCIAL=Human.PALS_B12.LEFT_AVG_B1-12.FIDUCIAL.clean.align.73730.coord > OPENTOPO=Human.sphere_6.LEFT_HEM_OPEN.73730.topo > DISTMETRIC=Human.PALS_B12.B1-12_LEFT_DISTORTION-vs-AVG-FIDUCIAL_ONLY.737 > 30.metric > COMPOSITE_IN_A=Composite.DEPTH.LEFT.KIDS.73730.surface_shape > COMPOSITE_IN_B=Composite.DEPTH.LEFT.ADULTS.73730.surface_shape > OUT_PREFIX=KIDS29vADULTS23_Depth.LEFT > COMMAND="caret_command -metric-statistics-two-sample-t-test NO_TRANSFORM > UNPOOLED" > COMMAND="$COMMAND $COMPOSITE_IN_A $COMPOSITE_IN_B" > COMMAND="$COMMAND $FIDUCIAL $OPENTOPO $DISTMETRIC $DISTCOL $OUT_PREFIX" > COMMAND="$COMMAND $ITERATIONS $THRESHNEG $THRESHPOS $P_VALUE" > COMMAND="$COMMAND $VAR_SMOOTH_ITERATIONS $VAR_SMOOTH_STRENGTH" > COMMAND="$COMMAND $DO_TMAP_DOF $DO_TMAP_PVALUE $THREADS" > $COMMAND > > These sample scripts (e.g., tstt.sh and gen_composites.sh) are here: > > http://brainmap.wustl.edu/pub/donna/WUSTL/BURTON/SCRIPTS/SBM/ > login pub > password download > > But note that this takes two composite shape files (imagine your 1D > files concatenated together in one file, with one 1D column per subject, > with the node number preceding all os them). I don't think generating > composites is hard (see below about adding header). > > Rather, the tricky bit here is that you already have your stat -- not > raw scores/activations. The example above uses sulcal depth scores, but > this could be any measure. But if it is already a stat, then it's less > clear to me how to randomize it. With the two-sample t-test, a t-map is > generated using the two groups' composites. Then, the composites are > combined and group memberships randomized to create permuted t-maps. > The max size of each permuted map is stored to create a distribution, > and the size of the 95th percentile is used as the significance cut-off. > > If you've got stats already, how are we randomizing? > > If you can create your own permuted t- or f-maps, as well as your own > real t- or f-map, then there is a newer caret_stats feature that can > take these real and permuted maps and do either cluster operations or > TFCE on them. > > But in your case, it isn't clear to me where/how the randomizing comes > in. > > >> >> How can I convert a 1D surface file into a metric or surface_shape >> > file? > This part is easy: > > 1. Add a node number in column 1, using matlab or some other tool. > > 2. Slap on a header as shown in > http://brainvis.wustl.edu/CaretHelpAccount/caret5_help/file_formats/file > _formats.html#metricFile: > > metric-version 2 > tag-number-of-nodes 71723 > tag-number-of-columns 2 > tag-title untitled > tag-column-name 0 Depth > tag-column-name 1 Smoothed Depth > tag-column-color-mapping 0 -1.000000 1.000000 tag-BEGIN-DATA 0 1.803019 > -0.045549 > 1 1.704132 -0.007309 > 2 1.523951 0.026103 > ... > 71722 .822899 0.112485 > > If you can generate metric/surface_shape files from all your 1D files, > then it's not hard to generate a composite using something like the > gen_composites.sh script mentioned above). > >> Which way to perform the correction? >> >> I appreciate the help, >> Casper >> ---------------------------------------------------------------------- >> -- >> >> /De informatie opgenomen in dit bericht kan vertrouwelijk zijn en is >> uitsluitend bestemd voor de geadresseerde. Indien u dit bericht >> onterecht ontvangt, wordt u verzocht de inhoud niet te gebruiken en de >> > > >> afzender direct te informeren door het bericht te retourneren. Het >> Universitair Medisch Centrum Utrecht is een publiekrechtelijke >> rechtspersoon in de zin van de W.H.W. (Wet Hoger Onderwijs en >> Wetenschappelijk Onderzoek) en staat geregistreerd bij de Kamer van >> Koophandel voor Midden-Nederland onder nr. 30244197. / >> >> / Denk s.v.p aan het milieu voor u deze e-mail afdrukt. / >> >> ---------------------------------------------------------------------- >> -- >> >> /This message may contain confidential information and is intended >> exclusively for the addressee. If you receive this message >> unintentionally, please do not use the contents but notify the sender >> immediately by return e-mail. University Medical Center Utrecht is a >> legal person by public law and is registered at the Chamber of >> Commerce for Midden-Nederland under no. 30244197. / >> >> / Please consider the environment before printing this e-mail. / >> >> ---------------------------------------------------------------------- >> -- >> >> _______________________________________________ >> caret-users mailing list >> [email protected] >> http://brainvis.wustl.edu/mailman/listinfo/caret-users >> >> > > _______________________________________________ > caret-users mailing list > [email protected] > http://brainvis.wustl.edu/mailman/listinfo/caret-users > > _______________________________________________ > caret-users mailing list > [email protected] > http://brainvis.wustl.edu/mailman/listinfo/caret-users > _______________________________________________ caret-users mailing list [email protected] http://brainvis.wustl.edu/mailman/listinfo/caret-users
