Hi Donna, thank you so much for your detailed reply. It helped a great deal. Three things remain somewhat unclear to me:
1. I think I figured out why the list of permuted clusters in xx.metric_TMap_Significant_Clusters.txt only has 696 instead of 1000 (=number of iterations) columns in the example contrast. My input columns were n=13, which would allow up to 2^13=8192 iterations. But only 696 of the 1000 iterations produce clusters with a T-Value above my defined threshold. Theses are the ones listed in the output file. Is that possible? 2. The significant cluster in the example has an area-corrected value of 1242.687012 and is assigned a p-value of 0.036. This means that it resides on rank 36 out of my 1000 iterations, right? When I looked up the 100 largest clusters list (my predefined P-threshold was .1) in my output file the significant cluster would actually take rank 38. what am I getting wrong here? 3.The correction for multiple comparisons is done by thresholding all real T-map clusters above the predefined T-Value with the smallest iterations*alpha cluster of the permutation distribution according to area-corrected? The corrected P-value for real T-map clusters that survive this threshold is derived from the rank the clusters hold in the total iterations list according to area-corrected? Again, thanks a lot for your answers!! Julia > Hi Julia, > > Bear with me, because it's been years since I used the caret_command > tests, which are mostly cluster tests. We switched from cluster to TFCE > a year or so ago: > > http://brainvis.wustl.edu/wiki/index.php/Caret:Documentation:Statistics#Threshold-Free_Cluster_Enhancement_.28TFCE.29 > > But those tests are in caret_stats, which is a separate tool/package > based on java. Let me know if you want to know more about that. > > My inline replies below reflect my best recollection of the cluster tests. > > Donna > > On 10/08/2010 08:16 AM, Julia Bender wrote: >> Hi Donna, >> >> I've looked more into the "caret_command >> -metric-statistics-one-sample-t-test" output files. I'm having trouble >> understanding what all the information means. Maybe you can correct me: >> >> 1. xx.metric_TMap.metric: >> Map of T-Values for the wholebrain (?) for the contrast defined in the >> xx.metric input files. Those are T-Values exceeding the T-Value >> thresholds >> and alpha level I specified in >> "caret_command-metric-statistics-one-sample-t-test" (eg -300000 2.5 0.1 >> in >> my case). > They are not thresholded. The 2.5 and 0.1 affect the cluster size, > which affects downstream outputs. But this TMap is unthresholded. >> They are not corrected for multiple comparisons(?). > No, definitely not. >> This is the one I should load onto my surface. When I load >> that TMap, why can I still see T-Values below my defined threshold? >> > Right. I like to use the unthresholded t/f-map for my figures, but > generate a border around the clusters that were significant, and show > the border overlaid on the unthresholded t-map. >> 2.xx.metric_ShuffledTMap.metric: >> Distribution of T-Values derived from permuting + and - on each element >> in >> xx in N iterations (in my case 1000) >> > Right, and less than 1000, depending on how many columns there are in > your input composite metric. >> 3.xx.metric_TMap_Significant_Clusters.txt: >> This is what the help page says: >> "1. Find the biggest cluster in each column of the permutation T-Map >> metric/shape file and sort them by cluster size." >> I see two lists of clusters in the output file. I assume the one that is >> the result of this sorting is the lower one. It has about 700 rows >> depending on xx, why does it not have 1000 rows, one for each >> permutation? >> > See Sample Report: Two Sample T-Test here: > > http://brainvis.wustl.edu/OLD/courses/stats_neurosci/2008_BMEcourse/BME_dld_talk.htm > > Yours is a one-sample t-test, but I think the cluster lists will be the > same. > > If your last list of clusters has less than 1000 rows, then you had > fewer than 10 columns in your input composite metric. If n is the > number of columns, and 2 raised to the n is less than your input > iterations, then Caret will stop at 2^n. >> Are the clusters sorted by Num-Nodes, Area or Area corrected? > Descending area-corrected. >> In my files >> they seem kind of sorted by both... How is a cluster defined? >> > By your input thresholds. The permuted t-maps are thresholded at the > level specified, and then clusters of contiguous supra-threshold nodes > are found. Only the largest in each iteration is saved. Then they are > listed in descending order of area-corrected size. >> "2. Find the largest (alpha)(iterations) clusters in the Permutation >> T-Map >> and use its cluster size as the Significant Cluster Cutoff." >> I assume this is the list of T-Values right below the above list of >> clusters, why does it contain clusters with a P-Value that is above my >> defined alpha? >> > The last table lists all the largest clusters for each iteration, > regardless of its p. The second table is what you want. >> "3. Find clusters in the Real T-Map file." This must be the upper list >> of >> clusters (containing much less rows than the lower one) >> > Correct. That link above shows this the best. >> "4. Report all clusters in Real T-Map file that are larger than >> Significant Cluster Cutoff." This is the list of T-Values below this >> list, >> containing only clusters that are bigger than the cluster with the >> highest >> P-Value found in 2. that pass the alpha and T-Value thresholds I >> specified >> in "caret_command-metric-statistics-one-sample-t-test". >> > Again, here is the relevant excerpt from the link above: > > Significant Area: 306.226 <--- area of smallest cluster listed in next > section > > Shuffled TMap <--- Top alpha*iterations biggest clusters are listed below, > in descending area-corrected sequence. > (i.e., the smallest of which determines the signifiance area cut-off) > ------------- > Column Thresh Num-Nodes Area Area-Corrected COG-X > COG-Y COG-Z P-Value > 821 2.660 2300 1439.637207 1559.143188 -37.249 > 0.007 -3.456 > 150 2.660 792 598.519104 858.216492 -49.142 > -32.369 11.918 > 548 2.660 1380 641.563843 643.790466 -35.004 > 2.519 -14.981 > ... (middle biggest alpha*iterations entries omitted) > 681 -2.660 279 249.739059 312.872681 -48.189 > -9.576 10.352 > 649 2.660 237 198.468857 311.602844 -16.170 > -79.223 30.149 > 633 -2.660 217 119.337837 306.226257 -42.479 > -48.846 41.505 > > > TMap <--- Significant real tmap clusters (i.e., >= significant area) are > listed here; no entries here means no clusters were significant. > ---- > Column Thresh Num-Nodes Area Area-Corrected COG-X > COG-Y COG-Z P-Value > 3 2.660 344 229.353973 362.322113 -45.677 > -27.442 18.039 0.029000 > > Shuffled TMap <--- All iterations max clusters are listed below, in > descending area-corrected sequence. > ------------- > Column Thresh Num-Nodes Area Area-Corrected COG-X > COG-Y COG-Z P-Value > 821 2.660 2300 1439.637207 1559.143188 -37.249 > 0.007 -3.456 > 150 2.660 792 598.519104 858.216492 -49.142 > -32.369 11.918 > 548 2.660 1380 641.563843 643.790466 -35.004 > 2.519 -14.981 > ... (middle biggest alpha*iterations entries omitted) > 3 2.660 1 0.475079 0.472114 -18.520 > -24.418 -21.858 0.999000 > 3 -2.660 1 0.474738 0.456814 -21.478 > -36.579 -17.077 0.999000 > 3 2.660 1 0.000000 0.000000 -6.822 > -46.751 8.581 0.999000 > > ------------------------------------------------------------------------ > >> 4. T-Map_LH_cCue_EndoL_vs_Cue_ExoL.metric_TMapClusters.metric >> Is this the map of clusters defined above? >> > This is what I believe you thought xx.metric_TMap.metric was, but it is > unthresholded. > > This TMapClusters one zeroes out all nodes that are NOT within a > significant cluster. >> I'm sorry, I know these are a many questions. Thanks a lot for your >> help! >> >> Julia >> >> >> >> >>> Julia, >>> I looked at your report and your t-map, which is consistent with the >>> >> caret_command -metric-information output you included below. >> >>> Just making sure you understand this part of the report: >>> TMap >>> ---- >>> Column Thresh Num-Nodes Area Area-Corrected COG-X >>> >> COG-Y COG-Z P- >> >>> Value >>> 1 2.500 3064 2223.228027 2245.943848 -30.317 >>> -73.796 -12.228 0.0 >>> 12000 >>> 1 2.500 3372 1865.999878 1863.883423 -21.516 >>> -62.842 46.382 0.0 >>> 17000 >>> 1 2.500 1557 681.308838 674.981384 -32.967 >>> -5.873 48.701 0.0 >>> 59000 >>> These are the clusters in your real t-map that were significant at the >>> >> 0.1 >> >>> alpha you specified, using the 2.5 threshold. (Note that all the >>> >> significant clusters were at the positive end.) >> >>> I believe the reason you saw different max/min in the Caret GUI was >>> that >>> >> you had the permuted t-map loaded, instead of the real one. In your >> message below, you said, "Adjustment:Column: permuted T-Values,Threshold >> type". There is nothing about permuted in the file you uploaded. If >> you >> >>> were viewing the permuted/shuffled t-map, this would also explain why >>> >> little would survive a low threshold. >> >>> But we don't necessarily (or even usually) use the same values we used >>> >> for >> >>> cluster thresholds as the threshold for displaying t-maps, e.g., for >>> >> publication purposes. I think we like to see some color differentiation >> beyond the cluster threshold max. If they are the same, the color will >> saturate at the max. Sometimes we'll use a p-value derived from the >> degrees of freedom and get a corresponding t-value from that, and use >> that >> >>> for thresholding. Other times we might just use, say, +/-4.0 or >>> higher, >>> >> depending on how big the values get in the data. Usually we'll use a >> symmetric scale (i.e., -x to +x -- rather than different min/max). Donna >> >>> On 08/11/2010 09:12 AM, Julia Bender wrote: >>> >>>> Hi Donna, >>>> I've just uploaded the two files. >>>> Thanks for your help! >>>> Julia >>>> >>>>> Julia, >>>>> It will be easier for me to get my head around your question if I can >>>>> >> get two files: >> >>>>> * T-Map_LH_cCue_Endo.metric_TMap.metric (whatever the final output >>>>> >> metric was, but NOT the permuted/shuffled tmap file). >> >>>>> * The report named something like *Signicance*.txt >>>>> Could you upload those here: >>>>> http://pulvinar.wustl.edu/cgi-bin/upload.cgi >>>>> My brain would be ever so grateful. >>>>> Donna >>>>> On 08/11/2010 07:07 AM, Julia Bender wrote: >>>>> >>>>>> Hey everyone, >>>>>> I'm a bit confused about how to threshold my T-Maps in caret5. I >>>>>> >> created >> >>>>>> the maps with the following command: >>>>>> /usr/local/caret/bin_linux/caret_command >>>>>> -metric-statistics-one-sample-t-test $EACHMETRIC $FIDUCIAL_COORD >>>>>> >> $OPEN_TOPO $SURFACE_SHAPE 3 T-Map_$EACHMETRIC -300000.0 2.5 0.10 10 1 >> 1000 >> >>>>>> 0 4 >>>>>> So I put the negative threshold to -300000 and the positive >>>>>> threshold >>>>>> >> to >> >>>>>> 2.5. When I look at the resulting Tmap.metric files it gives me >>>>>> >> something >> >>>>>> like this: >>>>>> Filename: T-Map_LH_cCue_Endo.metric_TMap.metric >>>>>> Number of Nodes: 73730 >>>>>> Number of Columns: 1 >>>>>> Column Minimum Maximum Mean Sample Dev % >>>>>> >> Positive % Negative Column Name >> >>>>>> 1 -9.785 6.076 -0.950 2.639 >>>>>> 36.234 63.766 T-Values >>>>>> As far as I understand, this means the maximum negative T value in >>>>>> >> this >> >>>>>> metric is -9.785 and the maximum positive T value is 6.076. When I >>>>>> >> open >> >>>>>> the file in caret though (Color mapping: Auto Scale, Display mode: >>>>>> >> Both, >> >>>>>> Display Color Bar, Threshold Adjustment:Column: permuted >>>>>> T-Values,Threshold type: Column) the bar tells me that my maximum >>>>>> >> negative >> >>>>>> value is -3.7 and my maximum positive value is 2.6. This also holds >>>>>> >> when >> >>>>>> I >>>>>> adjust the thresholds in the fields below to -2 and 2, when almost >>>>>> >> all >> >>>>>> activation disappears. >>>>>> Which is the correct information? >>>>>> Thanks a lot for your help! >>>>>> Julia >>>>>> Dipl. Psych. Julia Bender >>>>>> Humboldt Universität zu Berlin >>>>>> Mathematisch - Naturwissenschaftliche Fakultät II >>>>>> Institut für Psychologie, Abt. Klinische Psychologie >>>>>> Unter den Linden 6 >>>>>> D-10099 Berlin >>>>>> > > _______________________________________________ > caret-users mailing list > [email protected] > http://brainvis.wustl.edu/mailman/listinfo/caret-users > Dipl. Psych. Julia Bender Humboldt Universit�t zu Berlin Mathematisch - Naturwissenschaftliche Fakult�t II Institut f�r Psychologie, Abt. Klinische Psychologie Unter den Linden 6 D-10099 Berlin _______________________________________________ caret-users mailing list [email protected] http://brainvis.wustl.edu/mailman/listinfo/caret-users
