On 11/04/2010 09:57 AM, Julia Bender wrote: > Hi Donna, > > thank you so much for your detailed reply. It helped a great deal. > Three things remain somewhat unclear to me: > > 1. I think I figured out why the list of permuted clusters in > xx.metric_TMap_Significant_Clusters.txt only has 696 instead of 1000 > (=number of iterations) columns in the example contrast. My input columns > were n=13, which would allow up to 2^13=8192 iterations. But only 696 of > the 1000 iterations produce clusters with a T-Value above my defined > threshold. Theses are the ones listed in the output file. Is that > possible? > Sure, it certainly is possible for the data to not survive threshold, but I'm not used to it. > 2. The significant cluster in the example has an area-corrected value of > 1242.687012 and is assigned a p-value of 0.036. This means that it resides > on rank 36 out of my 1000 iterations, right? When I looked up the 100 > largest clusters list (my predefined P-threshold was .1) in my output file > the significant cluster would actually take rank 38. what am I getting > wrong here? > Keep in mind that the p-values are based on the distribution built by the random tmaps, but the area of clusters on the real tmap is typically in between two areas on the randomized list.
If your real tmap cluster is bigger than the 38th biggest random cluster, but smaller than the 37th, then I'd expect it to have a p-value of 38/iterations. If this is not happening, I don't know why. Upload your significance report, and I'll have a look: http://pulvinar.wustl.edu/cgi-bin/upload.cgi > 3.The correction for multiple comparisons is done by thresholding all real > T-map clusters above the predefined T-Value with the smallest > iterations*alpha cluster of the permutation distribution according to > area-corrected? The corrected P-value for real T-map clusters that survive > this threshold is derived from the rank the clusters hold in the total > iterations list according to area-corrected? > I would put it differently. Any clusters in the real tmap surviving the threshold that exceed the minimum significance cut-off in corrected area are significant. > Again, thanks a lot for your answers!! > > Julia > > >> Hi Julia, >> >> Bear with me, because it's been years since I used the caret_command >> tests, which are mostly cluster tests. We switched from cluster to TFCE >> a year or so ago: >> >> http://brainvis.wustl.edu/wiki/index.php/Caret:Documentation:Statistics#Threshold-Free_Cluster_Enhancement_.28TFCE.29 >> >> But those tests are in caret_stats, which is a separate tool/package >> based on java. Let me know if you want to know more about that. >> >> My inline replies below reflect my best recollection of the cluster tests. >> >> Donna >> >> On 10/08/2010 08:16 AM, Julia Bender wrote: >> >>> Hi Donna, >>> >>> I've looked more into the "caret_command >>> -metric-statistics-one-sample-t-test" output files. I'm having trouble >>> understanding what all the information means. Maybe you can correct me: >>> >>> 1. xx.metric_TMap.metric: >>> Map of T-Values for the wholebrain (?) for the contrast defined in the >>> xx.metric input files. Those are T-Values exceeding the T-Value >>> thresholds >>> and alpha level I specified in >>> "caret_command-metric-statistics-one-sample-t-test" (eg -300000 2.5 0.1 >>> in >>> my case). >>> >> They are not thresholded. The 2.5 and 0.1 affect the cluster size, >> which affects downstream outputs. But this TMap is unthresholded. >> >>> They are not corrected for multiple comparisons(?). >>> >> No, definitely not. >> >>> This is the one I should load onto my surface. When I load >>> that TMap, why can I still see T-Values below my defined threshold? >>> >>> >> Right. I like to use the unthresholded t/f-map for my figures, but >> generate a border around the clusters that were significant, and show >> the border overlaid on the unthresholded t-map. >> >>> 2.xx.metric_ShuffledTMap.metric: >>> Distribution of T-Values derived from permuting + and - on each element >>> in >>> xx in N iterations (in my case 1000) >>> >>> >> Right, and less than 1000, depending on how many columns there are in >> your input composite metric. >> >>> 3.xx.metric_TMap_Significant_Clusters.txt: >>> This is what the help page says: >>> "1. Find the biggest cluster in each column of the permutation T-Map >>> metric/shape file and sort them by cluster size." >>> I see two lists of clusters in the output file. I assume the one that is >>> the result of this sorting is the lower one. It has about 700 rows >>> depending on xx, why does it not have 1000 rows, one for each >>> permutation? >>> >>> >> See Sample Report: Two Sample T-Test here: >> >> http://brainvis.wustl.edu/OLD/courses/stats_neurosci/2008_BMEcourse/BME_dld_talk.htm >> >> Yours is a one-sample t-test, but I think the cluster lists will be the >> same. >> >> If your last list of clusters has less than 1000 rows, then you had >> fewer than 10 columns in your input composite metric. If n is the >> number of columns, and 2 raised to the n is less than your input >> iterations, then Caret will stop at 2^n. >> >>> Are the clusters sorted by Num-Nodes, Area or Area corrected? >>> >> Descending area-corrected. >> >>> In my files >>> they seem kind of sorted by both... How is a cluster defined? >>> >>> >> By your input thresholds. The permuted t-maps are thresholded at the >> level specified, and then clusters of contiguous supra-threshold nodes >> are found. Only the largest in each iteration is saved. Then they are >> listed in descending order of area-corrected size. >> >>> "2. Find the largest (alpha)(iterations) clusters in the Permutation >>> T-Map >>> and use its cluster size as the Significant Cluster Cutoff." >>> I assume this is the list of T-Values right below the above list of >>> clusters, why does it contain clusters with a P-Value that is above my >>> defined alpha? >>> >>> >> The last table lists all the largest clusters for each iteration, >> regardless of its p. The second table is what you want. >> >>> "3. Find clusters in the Real T-Map file." This must be the upper list >>> of >>> clusters (containing much less rows than the lower one) >>> >>> >> Correct. That link above shows this the best. >> >>> "4. Report all clusters in Real T-Map file that are larger than >>> Significant Cluster Cutoff." This is the list of T-Values below this >>> list, >>> containing only clusters that are bigger than the cluster with the >>> highest >>> P-Value found in 2. that pass the alpha and T-Value thresholds I >>> specified >>> in "caret_command-metric-statistics-one-sample-t-test". >>> >>> >> Again, here is the relevant excerpt from the link above: >> >> Significant Area: 306.226 <--- area of smallest cluster listed in next >> section >> >> Shuffled TMap <--- Top alpha*iterations biggest clusters are listed below, >> in descending area-corrected sequence. >> (i.e., the smallest of which determines the signifiance area cut-off) >> ------------- >> Column Thresh Num-Nodes Area Area-Corrected COG-X >> COG-Y COG-Z P-Value >> 821 2.660 2300 1439.637207 1559.143188 -37.249 >> 0.007 -3.456 >> 150 2.660 792 598.519104 858.216492 -49.142 >> -32.369 11.918 >> 548 2.660 1380 641.563843 643.790466 -35.004 >> 2.519 -14.981 >> ... (middle biggest alpha*iterations entries omitted) >> 681 -2.660 279 249.739059 312.872681 -48.189 >> -9.576 10.352 >> 649 2.660 237 198.468857 311.602844 -16.170 >> -79.223 30.149 >> 633 -2.660 217 119.337837 306.226257 -42.479 >> -48.846 41.505 >> >> >> TMap <--- Significant real tmap clusters (i.e., >= significant area) are >> listed here; no entries here means no clusters were significant. >> ---- >> Column Thresh Num-Nodes Area Area-Corrected COG-X >> COG-Y COG-Z P-Value >> 3 2.660 344 229.353973 362.322113 -45.677 >> -27.442 18.039 0.029000 >> >> Shuffled TMap <--- All iterations max clusters are listed below, in >> descending area-corrected sequence. >> ------------- >> Column Thresh Num-Nodes Area Area-Corrected COG-X >> COG-Y COG-Z P-Value >> 821 2.660 2300 1439.637207 1559.143188 -37.249 >> 0.007 -3.456 >> 150 2.660 792 598.519104 858.216492 -49.142 >> -32.369 11.918 >> 548 2.660 1380 641.563843 643.790466 -35.004 >> 2.519 -14.981 >> ... (middle biggest alpha*iterations entries omitted) >> 3 2.660 1 0.475079 0.472114 -18.520 >> -24.418 -21.858 0.999000 >> 3 -2.660 1 0.474738 0.456814 -21.478 >> -36.579 -17.077 0.999000 >> 3 2.660 1 0.000000 0.000000 -6.822 >> -46.751 8.581 0.999000 >> >> ------------------------------------------------------------------------ >> >> >>> 4. T-Map_LH_cCue_EndoL_vs_Cue_ExoL.metric_TMapClusters.metric >>> Is this the map of clusters defined above? >>> >>> >> This is what I believe you thought xx.metric_TMap.metric was, but it is >> unthresholded. >> >> This TMapClusters one zeroes out all nodes that are NOT within a >> significant cluster. >> >>> I'm sorry, I know these are a many questions. Thanks a lot for your >>> help! >>> >>> Julia >>> >>> >>> >>> >>> >>>> Julia, >>>> I looked at your report and your t-map, which is consistent with the >>>> >>>> >>> caret_command -metric-information output you included below. >>> >>> >>>> Just making sure you understand this part of the report: >>>> TMap >>>> ---- >>>> Column Thresh Num-Nodes Area Area-Corrected COG-X >>>> >>>> >>> COG-Y COG-Z P- >>> >>> >>>> Value >>>> 1 2.500 3064 2223.228027 2245.943848 -30.317 >>>> -73.796 -12.228 0.0 >>>> 12000 >>>> 1 2.500 3372 1865.999878 1863.883423 -21.516 >>>> -62.842 46.382 0.0 >>>> 17000 >>>> 1 2.500 1557 681.308838 674.981384 -32.967 >>>> -5.873 48.701 0.0 >>>> 59000 >>>> These are the clusters in your real t-map that were significant at the >>>> >>>> >>> 0.1 >>> >>> >>>> alpha you specified, using the 2.5 threshold. (Note that all the >>>> >>>> >>> significant clusters were at the positive end.) >>> >>> >>>> I believe the reason you saw different max/min in the Caret GUI was >>>> that >>>> >>>> >>> you had the permuted t-map loaded, instead of the real one. In your >>> message below, you said, "Adjustment:Column: permuted T-Values,Threshold >>> type". There is nothing about permuted in the file you uploaded. If >>> you >>> >>> >>>> were viewing the permuted/shuffled t-map, this would also explain why >>>> >>>> >>> little would survive a low threshold. >>> >>> >>>> But we don't necessarily (or even usually) use the same values we used >>>> >>>> >>> for >>> >>> >>>> cluster thresholds as the threshold for displaying t-maps, e.g., for >>>> >>>> >>> publication purposes. I think we like to see some color differentiation >>> beyond the cluster threshold max. If they are the same, the color will >>> saturate at the max. Sometimes we'll use a p-value derived from the >>> degrees of freedom and get a corresponding t-value from that, and use >>> that >>> >>> >>>> for thresholding. Other times we might just use, say, +/-4.0 or >>>> higher, >>>> >>>> >>> depending on how big the values get in the data. Usually we'll use a >>> symmetric scale (i.e., -x to +x -- rather than different min/max). Donna >>> >>> >>>> On 08/11/2010 09:12 AM, Julia Bender wrote: >>>> >>>> >>>>> Hi Donna, >>>>> I've just uploaded the two files. >>>>> Thanks for your help! >>>>> Julia >>>>> >>>>> >>>>>> Julia, >>>>>> It will be easier for me to get my head around your question if I can >>>>>> >>>>>> >>> get two files: >>> >>> >>>>>> * T-Map_LH_cCue_Endo.metric_TMap.metric (whatever the final output >>>>>> >>>>>> >>> metric was, but NOT the permuted/shuffled tmap file). >>> >>> >>>>>> * The report named something like *Signicance*.txt >>>>>> Could you upload those here: >>>>>> http://pulvinar.wustl.edu/cgi-bin/upload.cgi >>>>>> My brain would be ever so grateful. >>>>>> Donna >>>>>> On 08/11/2010 07:07 AM, Julia Bender wrote: >>>>>> >>>>>> >>>>>>> Hey everyone, >>>>>>> I'm a bit confused about how to threshold my T-Maps in caret5. I >>>>>>> >>>>>>> >>> created >>> >>> >>>>>>> the maps with the following command: >>>>>>> /usr/local/caret/bin_linux/caret_command >>>>>>> -metric-statistics-one-sample-t-test $EACHMETRIC $FIDUCIAL_COORD >>>>>>> >>>>>>> >>> $OPEN_TOPO $SURFACE_SHAPE 3 T-Map_$EACHMETRIC -300000.0 2.5 0.10 10 1 >>> 1000 >>> >>> >>>>>>> 0 4 >>>>>>> So I put the negative threshold to -300000 and the positive >>>>>>> threshold >>>>>>> >>>>>>> >>> to >>> >>> >>>>>>> 2.5. When I look at the resulting Tmap.metric files it gives me >>>>>>> >>>>>>> >>> something >>> >>> >>>>>>> like this: >>>>>>> Filename: T-Map_LH_cCue_Endo.metric_TMap.metric >>>>>>> Number of Nodes: 73730 >>>>>>> Number of Columns: 1 >>>>>>> Column Minimum Maximum Mean Sample Dev % >>>>>>> >>>>>>> >>> Positive % Negative Column Name >>> >>> >>>>>>> 1 -9.785 6.076 -0.950 2.639 >>>>>>> 36.234 63.766 T-Values >>>>>>> As far as I understand, this means the maximum negative T value in >>>>>>> >>>>>>> >>> this >>> >>> >>>>>>> metric is -9.785 and the maximum positive T value is 6.076. When I >>>>>>> >>>>>>> >>> open >>> >>> >>>>>>> the file in caret though (Color mapping: Auto Scale, Display mode: >>>>>>> >>>>>>> >>> Both, >>> >>> >>>>>>> Display Color Bar, Threshold Adjustment:Column: permuted >>>>>>> T-Values,Threshold type: Column) the bar tells me that my maximum >>>>>>> >>>>>>> >>> negative >>> >>> >>>>>>> value is -3.7 and my maximum positive value is 2.6. This also holds >>>>>>> >>>>>>> >>> when >>> >>> >>>>>>> I >>>>>>> adjust the thresholds in the fields below to -2 and 2, when almost >>>>>>> >>>>>>> >>> all >>> >>> >>>>>>> activation disappears. >>>>>>> Which is the correct information? >>>>>>> Thanks a lot for your help! >>>>>>> Julia >>>>>>> Dipl. Psych. Julia Bender >>>>>>> Humboldt Universität zu Berlin >>>>>>> Mathematisch - Naturwissenschaftliche Fakultät II >>>>>>> Institut für Psychologie, Abt. Klinische Psychologie >>>>>>> Unter den Linden 6 >>>>>>> D-10099 Berlin >>>>>>> _______________________________________________ caret-users mailing list [email protected] http://brainvis.wustl.edu/mailman/listinfo/caret-users
