Julia,
I see what you mean. Based on the report you uploaded, the p-values
listed here seem off by .002:
1 2.500 1712 1229.060425 1242.687012 -27.858
-75.308 -13.617 0.036000
1 2.500 1960 1059.480713 1062.812378 -21.474
-62.266 50.506 0.044000
I'll ask John about it.
Donna
On 11/08/2010 04:07 AM, Julia Bender wrote:
> Hi Donna,
>
> thanks for your answers. I've uploaded
> "T-Map_LH_cCue_EndoLeft.metric_TMap_Significant_Clusters.txt" which is the
> example file we've been talking about. As far as I see it the cluster
>
> TMap
> ----
> Column Thresh Num-Nodes Area Area-Corrected COG-X
> COG-Y COG-Z P-Value
> 1 2.500 1712 1229.060425 1242.687012 -27.858
> -75.308 -13.617 0.036000
>
> fits in between rank 37 (area corrected 1273.066772) and 38
> (area-corrected 1240.362915).
>
> Thanks for your help,
>
> Julia
>
>
>
>
>> On 11/04/2010 09:57 AM, Julia Bender wrote:
>>
>>> Hi Donna,
>>>
>>> thank you so much for your detailed reply. It helped a great deal.
>>> Three things remain somewhat unclear to me:
>>>
>>> 1. I think I figured out why the list of permuted clusters in
>>> xx.metric_TMap_Significant_Clusters.txt only has 696 instead of 1000
>>> (=number of iterations) columns in the example contrast. My input
>>> columns
>>> were n=13, which would allow up to 2^13=8192 iterations. But only 696 of
>>> the 1000 iterations produce clusters with a T-Value above my defined
>>> threshold. Theses are the ones listed in the output file. Is that
>>> possible?
>>>
>>>
>> Sure, it certainly is possible for the data to not survive threshold,
>> but I'm not used to it.
>>
>>> 2. The significant cluster in the example has an area-corrected value of
>>> 1242.687012 and is assigned a p-value of 0.036. This means that it
>>> resides
>>> on rank 36 out of my 1000 iterations, right? When I looked up the 100
>>> largest clusters list (my predefined P-threshold was .1) in my output
>>> file
>>> the significant cluster would actually take rank 38. what am I getting
>>> wrong here?
>>>
>>>
>> Keep in mind that the p-values are based on the distribution built by
>> the random tmaps, but the area of clusters on the real tmap is typically
>> in between two areas on the randomized list.
>>
>> If your real tmap cluster is bigger than the 38th biggest random
>> cluster, but smaller than the 37th, then I'd expect it to have a p-value
>> of 38/iterations.
>>
>> If this is not happening, I don't know why. Upload your significance
>> report, and I'll have a look:
>>
>> http://pulvinar.wustl.edu/cgi-bin/upload.cgi
>>
>>> 3.The correction for multiple comparisons is done by thresholding all
>>> real
>>> T-map clusters above the predefined T-Value with the smallest
>>> iterations*alpha cluster of the permutation distribution according to
>>> area-corrected? The corrected P-value for real T-map clusters that
>>> survive
>>> this threshold is derived from the rank the clusters hold in the total
>>> iterations list according to area-corrected?
>>>
>>>
>> I would put it differently. Any clusters in the real tmap surviving the
>> threshold that exceed the minimum significance cut-off in corrected area
>> are significant.
>>
>>> Again, thanks a lot for your answers!!
>>>
>>> Julia
>>>
>>>
>>>
>>>> Hi Julia,
>>>>
>>>> Bear with me, because it's been years since I used the caret_command
>>>> tests, which are mostly cluster tests. We switched from cluster to
>>>> TFCE
>>>> a year or so ago:
>>>>
>>>> http://brainvis.wustl.edu/wiki/index.php/Caret:Documentation:Statistics#Threshold-Free_Cluster_Enhancement_.28TFCE.29
>>>>
>>>> But those tests are in caret_stats, which is a separate tool/package
>>>> based on java. Let me know if you want to know more about that.
>>>>
>>>> My inline replies below reflect my best recollection of the cluster
>>>> tests.
>>>>
>>>> Donna
>>>>
>>>> On 10/08/2010 08:16 AM, Julia Bender wrote:
>>>>
>>>>
>>>>> Hi Donna,
>>>>>
>>>>> I've looked more into the "caret_command
>>>>> -metric-statistics-one-sample-t-test" output files. I'm having trouble
>>>>> understanding what all the information means. Maybe you can correct
>>>>> me:
>>>>>
>>>>> 1. xx.metric_TMap.metric:
>>>>> Map of T-Values for the wholebrain (?) for the contrast defined in the
>>>>> xx.metric input files. Those are T-Values exceeding the T-Value
>>>>> thresholds
>>>>> and alpha level I specified in
>>>>> "caret_command-metric-statistics-one-sample-t-test" (eg -300000 2.5
>>>>> 0.1
>>>>> in
>>>>> my case).
>>>>>
>>>>>
>>>> They are not thresholded. The 2.5 and 0.1 affect the cluster size,
>>>> which affects downstream outputs. But this TMap is unthresholded.
>>>>
>>>>
>>>>> They are not corrected for multiple comparisons(?).
>>>>>
>>>>>
>>>> No, definitely not.
>>>>
>>>>
>>>>> This is the one I should load onto my surface. When I load
>>>>> that TMap, why can I still see T-Values below my defined threshold?
>>>>>
>>>>>
>>>>>
>>>> Right. I like to use the unthresholded t/f-map for my figures, but
>>>> generate a border around the clusters that were significant, and show
>>>> the border overlaid on the unthresholded t-map.
>>>>
>>>>
>>>>> 2.xx.metric_ShuffledTMap.metric:
>>>>> Distribution of T-Values derived from permuting + and - on each
>>>>> element
>>>>> in
>>>>> xx in N iterations (in my case 1000)
>>>>>
>>>>>
>>>>>
>>>> Right, and less than 1000, depending on how many columns there are in
>>>> your input composite metric.
>>>>
>>>>
>>>>> 3.xx.metric_TMap_Significant_Clusters.txt:
>>>>> This is what the help page says:
>>>>> "1. Find the biggest cluster in each column of the permutation T-Map
>>>>> metric/shape file and sort them by cluster size."
>>>>> I see two lists of clusters in the output file. I assume the one that
>>>>> is
>>>>> the result of this sorting is the lower one. It has about 700 rows
>>>>> depending on xx, why does it not have 1000 rows, one for each
>>>>> permutation?
>>>>>
>>>>>
>>>>>
>>>> See Sample Report: Two Sample T-Test here:
>>>>
>>>> http://brainvis.wustl.edu/OLD/courses/stats_neurosci/2008_BMEcourse/BME_dld_talk.htm
>>>>
>>>> Yours is a one-sample t-test, but I think the cluster lists will be the
>>>> same.
>>>>
>>>> If your last list of clusters has less than 1000 rows, then you had
>>>> fewer than 10 columns in your input composite metric. If n is the
>>>> number of columns, and 2 raised to the n is less than your input
>>>> iterations, then Caret will stop at 2^n.
>>>>
>>>>
>>>>> Are the clusters sorted by Num-Nodes, Area or Area corrected?
>>>>>
>>>>>
>>>> Descending area-corrected.
>>>>
>>>>
>>>>> In my files
>>>>> they seem kind of sorted by both... How is a cluster defined?
>>>>>
>>>>>
>>>>>
>>>> By your input thresholds. The permuted t-maps are thresholded at the
>>>> level specified, and then clusters of contiguous supra-threshold nodes
>>>> are found. Only the largest in each iteration is saved. Then they are
>>>> listed in descending order of area-corrected size.
>>>>
>>>>
>>>>> "2. Find the largest (alpha)(iterations) clusters in the Permutation
>>>>> T-Map
>>>>> and use its cluster size as the Significant Cluster Cutoff."
>>>>> I assume this is the list of T-Values right below the above list of
>>>>> clusters, why does it contain clusters with a P-Value that is above my
>>>>> defined alpha?
>>>>>
>>>>>
>>>>>
>>>> The last table lists all the largest clusters for each iteration,
>>>> regardless of its p. The second table is what you want.
>>>>
>>>>
>>>>> "3. Find clusters in the Real T-Map file." This must be the upper list
>>>>> of
>>>>> clusters (containing much less rows than the lower one)
>>>>>
>>>>>
>>>>>
>>>> Correct. That link above shows this the best.
>>>>
>>>>
>>>>> "4. Report all clusters in Real T-Map file that are larger than
>>>>> Significant Cluster Cutoff." This is the list of T-Values below this
>>>>> list,
>>>>> containing only clusters that are bigger than the cluster with the
>>>>> highest
>>>>> P-Value found in 2. that pass the alpha and T-Value thresholds I
>>>>> specified
>>>>> in "caret_command-metric-statistics-one-sample-t-test".
>>>>>
>>>>>
>>>>>
>>>> Again, here is the relevant excerpt from the link above:
>>>>
>>>> Significant Area: 306.226 <--- area of smallest cluster listed in
>>>> next
>>>> section
>>>>
>>>> Shuffled TMap <--- Top alpha*iterations biggest clusters are listed
>>>> below,
>>>> in descending area-corrected sequence.
>>>> (i.e., the smallest of which determines the signifiance area cut-off)
>>>> -------------
>>>> Column Thresh Num-Nodes Area Area-Corrected COG-X
>>>> COG-Y COG-Z P-Value
>>>> 821 2.660 2300 1439.637207 1559.143188 -37.249
>>>> 0.007 -3.456
>>>> 150 2.660 792 598.519104 858.216492 -49.142
>>>> -32.369 11.918
>>>> 548 2.660 1380 641.563843 643.790466 -35.004
>>>> 2.519 -14.981
>>>> ... (middle biggest alpha*iterations entries omitted)
>>>> 681 -2.660 279 249.739059 312.872681 -48.189
>>>> -9.576 10.352
>>>> 649 2.660 237 198.468857 311.602844 -16.170
>>>> -79.223 30.149
>>>> 633 -2.660 217 119.337837 306.226257 -42.479
>>>> -48.846 41.505
>>>>
>>>>
>>>> TMap <--- Significant real tmap clusters (i.e., >= significant area)
>>>> are
>>>> listed here; no entries here means no clusters were significant.
>>>> ----
>>>> Column Thresh Num-Nodes Area Area-Corrected COG-X
>>>> COG-Y COG-Z P-Value
>>>> 3 2.660 344 229.353973 362.322113 -45.677
>>>> -27.442 18.039 0.029000
>>>>
>>>> Shuffled TMap <--- All iterations max clusters are listed below, in
>>>> descending area-corrected sequence.
>>>> -------------
>>>> Column Thresh Num-Nodes Area Area-Corrected COG-X
>>>> COG-Y COG-Z P-Value
>>>> 821 2.660 2300 1439.637207 1559.143188 -37.249
>>>> 0.007 -3.456
>>>> 150 2.660 792 598.519104 858.216492 -49.142
>>>> -32.369 11.918
>>>> 548 2.660 1380 641.563843 643.790466 -35.004
>>>> 2.519 -14.981
>>>> ... (middle biggest alpha*iterations entries omitted)
>>>> 3 2.660 1 0.475079 0.472114 -18.520
>>>> -24.418 -21.858 0.999000
>>>> 3 -2.660 1 0.474738 0.456814 -21.478
>>>> -36.579 -17.077 0.999000
>>>> 3 2.660 1 0.000000 0.000000 -6.822
>>>> -46.751 8.581 0.999000
>>>>
>>>> ------------------------------------------------------------------------
>>>>
>>>>
>>>>
>>>>> 4. T-Map_LH_cCue_EndoL_vs_Cue_ExoL.metric_TMapClusters.metric
>>>>> Is this the map of clusters defined above?
>>>>>
>>>>>
>>>>>
>>>> This is what I believe you thought xx.metric_TMap.metric was, but it is
>>>> unthresholded.
>>>>
>>>> This TMapClusters one zeroes out all nodes that are NOT within a
>>>> significant cluster.
>>>>
>>>>
>>>>> I'm sorry, I know these are a many questions. Thanks a lot for your
>>>>> help!
>>>>>
>>>>> Julia
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> Julia,
>>>>>> I looked at your report and your t-map, which is consistent with the
>>>>>>
>>>>>>
>>>>>>
>>>>> caret_command -metric-information output you included below.
>>>>>
>>>>>
>>>>>
>>>>>> Just making sure you understand this part of the report:
>>>>>> TMap
>>>>>> ----
>>>>>> Column Thresh Num-Nodes Area Area-Corrected COG-X
>>>>>>
>>>>>>
>>>>>>
>>>>> COG-Y COG-Z P-
>>>>>
>>>>>
>>>>>
>>>>>> Value
>>>>>> 1 2.500 3064 2223.228027 2245.943848 -30.317
>>>>>> -73.796 -12.228 0.0
>>>>>> 12000
>>>>>> 1 2.500 3372 1865.999878 1863.883423 -21.516
>>>>>> -62.842 46.382 0.0
>>>>>> 17000
>>>>>> 1 2.500 1557 681.308838 674.981384 -32.967
>>>>>> -5.873 48.701 0.0
>>>>>> 59000
>>>>>> These are the clusters in your real t-map that were significant at
>>>>>> the
>>>>>>
>>>>>>
>>>>>>
>>>>> 0.1
>>>>>
>>>>>
>>>>>
>>>>>> alpha you specified, using the 2.5 threshold. (Note that all the
>>>>>>
>>>>>>
>>>>>>
>>>>> significant clusters were at the positive end.)
>>>>>
>>>>>
>>>>>
>>>>>> I believe the reason you saw different max/min in the Caret GUI was
>>>>>> that
>>>>>>
>>>>>>
>>>>>>
>>>>> you had the permuted t-map loaded, instead of the real one. In your
>>>>> message below, you said, "Adjustment:Column: permuted
>>>>> T-Values,Threshold
>>>>> type". There is nothing about permuted in the file you uploaded. If
>>>>> you
>>>>>
>>>>>
>>>>>
>>>>>> were viewing the permuted/shuffled t-map, this would also explain why
>>>>>>
>>>>>>
>>>>>>
>>>>> little would survive a low threshold.
>>>>>
>>>>>
>>>>>
>>>>>> But we don't necessarily (or even usually) use the same values we
>>>>>> used
>>>>>>
>>>>>>
>>>>>>
>>>>> for
>>>>>
>>>>>
>>>>>
>>>>>> cluster thresholds as the threshold for displaying t-maps, e.g., for
>>>>>>
>>>>>>
>>>>>>
>>>>> publication purposes. I think we like to see some color
>>>>> differentiation
>>>>> beyond the cluster threshold max. If they are the same, the color
>>>>> will
>>>>> saturate at the max. Sometimes we'll use a p-value derived from the
>>>>> degrees of freedom and get a corresponding t-value from that, and use
>>>>> that
>>>>>
>>>>>
>>>>>
>>>>>> for thresholding. Other times we might just use, say, +/-4.0 or
>>>>>> higher,
>>>>>>
>>>>>>
>>>>>>
>>>>> depending on how big the values get in the data. Usually we'll use a
>>>>> symmetric scale (i.e., -x to +x -- rather than different min/max).
>>>>> Donna
>>>>>
>>>>>
>>>>>
>>>>>> On 08/11/2010 09:12 AM, Julia Bender wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>> Hi Donna,
>>>>>>> I've just uploaded the two files.
>>>>>>> Thanks for your help!
>>>>>>> Julia
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> Julia,
>>>>>>>> It will be easier for me to get my head around your question if I
>>>>>>>> can
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>> get two files:
>>>>>
>>>>>
>>>>>
>>>>>>>> * T-Map_LH_cCue_Endo.metric_TMap.metric (whatever the final output
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>> metric was, but NOT the permuted/shuffled tmap file).
>>>>>
>>>>>
>>>>>
>>>>>>>> * The report named something like *Signicance*.txt
>>>>>>>> Could you upload those here:
>>>>>>>> http://pulvinar.wustl.edu/cgi-bin/upload.cgi
>>>>>>>> My brain would be ever so grateful.
>>>>>>>> Donna
>>>>>>>> On 08/11/2010 07:07 AM, Julia Bender wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> Hey everyone,
>>>>>>>>> I'm a bit confused about how to threshold my T-Maps in caret5. I
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>> created
>>>>>
>>>>>
>>>>>
>>>>>>>>> the maps with the following command:
>>>>>>>>> /usr/local/caret/bin_linux/caret_command
>>>>>>>>> -metric-statistics-one-sample-t-test $EACHMETRIC $FIDUCIAL_COORD
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>> $OPEN_TOPO $SURFACE_SHAPE 3 T-Map_$EACHMETRIC -300000.0 2.5 0.10 10 1
>>>>> 1000
>>>>>
>>>>>
>>>>>
>>>>>>>>> 0 4
>>>>>>>>> So I put the negative threshold to -300000 and the positive
>>>>>>>>> threshold
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>> to
>>>>>
>>>>>
>>>>>
>>>>>>>>> 2.5. When I look at the resulting Tmap.metric files it gives me
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>> something
>>>>>
>>>>>
>>>>>
>>>>>>>>> like this:
>>>>>>>>> Filename: T-Map_LH_cCue_Endo.metric_TMap.metric
>>>>>>>>> Number of Nodes: 73730
>>>>>>>>> Number of Columns: 1
>>>>>>>>> Column Minimum Maximum Mean Sample Dev
>>>>>>>>> %
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>> Positive % Negative Column Name
>>>>>
>>>>>
>>>>>
>>>>>>>>> 1 -9.785 6.076 -0.950 2.639
>>>>>>>>> 36.234 63.766 T-Values
>>>>>>>>> As far as I understand, this means the maximum negative T value in
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>> this
>>>>>
>>>>>
>>>>>
>>>>>>>>> metric is -9.785 and the maximum positive T value is 6.076. When I
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>> open
>>>>>
>>>>>
>>>>>
>>>>>>>>> the file in caret though (Color mapping: Auto Scale, Display mode:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>> Both,
>>>>>
>>>>>
>>>>>
>>>>>>>>> Display Color Bar, Threshold Adjustment:Column: permuted
>>>>>>>>> T-Values,Threshold type: Column) the bar tells me that my maximum
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>> negative
>>>>>
>>>>>
>>>>>
>>>>>>>>> value is -3.7 and my maximum positive value is 2.6. This also
>>>>>>>>> holds
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>> when
>>>>>
>>>>>
>>>>>
>>>>>>>>> I
>>>>>>>>> adjust the thresholds in the fields below to -2 and 2, when almost
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>> all
>>>>>
>>>>>
>>>>>
>>>>>>>>> activation disappears.
>>>>>>>>> Which is the correct information?
>>>>>>>>> Thanks a lot for your help!
>>>>>>>>> Julia
>>>>>>>>> Dipl. Psych. Julia Bender
>>>>>>>>> Humboldt Universität zu Berlin
>>>>>>>>> Mathematisch - Naturwissenschaftliche Fakultät II
>>>>>>>>> Institut für Psychologie, Abt. Klinische Psychologie
>>>>>>>>> Unter den Linden 6
>>>>>>>>> D-10099 Berlin
>>>>>>>>>
>>>>>>>>>
>> _______________________________________________
>> caret-users mailing list
>> [email protected]
>> http://brainvis.wustl.edu/mailman/listinfo/caret-users
>>
>>
>
>
> Dipl. Psych. Julia Bender
> Humboldt Universit�t zu Berlin
> Mathematisch - Naturwissenschaftliche Fakult�t II
> Institut f�r Psychologie, Abt. Klinische Psychologie
> Unter den Linden 6
> D-10099 Berlin
>
> _______________________________________________
> caret-users mailing list
> [email protected]
> http://brainvis.wustl.edu/mailman/listinfo/caret-users
>
_______________________________________________
caret-users mailing list
[email protected]
http://brainvis.wustl.edu/mailman/listinfo/caret-users
