-------- Original Message --------
Subject: Re: Tribulations with permutations
Date: Tue, 12 Jul 2011 17:29:55 -0400
From: Dean Adams <[email protected]>
To: [email protected]

Hans,

How to best implement a resampling procedure depends very heavily on the
question one wishes to ask.

The scenario you proposed was one of testing 2 groups for shape
differences. Using parametric statistical procedures, this is often
examined using a multivariate t-test (Hotelling's T), which is simply
the squared Euclidean distance between group means standardized by
pooled-within group covariance (and some sample size adjustments).

For testing such hypotheses, I prefer using permutation (resampling
without replacement), rather than bootstrap. As such I would compare the
observed Euclidean distance between groups to a distribution of values
obtained from data sets where specimens are re-assigned to groups via
permutation. For 3 or more groups, permutational MANOVA follows this
same general logic. Note that here no additional GPA is necessary for
each permuted data set. One reason is that all specimens are found in
each iteration of a permutation, and thus the each resampled dataset is
already optimally aligned to the same reference.

An alternative that appears logical, but that I would not recommend, is
to perform the permutation test on Hotelling's T-square or Mahalanobis
distance (which can be mathematically derived from T-square).  The
reason I would not recommend this is that in each iteration of this
procedure, both the Euclidean distance between groups and the
pooled-within group covariance matrix are altered in each iteration.
Thus, if a significant result is obtained, one does not know whether the
groups are farther apart than expected, or if the within-group
covariance is smaller than expected.  Better to just test the
'numerator' of the T-square, which is simply the Euclidean distance
between group means.

There are quite a few morphometric-oriented papers where such
permutation procedures are described; and even more where they are
implemented. A few of my own describing resampling procedures for
testing various hypotheses include: Adams (2004: Ecology), Collyer and
Adams (2007: Ecology), Adams and Collyer (2009: Evolution), Adams and
Nistri (2010: BMC Evol. Biol.).  These can be found on my web page.

Note that there are MANY other MorphMet participants who have proposed
and implemented various permutation methods for testing patterns and
hypotheses of shape variation.

I'm certain others will chime in with additional thoughts and
references, and probably alternative perspectives.

Hope this helps and all the best,

Dean

--
Dr. Dean C. Adams
Associate Professor
Department of Ecology, Evolution, and Organismal Biology
Department of Statistics
Iowa State University
Ames, Iowa
50011
www.public.iastate.edu/~dcadams/
phone: 515-294-3834


On 7/12/2011 10:50 AM, morphmet wrote:

-------- Original Message --------
Subject:        Tribulations with permutations
Date:   Mon, 11 Jul 2011 17:07:54 -0400
From:   Hans Wellens<[email protected]>
To:     <[email protected]>
CC:     <[email protected]>



Dear All,

Various packages are available to perform permutation tests for shape
differences between groups: for instance the IMP package (Twogroup), or
the Shapes package (resampletest) (amongst others). The methodologies
employed by the various available programs seem to differ somewhat. To
gain some additional insight, I decided to program a permutation test in
R. The aim of this permutation test would be to determine whether there
is a difference in craniofacial shape between a male and female patient
sample. A few different scenarios could be envisioned. My hope is that
the formidable combined intelligence which is the morphmet list would be
so kind to enlighten me on the correct path to choose…

1.Perform a pooled generalized Procrustes analysis, taking care of
location, scale and rotation.

2.Calculate the male and female mean configurations

3.Then two options seem to be available:

a.Calculate the Procrustes distance (PD) between the male and female
mean configurations.

b.Calculate the squared PD between the male configurations and male mean
configuration and between the female configurations and female mean
configuration.

The first would directly generate the test statistic for comparison to
the permutation results, while the latter would need the calculation of
the F-ratio to compare the variances, in order to do the same.

4.Reshuffle the groups, generating pairs of groups of the same size as
the original ones. Let`s say one thousand groups are generated, and
let`s suppose a bootstrap is performed (permutation with replacement).

Again 2 scenarios present themselves:

a.Some people propose resampling from the pooled sample (males and
females combined).

b.In the Shapes package, resampling with replacement is performed WITHIN
groups.

Which one is preferable when resampling with replacement is performed?.
Also, is a renewed Procrustes superimposition required of each
permutation group? A very bright colleague of mine is probably right
when pointing out this would not be necessary since scale and location
and have been dealt with in the original Procrustes analysis. So this
would mean that the Procrustes coordinates are reshuffled, and not the
original ones.

5.Recalculate the male and female mean configurations, and the test
statistic from point 3 for each permutation group.

6.Compare the resulting values to the original one. The number of values
equaling or exceeding the original one serves as the p-value under the
null hypothesis.

Additional questions: should tangent space projection be considered in
the permutation test? Would there be additional value in performing this
test in form space?

Best regards,

Hans Wellens

Orthodontist

Groene-Poortdreef 16

8200 Sint-Michiels, Brugge

[email protected]


Reply via email to