-------- Original Message --------
Subject: Tribulations with permutations
Date: Mon, 11 Jul 2011 17:07:54 -0400
From: Hans Wellens <[email protected]>
To: <[email protected]>
CC: <[email protected]>
Dear All,
Various packages are available to perform permutation tests for shape
differences between groups: for instance the IMP package (Twogroup), or
the Shapes package (resampletest) (amongst others). The methodologies
employed by the various available programs seem to differ somewhat. To
gain some additional insight, I decided to program a permutation test in
R. The aim of this permutation test would be to determine whether there
is a difference in craniofacial shape between a male and female patient
sample. A few different scenarios could be envisioned. My hope is that
the formidable combined intelligence which is the morphmet list would be
so kind to enlighten me on the correct path to choose…
1.Perform a pooled generalized Procrustes analysis, taking care of
location, scale and rotation.
2.Calculate the male and female mean configurations
3.Then two options seem to be available:
a.Calculate the Procrustes distance (PD) between the male and female
mean configurations.
b.Calculate the squared PD between the male configurations and male mean
configuration and between the female configurations and female mean
configuration.
The first would directly generate the test statistic for comparison to
the permutation results, while the latter would need the calculation of
the F-ratio to compare the variances, in order to do the same.
4.Reshuffle the groups, generating pairs of groups of the same size as
the original ones. Let`s say one thousand groups are generated, and
let`s suppose a bootstrap is performed (permutation with replacement).
Again 2 scenarios present themselves:
a.Some people propose resampling from the pooled sample (males and
females combined).
b.In the Shapes package, resampling with replacement is performed WITHIN
groups.
Which one is preferable when resampling with replacement is performed?.
Also, is a renewed Procrustes superimposition required of each
permutation group? A very bright colleague of mine is probably right
when pointing out this would not be necessary since scale and location
and have been dealt with in the original Procrustes analysis. So this
would mean that the Procrustes coordinates are reshuffled, and not the
original ones.
5.Recalculate the male and female mean configurations, and the test
statistic from point 3 for each permutation group.
6.Compare the resulting values to the original one. The number of values
equaling or exceeding the original one serves as the p-value under the
null hypothesis.
Additional questions: should tangent space projection be considered in
the permutation test? Would there be additional value in performing this
test in form space?
Best regards,
Hans Wellens
Orthodontist
Groene-Poortdreef 16
8200 Sint-Michiels, Brugge
[email protected]