Dear all,
I would like to thank Dr Bookstein for posting his thoughtful and
congenial critical comments regarding our maximum likelihood (ML)
method for Procrustes analysis (Theobald and Wuttke 2006 PNAS
103:18521, implemented in our program THESEUS). I was very
interested to learn his views on the morphometric scientific program,
and, as can often happen, from disagreement springs new understanding.
In the following, I respond to Bookstein's three main comments,
quoting only limited extracts of his original text when necessary.
Bookstein's second and third comments are somewhat entangled, so I
respond to them together. First, I briefly consider two early (and
easier) comments later elaborated at the end of his post:
On Feb 8, 2007, at 4:03 PM, Fred Bookstein wrote:
> The TW paper begins (in a statement I will disagree with below):
>
> (*) "The goal of Procrustes analysis is to superpose non-identical
> shapes in an optimal manner via scalings, translations, and
> rotations."
> 3. The sentence (*) from TW does not actually match its use in the
> scientific context sketched in (2). In those sciences, Procrustes
> analysis is the multivariate analysis of one of two different
> Procrustes distances -- either Procrustes shape distance or
> Procrustes form distance -- and the role of the superpositions is
> to supply a set of variables that account for the appropriate sums
> of squares.
Our use of the term "Procrustes analysis" follows the broad
mathematical convention, which concerns, very generally, how to
relate matrices via similarity transformations. As Bookstein
remarks, our use may differ somewhat from how "Procrustes" is used in
morphometrics. Currently, one main difference is that our ML
Procrustes method applies only to landmark analysis, not to semi-
landmarks, curves, or surfaces, though this is potential work for the
future. Our use of "Procrustes analysis" is consistent, however,
with how many authors in shape theory use it, for example in Gower
and Dijksterhuis 2004 _Procrustes Problems_, Dryden and Mardia 1998
_Statistical Shape Analysis_, Small 1996 _The Statistical Theory of
Shape, _and various papers by C. Goodall -- all of which define and
discuss "Procrustes" in words highly similar to ours.
Also,
> Applications of GMM in molecular and structural biology ... would
> need to be in the version of the analysis that involves form space,
> not shape space.
Bookstein notes that superpositions in molecular/structural biology
should not involve scaling of size. I certainly agree that molecular
structures should not be scaled, since covalent molecular bonds are
essentially of fixed length. Consequently, by default our
implementation in THESEUS does not estimate scale factors for
macromolecules. Our method and program ultimately leave it up to the
researcher's judgement as to whether scale factors should be estimated.
Moving on to the substantial meat of Bookstein's criticism:
************************************************************
Regarding comment #1:
> 1. The concern for an optimal superposition arises mainly in
> sciences for which such a superposition has a scientific meaning
> that is the subject of explanation or verification at other
> levels. ... But applications in developmental biology,
> evolutionary biology, and medicine seem uninformed by those questions.
Bookstein asserts that there is no specific or testable biological
theory predicting that biological forms should have a unique and
privileged superposition. He specifically questions whether
morphometric figures should conform to our Gaussian perturbation
model. He expands on this point later:
> TW's Gaussian perturbation model is a special case of the general
> Mardia-Dryden model according to which the general covariance
> structure of that model takes a factored form. For assumptions
> about this covariance structure to make scientific sense, it must
> be the case that some scientific theory is available that makes a
> claim of this sort ...
However, strong empirical and theoretical reasons exist for expecting
biological (morphological) forms to be well approximated by our
model. As far as the factored covariance structure goes, yes, in
many cases, the factored covariance model is physically unrealistic
(because it assumes that each of the landmarks have the same
ellipsoidal principal axes of variability). However, we accept this
assumption as a simplifying approximation, noting that it is
considerably more general and less restrictive than the assumptions
underlying classical LS Procrustes analysis. Least-squares
Procrustes also corresponds to "a special case of the general Mardia-
Dryden model", one which assumes that (1) each landmark has the same
variance, (2) all landmarks are uncorrelated, and (3) all landmarks
have equal variance in all spatial dimensions. Our factored
covariance model relaxes all three of these very restrictive
assumptions.
More broadly, a Gaussian distribution is a natural choice for
morphometrics, as any process that involves a large number of
independent additive effects will produce data which approximates a
Gaussian (by the Central Limit Theorem). The exact process need not
be known or completely specified. For example, consider
superpositioning a set of adult hominid skulls, the original
Procrustes problem posed and solved by least-squares in 1905 by Franz
Boaz, the "father of American anthropology" (Science 21:862). We
know that the hominid skulls are evolutionarily related by common
ancestry, and thus that each has a common genealogical derivation.
The differences among the skulls can be accounted for by relatively
small evolved changes in the various developmental processes that
generate the final adult form, processes which have both genetic and
environmental components. Thus, we have a large number of (mostly
unknown) random environmental, genetic, and epigenetic effects that
have perturbed the common ancestral developmental program for making
hominid skulls. This is a classic case for expecting Gaussian
distributed data.
Additionally, the principle of maximum entropy provides a deep
theoretical justification for using a Gaussian distribution when very
little is known about the underlying process that generated the
data. If all we can know about our data is the average and the
variance, then the Gaussian distribution assumes the absolute minimum
necessary about the structure of our data [footnote 1].
We expect, furthermore, that certain regions of the skulls will have
varied more than others during evolution (for example, selection for
increased cranial capacity versus mandible size). We also expect
that some regions of the skulls will co-vary with other regions,
especially when two bones are physically conjoined or are correlated
by allometry or genetic pleiotropy. Thus we have clear conditions
for heterogeneous variances and correlations, both of which are
explicitly parameterized in our ML Gaussian perturbation model (yet
ignored by least-squares).
Bookstein maintains that there is no "true" superposition of any
value in morphometrics. I firmly disagree and think there is a very
relevant sense in which a "true" superposition exists, if only
abstractly, in developmental biology, and by extension, in
evolutionary biology. The "true" superposition should represent the
relative orientations of the skulls such that the shape differences
among them are solely due to real, local biological effects based in
differential developmental processes -- rather than to arbitrary
differences in reference frame. It is as if one could hold a skull
stationary and view the effects (either random or deterministic) of
developmental changes resulting from different environments or
genetic/epigenetic backgrounds. Our ML Procrustes method provides
the ML estimate of this "true" superposition.
If, for example, the inion and porion at the rear of the skulls
superimpose well, but the prosthion at the upper lip does not, then
this immediately suggests hypotheses concerning differential
expression of developmental genes in the latter region which would
account for the local morphological differences there. It is
important to note that in this hypothetical example, both the ML
Procrustes superposition and principal components analysis (PCA) will
give an answer significantly different from the conventional least-
squares Procrustes (in opposition to Bookstein's claim that "the TW
approach ... would lead to almost identical data analyses at the
final stage, principal coordinates analysis ..."). Since our ML
Procrustes method relaxes the unreasonable assumption that the front
and rear of the skull must have the same variance, the estimated
covariance matrix is usually very different, as is the corresponding
PCA, and one may expect the ordinations and splines to differ
accordingly. The accuracy of estimated principal components relies
directly on the accuracy of the covariance matrix; when estimating a
covariance matrix, more realistic assumptions will result in a
correspondingly more realistic (accurate) covariance matrix and
principal components.
For instance, here are the results from a least-squares (LS) vs a
maximum likelihood (ML) analysis of a real set of 40 hominid skulls
with 26 landmarks each (kindly donated by a generous reader). These
are the factor loadings for the first principle components in the LS
vs ML analyses, respectively:
lmark LS ML
1 -0.816 -0.002
2 -0.859 -0.134
3 -0.710 -0.541
4 -0.779 -0.455
5 0.385 -0.847
6 0.347 -0.828
7 -0.315 0.048
8 0.515 -0.382
9 0.582 -0.379
10 -0.859 0.130
11 -0.709 0.538
12 -0.778 0.451
13 0.386 0.849
14 0.348 0.830
15 -0.315 -0.049
16 0.516 0.384
17 0.582 0.382
18 0.580 0.001
19 0.395 -0.697
20 0.396 0.699
21 0.187 -0.798
22 0.481 -0.377
23 0.188 0.799
24 0.482 0.380
25 -0.043 -0.490
26 -0.042 0.490
As it turns out, in this case the ML analysis fits the data
considerably better than the LS analysis (see the statistics
discussed below).
In a later related comment, Bookstein questions the morphometric
applicability of our theoretical Gaussian model because we must
estimate a mean form:
> ... the mean form itself is a nuisance parameter right up there
> with the similarity transformations. ... In all of the
> paleoanthropological applications, for instance, explanations of
> the mean form are utterly obscured in half a billion years of
> vertebrate evolutionary history.
However, it is unnecessary to consider the mean form to be "real".
Nobody believes that there is a real, single, ideal American male who
is exactly 5' 8.5" tall (OK, maybe some starry-eyed optimists do :),
or, more pointedly, that an ideal American family exists with 2.3
children. Nevertheless, the mean in these examples carries useful
information as an abstraction. And, in the first case, the data are
(uncontroversially) well-approximated by a Gaussian distribution,
even though the underlying biological processes that result in
varying male heights are essentially unknown. Analogous arguments
apply equally to the Procrustes problem, where the mean form can be
regarded as an idealized form summarizing the common morphological
elements of the ensemble of configurations. Admittedly, one of the
drawbacks of the mean configuration in the usual LS Procrustes
analysis is that its precision is unknown. In contrast, our ML
Procrustes analysis provides a measure of uncertainty for each
landmark in the mean (that is, we provide the variance for each
landmark in addition to the average).
Ultimately, a ML superposition can be treated as a hypothesis to be
tested empirically: Does my ensemble of configurations fit well to
the Gaussian perturbation model? If so, then this is direct
empirical evidence that an underlying biological process is
approximated by the model. Similarly, it is common scientific
practice to fit a series of two-dimensional data points to a straight
line and to calculate the chi^2 for assessing the goodness of fit. A
good chi^2 implies that the data are related, at least approximately,
by some underlying linear model. The maximum likelihood framework
also allows one to bring all the power of Model Selection Theory to
the Procrustes problem. For instance, one can directly compare
various models (such as least-squares versus ML) with the likelihood
ratio test (LRT) or the Akaike Information Criterion (AIC) to
determine which model is closest to the truth (http://
en.wikipedia.org/wiki/Akaike_information_criterion , http://
www.amazon.com/Selection-Multi-Model-Inference-Kenneth-Burnham/dp/
0387953647). Calculating the chi^2, likelihood, and AIC for a
superposition is straightforward, and THESEUS produces all by default.
For example, here is the model selection comparison of the hominid
skull analysis described earlier:
likelihood AIC
LS -8536 -8882
ML -8456 -8846
diff +70 +36
Both the likelihood and the AIC are higher for the ML analysis,
indicating that the ML analysis fits the data considerably better and
is much closer to the truth. A nested likelihood ratio test in this
case gives a P-value of 0.0 (extremely significant), while the
Bayesian interpretation of the AIC gives a 100% probability that the
ML model approximates the truth better than the LS model. Thus, we
have reason to expect that the ML principal component reported above
should be markedly more accurate than its least-squares counterpart.
In sum, from a statistical point of view, there are solid scientific
reasons for fitting biological morphologies to our Gaussian
perturbation model. These justifications are especially compelling
when the underlying biological processes are complex and various,
particularly when the details are mostly unknown.
************************************************************
Bookstein's comment #2 & #3:
> 2. The least-squares property of the usual Procrustes
> superpositions has nothing to do with any aspect of real data ...
> The least-squares formulas express some symmetries of our preferred
> descriptive style; they are not associated with any particular
> protocol of estimation of anything.
Bookstein's second comment is primarily a philosophical point about
what Procrustes methods are "actually doing". It is here that
Bookstein and I differ the most. Bookstein prefers a "geometry-
first" paradigm for viewing morphometric statistics, whereas I offer
a model-based, statistical approach to morphometric geometry. As I
argue below, the latter view is necessary to provide a scientific
justification for the use of morphometric Procrustes methods, whether
least-squares or our maximum-likelihood.
Later Bookstein reinforces these sentiments expressed in the quote
above:
> Procrustes analysis really has nothing much to do with "finding the
> optimal superposition," if by optimal is meant some reference to a
> real-world phenomenon.
and
> If we were doing parameter estimations, we would know what the list
> of real parameters was -- in organismal biology, it is certainly
> not the list of landmark mean locations, which are an accident of
> the perceptual psychology by which we characterize certain loci as
> "landmarks" -- and in that case we would have no particular use for
> the Procrustes distances except as some sort of model of the
> scientist's own perceptual system.
In the above passages, it sounds as if Procrustes analysis has very
little connection to real data, if any at all. This should
immediately raise a red flag; the extent that a method "has nothing
to do with any aspect of real data" is the extent to which it is
scientifically inept. If, by performing a Procrustes analysis, we
are not trying to estimate anything about our data, then what exactly
are we doing that is scientifically relevant? However, I trust that
in practice morphometricians *do* believe that Procrustes methodology
is determining something useful and real about their data. In fact,
the abundance of figures of Procrustes superpositions in the
literature attests to the scientific value of superpositions for
conveying real properties of morphologies and for suggesting testable
hypotheses. I am also confident most researchers believe that
landmarks have an existence in reality that is independent of our
"own perceptual system", that they are not simply "accident[s] of the
perceptual psychology". Surely the nasion, pterion, bregma, etc. are
natural landmarks that objectively describes the morphology of the
skull, similar to how the location of an atom in a molecule
objectively defines its chemical structure.
Another philosophical point in Bookstein's second comment deals with
the significance of the Procrustes least-squares criterion, and why
least-squares is used rather than something else:
> The least-squares property of the usual Procrustes superpositions
> has nothing to do with any aspect of real data, but arises from the
> geometry of the Kendall quotient space within which our scientific
> descriptions of shape or form now reside.
Bookstein later expounds further:
> Procrustes superpositions are least-squares because the formulation
> of shape distance in the original Kendall context is least-squares
> in the corresponding Riemannian geometry. The TW comment has
> conflated two separate uses of the notion of least squares, one
> having to do with geometry, one with estimation. Kendall's least-
> squares is the _geometric_ usage, the closest (= least-squares)
> approach between two orbits in an elliptic Riemannian manifold.
> This geometry has nothing to do with multivariate statistics, ...
> ... the superposition is fixed by the rules of the game; it is not
> the topic of any estimation protocol.
Thus, according to Bookstein's geometric approach to morphometric
statistics, the Procrustes least-squares criterion is
methodologically prior because it is derived from the geometry of
Kendall's pre-shape space. The shortest Euclidean distance between
two pre-shapes (in pre-shape space) corresponds to the least-squares
Procrustes distance. For Bookstein, all subsequent statistical
analysis is based on that fundamental cornerstone. However, one must
immediately ask: "Why use the Euclidean distance metric in pre-shape
space, and why least-squares?" As Kendall noted (Bull London Math
Soc 1984 16:81), there are many other valid distance metrics. The
usual Euclidean norm, pleasantly corresponding to least-squares
Procrustes, is only one arbitrary choice from an infinite set (and
notably, it is a choice that Kendall did not prefer himself). More
generally, different choices for the shape distance metric can induce
different (non-Kendallian) geometries in shape space (e.g., see Rohlf
1999 Hystrix 11:1). Again, which metric and which corresponding
shape space to use is a choice that wants an objective
justification. Furthermore, least-squares Procrustes was used in
morphometrics for nearly 80 years before Kendall, and so the
justification for least-squares must lie elsewhere. Lacking any
physical or biological rationale, the question of "why Euclidean and
why least-squares?" must be answered with "no reason other than
mathematical convenience and familiarity".
However, if one adopts our model-based approach, then the standard
least-squares Procrustes method is justified if we assume that the
landmarks in our configurations are Gaussian with homogeneous
variances. Given this premise, the appropriate Procrustes distance
*must* be derived from least-squares superpositions, and the
corresponding distance metric in pre-shape space must be Euclidean.
Furthermore, one may generalize the Euclidean norm in Kendall's pre-
shape space by weighting the different dimensions (for example, by
the inverse of the covariance matrix for the landmarks in an ensemble
of configurations). If we choose to use *this* Mahalonobis distance
metric in pre-shape space instead, we find that it corresponds to our
maximum likelihood Procrustes distance in real figure space.
Therefore, from a model-based perspective, Bookstein's rationale is
backwards: in fact, the biological relevance of the Gaussian
recommends least-squares (and our Mahalonobis norm), thus specifying
the Euclidean (and Mahalonobis) distance metric in pre-shape space.
As in other scientific fields, our scientific goals for estimating
properties of real objects justify the mathematical geometry we use,
not the other way round.
I note that this disagreement is more fundamental than mere shape
theory or morphometrics; it really has to do with conflicting
paradigms of meta-analysis in the philosophy of science. Bookstein
believes that modern least-squares-based Procrustes and morphometrics
is "theory-neutral" analysis and "model-free" data mining. In
contrast, from a statistical model-based perspective, there can be no
such a thing as model-free data mining or theory-neutral analysis.
Ideally, all data analysis must have a strong theoretical foundation
of some sort, a theoretical foundation that is grounded in prior
empirical knowledge. Interestingly, there has been a highly
analogous controversy in phylogenetics/systematics over the past few
decades between the Hennigian proponents of parsimony (esp. pattern
cladists) and certain pheneticists, on the one hand, and the
advocates of model-based likelihood and Bayesian methods on the other.
In summary, the least-squares property cannot magically arise from
the abstract geometry of the Kendall quotient space in the absence of
any tie to reality. Rather, it is the geometrician who must choose a
distance metric, and that choice is arbitrary unless empirically
grounded in biological and physical reality. Fortunately for us, the
least-squares property of the usual Procrustes superpositions -- and
thus the superpositions themselves -- *do* have something to do with
real morphological data that can be modeled as Gaussian.
In the end, the proof of the pudding is in the eating of it. I hope
that the users of morphometric methods will try out our method.
Regardless of its theoretical advantages, the success of the method
will ultimately rely on its practical utility in analyzing morphology
and in generating productive hypotheses.
Cheers,
Douglas
************************************************************
Footnote 1:
For those who prefer a more poetic (or hyperbolic) justification for
using the Gaussian, here is the view of biometrician and statistician
Sir Francis Galton:
<quote>
I know of scarcely anything so apt to impress the imagination as the
wonderful form of cosmic order expressed by the "law of frequency of
error" [i.e., the Gaussian distribution]. Whenever a large sample of
chaotic elements is taken in hand and marshaled in the order of their
magnitude, this unexpected and most beautiful form of regularity
proves to have been latent all along. The law would have been
personified by the Greeks if they had known of it. It reigns with
serenity and complete self-effacement amidst the wildest confusion.
The larger the mob and the greater the apparent anarchy, the more
perfect is its sway. It is the supreme law of unreason.
</quote>
^`^`^`^`^`^`^`^`^`^`^`^`^`^`^`^`^`^`^`^`
Douglas L. Theobald
Department of Biochemistry
Brandeis University
Waltham, MA 02454-9110
[EMAIL PROTECTED]
^\
/` /^. / /\
/ / /`/ / . /`
/ / ' '
'
--
Replies will be sent to the list.
For more information visit http://www.morphometrics.org
