On 11/20/2022 5:31:00 AM, Carmelo Fruciano <[email protected]> wrote:
On 18/11/2022 5:06 pm, 'F. James Rohlf' via Morphmet wrote:
> I wonder whether it would help to be more strict about the use of the
> word "bias". There is the statistical meaning where there is a problem
> with the statistical estimate estimate being used. Must have to treat
> and correct for that differently than if the problem is that the
> investigator is making the measurements themselves incorrectly.
>
> With a statistic one can investigate properties assuming various
> statistical distributions. Not sure how to investigate theoretically
the
> effect of an investigator who systematically measures something a
little
> differently than intended or at least differently from other
> investigators working on the same or similar material. They are
> effectively measuring a different variable. Suggestions for a
different
> word?
Hi Jim and all,
I've been following the discussion and several interesting points which
have been raised this far.
About wording, in my mind, "systematic differences" is probably a quite
"neutral" (and current) wording to describe differences between
operators, devices, or other source which produces a variation in
multivariate mean. As others have suggested, depending on the context
other, less neutral, wording may also be appropriate.
Best,
Carmelo
--
==================
Carmelo Fruciano
Italian National Research Council (CNR)
IRBIM Messina
http://www.fruciano.org/
==================
> -------- Original message --------
> From: Mike Collyer
> Date: 11/8/22 1:16 PM (GMT-05:00)
> To: andrea cardini
> Cc: [email protected]
> Subject: Re: [MORPHMET2] Measurement error in geometric morphometrics
>
> Dear Andrea,
>
> I have to argue against one of your points.
>>
>> Nevertheless, I could miss a bias, but if ME has an Rsq of, say, less
>> than 1/30 of individual variation within species, when I test species
>> the bias will be negligible. This is, if I am correct, what you
>> implied when wrote that "one can argue that if measurement error is
>> very small, then randomness and homogeneity across groups are less of
>> an issue”.
>
> If we come full-circle to Philipp’s first point — that choice of
> individuals can mislead one’s interpretation — I believe it is
> dangerous to use a value of Rsq to conclude systematic ME (bias) is
> negligible. I hope I can demonstrate this with an example (in R).
>
> To set this up, I create 10 shapes based on a template that is a
square.
> I then add a digitizing bias by shifting two of the four landmarks
> (plus some random error).
>
> > # Create 10 specimens
> >
> > coords1 <- lapply(1:10, function(.) mat + rnorm(8, sd = 1))
> >
> > # Add digitizing bias for each, shifting two landmarks a little right
> > # plus add a little random error
> >
> > coords2 <- lapply(coords1, function(x)
> + x + matrix(c(0, 0, 1.5, 0, 0, 0, 1.5, 0), 4, 2, byrow = T) +
> rnorm(8, sd = 0.1))
> >
> > # string together and test for ME
> >
> > lmks <- simplify2array(c(coords1, coords2))
> > GPA <- gpagen(lmks, print.progress = FALSE)
> > ind <- factor(c(rep(1:10, 2)))
> > summary(procD.lm(coords ~ ind, data = GPA))
>
> Analysis of Variance, using Residual Randomization
> Permutation procedure: Randomization of null model residuals
> Number of permutations: 1000
> Estimation method: Ordinary Least Squares
> Sums of Squares and Cross-products: Type I
> Effect sizes (Z) based on F distributions
>
> Df SS MS Rsq F Z Pr(>F)
> ind 9 1.54733 0.171926 0.94906 20.7 5.5944 0.001 **
> Residuals 10 0.08306 0.008306 0.05094
> Total 19 1.63039
> ---
> Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
>
> Call: procD.lm(f1 = coords ~ ind, data = GPA)
>
>
>
> If we plot PC scores, the systematic bias is obvious:
>
>
>
> > # plot PC scores, with lines showing systematic ME
> >
> > PCA <- gm.prcomp(GPA$coords)
> > plot(PCA, pch = 19, asp = 1, col = rep(1:2, each = 10))
> >
> > for(i in 1:10) {
> + points(rbind(PCA$x[i,], PCA$x[10 + i,]),
> + type = "l",
> + lty = 3)
> + }
>
> PastedGraphic-1.tiff
>
> So one might see the bias in the plot and the 5% ME — if we want to
call
> it that based on Rsq in the ANOVA — might be too high for one’s
comfort.
> But now let's repeat the process on 10 specimens using instead of a
> square template, a long rectangle.
>
>
> > # Now add some more individuals to the mix, perhaps from
> > # a much differently shaped species (long rectangle, not square)
> > # using the same strategy
> >
> > mat3 <- matrix(c(0, 0, 50, 0, 0, 5, 50, 5), 4, 2, byrow = T)
> > coords3 <- lapply(1:10, function(.) mat3 + rnorm(8, sd = 1))
> > coords4 <- lapply(coords3, function(x)
> + x + matrix(c(0, 0, 1.5, 0, 0, 0, 1.5, 0), 4, 2, byrow = T) +
> rnorm(8, sd = 0.1))
> >
> >
> > lmks <- simplify2array(c(coords1, coords2, coords3, coords4))
> > GPA <- gpagen(lmks, print.progress = FALSE)
> > ind <- factor(c(rep(1:10, 2), rep(11:20, 2)))
> > summary(procD.lm(coords ~ ind, data = GPA))
>
> Analysis of Variance, using Residual Randomization
> Permutation procedure: Randomization of null model residuals
> Number of permutations: 1000
> Estimation method: Ordinary Least Squares
> Sums of Squares and Cross-products: Type I
> Effect sizes (Z) based on F distributions
>
> Df SS MS Rsq F Z Pr(>F)
> ind 19 4.9087 0.258351 0.98567 72.39 8.8918 0.001 **
> Residuals 20 0.0714 0.003569 0.01433
> Total 39 4.9801
> ---
> Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
>
> Call: procD.lm(f1 = coords ~ ind, data = GPA)
> >
> >
> > PCA <- gm.prcomp(GPA$coords)
> > P <- plot(PCA, pch = c(rep(19, 20), rep(20, 20)), asp = 1, col =
> rep(rep(1:2, each = 10), 2))
> >
> > for(i in 1:10) {
> + points(rbind(PCA$x[i,], PCA$x[10 + i,]),
> + type = "l",
> + lty = 3)
> + }
>
> PastedGraphic-2.tiff
>
>
> Note that the corresponding 10 vectors are shown in this PC plot as in
> the first, but 20 more values have been added (the cluster of points to
> the right). The mean is no longer the mean of 20 square-like shapes,
> but is the mean of 40 rectangles, with the square-like shapes now
having
> negative PC scores in the plot. Square shapes and long rectangle
shapes
> are clearly separated in this plot. Here is a transformation grid
> (scaled 1x) for the approximate middle of the points on the left:
>
> PastedGraphic-3.png
>
> and the same for the cluster of points on the right:
>
> PastedGraphic-4.png
>
> But let’s pay attention to the same 20 configurations in both plots.
> Now the systematic ME is clearly associated with the first PC, which
> is also representing more of the overall shape variation, and the
signal
> remains even though the ANOVA results suggest this is no big deal
(1.4 %
> of variation). Worse, the bias now appears to be associated with,
e.g.,
> species differences.
>
> The bias in this example did not become negligible in spite of changing
> the sample, and in spite of a conclusion to the contrary that might be
> made with ANOVA results. Again, evaluating the relative portion of
> variance explained (especially if based on dispersion of points, alone)
> is dangerous, and a comforting statistic should not be sufficient
> evidence to not worry about a systematic measurement error.
>
> Best,
> Mike
>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Morphmet" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to [email protected]
> .
> To view this discussion on the web visit
>
https://groups.google.com/d/msgid/morphmet2/C30FAD86-E64E-4AEB-8B8C-041768B131D8%40gmail.com
> .
>
> --
> You received this message because you are subscribed to the Google
> Groups "Morphmet" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to [email protected]
> .
> To view this discussion on the web visit
>
https://groups.google.com/d/msgid/morphmet2/6377ada3.050a0220.36294.e302%40mx.google.com
> .
--
You received this message because you are subscribed to the Google
Groups "Morphmet" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/morphmet2/85163e15-ee5a-9372-d6ae-315931bfd411%40unict.it.
