I think the precision used in terminology makes it hard to find a single word, as the contexts for measurement error are diverse. For example, if the goal is to assess the systematic difference between observers who digitize the same thing, there really is no “true value” from which to measure a departure, so even the terms, "observer bias” or “observational bias” are perhaps a little (or quite) off. “Observer preference" or "observer prejudice” might be better? But if one were to assess the ability of an automated digitizing process to record landmarks at exact locations a single observer would define, then one might assert that a tendency for the automated procedure to mis-locate landmarks in a certain way is a systematic procedural bias, as this would also lead to a bias in the estimation of the mean shape.
Perhaps the important thing to establish is that measurement error (ME) is an outcome rather than a process. By only using the terms, random ME or systematic ME, it does not say anything about the process that produces these phenomena. One can define the process in context; “Significant systematic ME as a result of an automated bias in the placement of landmarks…” or "Significant systematic ME as a result of divergent observer digitizing practices…” But I agree that avoiding the use of the term “systematic bias” of just “bias” to mean “systematic ME” is probably a good idea. Mike > On Nov 19, 2022, at 10:52 AM, 'F. James Rohlf' via Morphmet > <[email protected]> wrote: > > I couldn't think of a good single word either. I hoped one of the many clever > participants in the discussion would. Perhaps give them a couple of days? > > It is something that worried me back when I actually measured things. How > could I know I was really making the same measurement as someone else. I > think defining in terms of distances between 3D digitized landmarks helps a > lot. Someone else can also go back and look at the marked 3D images to figure > out what rules the prior investigator seemed to be using. Couldn't do that in > the old days! > > > Jim > __________________ > F. James Rohlf, Distinguished Prof. Emeritus > Dept. Anthropology and Ecology & Evolution > Stonybrook University > > > -------- Original message -------- > From: "Adams, Dean [EEOB]" <[email protected]> > Date: 11/19/22 2:35 AM (GMT-05:00) > To: [email protected] > Subject: RE: [MORPHMET2] Measurement error in geometric morphometrics > > Jim, > > > I agree that the term ‘bias’ without any clarification can be confusing. Your > classic 2003 was clear to state bias ‘in what’ up-front (bias in estimating > the mean shape) as well as provide clear definitions in terms of how you were > using the word. That was then connected to the statistical use of the term > regarding parameter estimation. > > > Bias in how something is digitized is somewhat different and discussions do > need to make that clear. Several posts in the thread did make it clear to > what they were referring by explicitly stating ‘bias in digitizing’ or ‘bias > in ME’. But simply using the term bias without any clarification can be > confusing and is not precise. > > > Off the top of my head I’m not sure of a single word that describes what we > the thread was discussing. However, with proper definition/description, I > think that any of these phrases could be appropriate: ‘digitizing bias’, > ‘bias in digitizing error’, ‘systematically-biased measurement error’, > ‘systematic measurement error’. > > > Dean > > > > Dr. Dean C. Adams (he/him) > > Distinguished Professor of Evolutionary Biology > > Department of Ecology, Evolution, and Organismal Biology > > Iowa State University > > https://faculty.sites.iastate.edu/dcadams/ > > phone: 515-294-3834 > > > From: 'F. James Rohlf' via Morphmet <[email protected]> > Sent: Friday, November 18, 2022 5:07 PM > To: Mike Collyer <[email protected]>; andrea cardini <[email protected]> > Cc: [email protected] > Subject: Re: [MORPHMET2] Measurement error in geometric morphometrics > > > I wonder whether it would help to be more strict about the use of the word > "bias". There is the statistical meaning where there is a problem with the > statistical estimate estimate being used. Must have to treat and correct for > that differently than if the problem is that the investigator is making the > measurements themselves incorrectly. > > > With a statistic one can investigate properties assuming various statistical > distributions. Not sure how to investigate theoretically the effect of an > investigator who systematically measures something a little differently than > intended or at least differently from other investigators working on the same > or similar material. They are effectively measuring a different variable. > Suggestions for a different word? > > > __________________ > > F. James Rohlf, Distinguished Prof. Emeritus > > Dept. Anthropology and Ecology & Evolution > > Stonybrook University > > > > -------- Original message -------- > > From: Mike Collyer <[email protected] <mailto:[email protected]>> > > Date: 11/8/22 1:16 PM (GMT-05:00) > > To: andrea cardini <[email protected] <mailto:[email protected]>> > > Cc: [email protected] <mailto:[email protected]> > Subject: Re: [MORPHMET2] Measurement error in geometric morphometrics > > > Dear Andrea, > > > I have to argue against one of your points. > > > Nevertheless, I could miss a bias, but if ME has an Rsq of, say, less than > 1/30 of individual variation within species, when I test species the bias > will be negligible. This is, if I am correct, what you implied when wrote > that "one can argue that if measurement error is very small, then randomness > and homogeneity across groups are less of an issue”. > > > If we come full-circle to Philipp’s first point — that choice of individuals > can mislead one’s interpretation — I believe it is dangerous to use a value > of Rsq to conclude systematic ME (bias) is negligible. I hope I can > demonstrate this with an example (in R). > > > To set this up, I create 10 shapes based on a template that is a square. I > then add a digitizing bias by shifting two of the four landmarks (plus some > random error). > > > > # Create 10 specimens > > > > > > coords1 <- lapply(1:10, function(.) mat + rnorm(8, sd = 1)) > > > > > > # Add digitizing bias for each, shifting two landmarks a little right > > > # plus add a little random error > > > > > > coords2 <- lapply(coords1, function(x) > > + x + matrix(c(0, 0, 1.5, 0, 0, 0, 1.5, 0), 4, 2, byrow = T) + rnorm(8, sd > = 0.1)) > > > > > > # string together and test for ME > > > > > > lmks <- simplify2array(c(coords1, coords2)) > > > GPA <- gpagen(lmks, print.progress = FALSE) > > > ind <- factor(c(rep(1:10, 2))) > > > summary(procD.lm(coords ~ ind, data = GPA)) > > > Analysis of Variance, using Residual Randomization > > Permutation procedure: Randomization of null model residuals > > Number of permutations: 1000 > > Estimation method: Ordinary Least Squares > > Sums of Squares and Cross-products: Type I > > Effect sizes (Z) based on F distributions > > > Df SS MS Rsq F Z Pr(>F) > > ind 9 1.54733 0.171926 0.94906 20.7 5.5944 0.001 ** > > Residuals 10 0.08306 0.008306 0.05094 > > Total 19 1.63039 > > --- > > Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 > > > Call: procD.lm(f1 = coords ~ ind, data = GPA) > > > > > If we plot PC scores, the systematic bias is obvious: > > > > > > # plot PC scores, with lines showing systematic ME > > > > > > PCA <- gm.prcomp(GPA$coords) > > > plot(PCA, pch = 19, asp = 1, col = rep(1:2, each = 10)) > > > > > > for(i in 1:10) { > > + points(rbind(PCA$x[i,], PCA$x[10 + i,]), > > + type = "l", > > + lty = 3) > > + } > > > > > > So one might see the bias in the plot and the 5% ME — if we want to call it > that based on Rsq in the ANOVA — might be too high for one’s comfort. But > now let's repeat the process on 10 specimens using instead of a square > template, a long rectangle. > > > > > # Now add some more individuals to the mix, perhaps from > > > # a much differently shaped species (long rectangle, not square) > > > # using the same strategy > > > > > > mat3 <- matrix(c(0, 0, 50, 0, 0, 5, 50, 5), 4, 2, byrow = T) > > > coords3 <- lapply(1:10, function(.) mat3 + rnorm(8, sd = 1)) > > > coords4 <- lapply(coords3, function(x) > > + x + matrix(c(0, 0, 1.5, 0, 0, 0, 1.5, 0), 4, 2, byrow = T) + rnorm(8, sd > = 0.1)) > > > > > > > > > lmks <- simplify2array(c(coords1, coords2, coords3, coords4)) > > > GPA <- gpagen(lmks, print.progress = FALSE) > > > ind <- factor(c(rep(1:10, 2), rep(11:20, 2))) > > > summary(procD.lm(coords ~ ind, data = GPA)) > > > Analysis of Variance, using Residual Randomization > > Permutation procedure: Randomization of null model residuals > > Number of permutations: 1000 > > Estimation method: Ordinary Least Squares > > Sums of Squares and Cross-products: Type I > > Effect sizes (Z) based on F distributions > > > Df SS MS Rsq F Z Pr(>F) > > ind 19 4.9087 0.258351 0.98567 72.39 8.8918 0.001 ** > > Residuals 20 0.0714 0.003569 0.01433 > > Total 39 4.9801 > > --- > > Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 > > > Call: procD.lm(f1 = coords ~ ind, data = GPA) > > > > > > > > > PCA <- gm.prcomp(GPA$coords) > > > P <- plot(PCA, pch = c(rep(19, 20), rep(20, 20)), asp = 1, col = > > rep(rep(1:2, each = 10), 2)) > > > > > > for(i in 1:10) { > > + points(rbind(PCA$x[i,], PCA$x[10 + i,]), > > + type = "l", > > + lty = 3) > > + } > > > > > > > Note that the corresponding 10 vectors are shown in this PC plot as in the > first, but 20 more values have been added (the cluster of points to the > right). The mean is no longer the mean of 20 square-like shapes, but is the > mean of 40 rectangles, with the square-like shapes now having negative PC > scores in the plot. Square shapes and long rectangle shapes are clearly > separated in this plot. Here is a transformation grid (scaled 1x) for the > approximate middle of the points on the left: > > > > > > and the same for the cluster of points on the right: > > > > > > But let’s pay attention to the same 20 configurations in both plots. Now the > systematic ME is clearly associated with the first PC, which is also > representing more of the overall shape variation, and the signal remains even > though the ANOVA results suggest this is no big deal (1.4 % of variation). > Worse, the bias now appears to be associated with, e.g., species differences. > > > The bias in this example did not become negligible in spite of changing the > sample, and in spite of a conclusion to the contrary that might be made with > ANOVA results. Again, evaluating the relative portion of variance explained > (especially if based on dispersion of points, alone) is dangerous, and a > comforting statistic should not be sufficient evidence to not worry about a > systematic measurement error. > > > Best, > > Mike > > > > -- > You received this message because you are subscribed to the Google Groups > "Morphmet" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] > <mailto:[email protected]>. > To view this discussion on the web visit > https://groups.google.com/d/msgid/morphmet2/C30FAD86-E64E-4AEB-8B8C-041768B131D8%40gmail.com > > <https://groups.google.com/d/msgid/morphmet2/C30FAD86-E64E-4AEB-8B8C-041768B131D8%40gmail.com?utm_medium=email&utm_source=footer>. > > -- > You received this message because you are subscribed to the Google Groups > "Morphmet" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] > <mailto:[email protected]>. > To view this discussion on the web > visithttps://groups.google.com/d/msgid/morphmet2/6377ada3.050a0220.36294.e302%40mx.google.com > > <https://groups.google.com/d/msgid/morphmet2/6377ada3.050a0220.36294.e302%40mx.google.com?utm_medium=email&utm_source=footer>. > > > -- > You received this message because you are subscribed to the Google Groups > "Morphmet" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] > <mailto:[email protected]>. > To view this discussion on the web visit > https://groups.google.com/d/msgid/morphmet2/CO6PR04MB8427C0F6FCC31066FDAA8BA8A2089%40CO6PR04MB8427.namprd04.prod.outlook.com > > <https://groups.google.com/d/msgid/morphmet2/CO6PR04MB8427C0F6FCC31066FDAA8BA8A2089%40CO6PR04MB8427.namprd04.prod.outlook.com?utm_medium=email&utm_source=footer>. > > -- > You received this message because you are subscribed to the Google Groups > "Morphmet" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] > <mailto:[email protected]>. > To view this discussion on the web visit > https://groups.google.com/d/msgid/morphmet2/6378fbc0.050a0220.e6871.0205%40mx.google.com > > <https://groups.google.com/d/msgid/morphmet2/6378fbc0.050a0220.e6871.0205%40mx.google.com?utm_medium=email&utm_source=footer>. -- You received this message because you are subscribed to the Google Groups "Morphmet" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/morphmet2/F512906F-0A84-4192-A0F8-28504A3D7E8E%40gmail.com.
