to srmillis and sci.stat.edu, On 28 Mar 2002 06:48:21 -0800, [EMAIL PROTECTED] (SR Millis) wrote:
> Rich Ulrich wrote: > > For contrasting 2 raters, I like using a paired t-test and the > > corresponding interclass correlation. That shows you both > > the main pieces of information, without confusing them or > > confounding them at all. You get r to measure parallelism; > > you get t to measure mean-difference. > > Lin has discussed the shortcomings of the t-test for assessing > concordance between raters (Biometrics, 1989, 45, 255-268). Among other > things, the paired t-test fails to detect poor agreement in pairs of > data such as (1,3)(2,3)(3,3)(4,3)(5,3). Thanks for the reference. I did not have that. Sure, the t-test fails to detect.... No one uses it *alone*, do they? I always say, you use *both* the r and the t-test. There are the two elements: you can look at them separately, or look at them confounded with each other. For publication, editors (historically) like a single number. For research, researchers ought to see what's what. Both errors matter, but they are very distinct: it takes far less training to get rid of *bias* in rating scores, than to generate *correlation* where it is absent. A decent paired t-test procedure (in SPSS, for example) shows you both the r and the t-test. (I don't know whether paired-t is still missing from SAS.) Here are lines I found in a help file, downloaded from a webpage on the Stata module for concordance -- http://ideas.uqam.ca/ideas/data/Softwares/bocbocodeS404501.html " Lin's coefficient increases in value as a function of the nearness of the data's reduced major axis to the line of perfect concordance (the accuracy of the data) and of the tightness of the data about its reduced major axis (the precision of the data). The Pearson correlation coefficient, r, the bias-correction factor, C_b, and the equation of the reduced major axis are reported to show these components. Note that the concordance correlation coefficient, rho_c, can be expressed as the product of r, the measure of precision, and C_b, the measure of accuracy. " Okay, Lin's measure might be fine for the editor. (I don't know what this C_b is, but it certainly starts out as obscure, compared to t-tests that are in every intro-stats course.) > Pearson correlation coefficient can be a good starting point for > detecting lack of agreement. But, a high r doesn't necessarily indicate > agreement. As a follow-up, Lin's concordance correlation coefficient or > the Bradley-Blackwood procedure can be useful supplements. Bradley-Blackwood is also new to me. I don't find much from google, except the reference, 1991, Journal of Quality Technology 23:12-16. It seems that there is an F-test, which seems to be another single-number report. -- Rich Ulrich, [EMAIL PROTECTED] http://www.pitt.edu/~wpilib/index.html . . ================================================================= Instructions for joining and leaving this list, remarks about the problem of INAPPROPRIATE MESSAGES, and archives are available at: . http://jse.stat.ncsu.edu/ . =================================================================
