- posted to sci.stat.edu, also to sci.stat.consult where a similar question was asked.
On Wed, 29 May 2002 10:42:14 GMT, Adrian <[EMAIL PROTECTED]> wrote: > Hello how is everybody, > > This is my first post here. I am wondering if I'm on the right track. Am > doing an experiment in which I now have to work out inter and intraobserver > error. From my reading sofar, it seems that this kappa value is the way to > go. I need a way to compare the results (in the format of 'severe', > 'moderate', 'mild', 'normal' etc...) from THREE raters (not TWO as most [ ... ] No, and no. You don't want kappa. It is most suited for dichotomies; it is sometimes used with multiple categories; it is a poor choice when you have well-ordered steps. When you have scores, use some version of correlation. If you are 'doing an experiment', then you ought to want to know the relations between the raters taken as pairs: is *any* one of them different? That's my opinion. If you are going to report on the multiple scorers, the intraclass correlation is probably most popular statistic; it is computed from an ANOVA table. On the other hand, I have long recommended looking at the pairs. That is easily done with the inter-class correlation, the ordinary Pearson r (which SPSS provides as an ancillary statistic of the paired-t test). Look at the r and the t and notice the standard deviations. In my area, the three-rater intraclass correlation is not needed for much except as a concise summary for the final write up. It can be computed from the ANOVA table. You should be able to search and find discussion of these at various places, including the SPSS web site. (I don't remember how much is in my own stats-FAQ on the subject, but there might be....) -- Rich Ulrich, [EMAIL PROTECTED] http://www.pitt.edu/~wpilib/index.html . . ================================================================= Instructions for joining and leaving this list, remarks about the problem of INAPPROPRIATE MESSAGES, and archives are available at: . http://jse.stat.ncsu.edu/ . =================================================================
