On Thu, 20 Apr 2000 09:55:13 +0200, Mats Carlsson <[EMAIL PROTECTED]>
wrote:
> Sorry if this has come up before, but
> here it goes.
I can't say the precise question has come up here, before -- What *is*
the precise question?
> Is there a way I can compare
> kappa-values? The backgound is as
Well, I think kappa is okay as a number to compare 2x2 tables, and
nothing bigger.
Generalized kappa is very much like Pearson r, isn't it? What are you
trying to learn, or what are you trying to show? If you were
comparing r, it would be comparison of "correlated correlations" but
that is better idea for correlations that are around .8 or lower, than
for correlations of .95 -- with the latter, N=100, you might be trying
to draw *statistical* conclusions from 2 or 3 discrepant judgments
(and the failure to meet the assumptions of asymptotic behavior will
invalidate testing).
Where you have one rater with his own alternative judgments, do you
just have a minor descriptive problem? or is there some independence
between judgments, and something going one that is more complicated
than moving a boundary between categories?
> follows:
> Four physicians has coded a 100 surgical
> notes.
> Each physician has coded each surgical
> note using all four different
> classifications. (thus coning the same
> note in four different ways).
> The classifications has differing
> numbers of catagories (one has 8, one
> 10, one 16 and so on).
>
> I've calculated the degree of agreement
> within each classification using
> generalized kappa. How can I compare
> these values? I'm not an experienced
> statistichian, so I'm kind of lost here.
> I've looked at Fleiss and Haas, but they
> don't seem to help in this issue.
I think you want to compare judgments rather than comparing Kappas,
but you need to define a purpose. In what fashion is something
expected to be better or worse?
--
Rich Ulrich, [EMAIL PROTECTED]
http://www.pitt.edu/~wpilib/index.html
===========================================================================
This list is open to everyone. Occasionally, less thoughtful
people send inappropriate messages. Please DO NOT COMPLAIN TO
THE POSTMASTER about these messages because the postmaster has no
way of controlling them, and excessive complaints will result in
termination of the list.
For information about this list, including information about the
problem of inappropriate messages and information about how to
unsubscribe, please see the web page at
http://jse.stat.ncsu.edu/
===========================================================================
