Doug Federman <[EMAIL PROTECTED]> wrote in sci.stat.edu:
>I have a dilemma which I haven't found a good solution for. I work with
>students who rotate with different preceptors on a monthly basis. A
>student will have at least 12 evaluations over a year's time. A
>preceptor usually will evaluate several students over the same year.
>Unfortunately, the preceptors rarely agree on the grades. One preceptor
>is biased towards the middle of the 1-9 likert scale and another may be
>biased towards the upper end. Rarely, does a given preceptor use the 1-9
>range completely. I suspect that a 6 from an "easy" grader is equivalent
>to a 3 from a "tough" grader.
First, it is rare that _any_ survey gets a significant number of
responses at either end. People tend to think, "Hmm, 1 to 9. Well, 1
would be perfect and 9 would be valueless [or vice versa]. Nobody's
perfect, so I'll write down a 2."
>I have considered using ranks to give a better evaluation for a given
>student, but I have a serious constraint. At the end of each year, I
>must submit to another body their evaluation on the original 1-9 scale,
>which is lost when using ranks.
>
>Any suggestions?
You could make a case for almost any jiggering of the numbers -- and
a case against it too. Since the 12 preceptors are grading the same
group of people, you could justify various forms of "normalizing".
But is it worth it? Don't the "easy graders" and :"tough graders"
pretty much cancel each other out anyway?
--
Stan Brown, Oak Road Systems, Cortland County, New York, USA
http://oakroadsystems.com
My reply address is correct as is. The courtesy of providing a correct
reply address is more important to me than time spent deleting spam.
=================================================================
Instructions for joining and leaving this list and remarks about
the problem of INAPPROPRIATE MESSAGES are available at
http://jse.stat.ncsu.edu/
=================================================================
