I have a dilemma which I haven't found a good solution for. I work with
students who rotate with different preceptors on a monthly basis. A
student will have at least 12 evaluations over a year's time. A
preceptor usually will evaluate several students over the same year.
Unfortunately, the preceptors rarely agree on the grades. One preceptor
is biased towards the middle of the 1-9 likert scale and another may be
biased towards the upper end. Rarely, does a given preceptor use the 1-9
range completely. I suspect that a 6 from an "easy" grader is equivalent
to a 3 from a "tough" grader.
I have considered using ranks to give a better evaluation for a given
student, but I have a serious constraint. At the end of each year, I
must submit to another body their evaluation on the original 1-9 scale,
which is lost when using ranks.
Any suggestions?
--
"It has often been remarked that an educated man has probably forgotten
most of the facts he acquired in school and university. Education is what
survives when what has been learned has been forgotten."
- B.F. Skinner New Scientist, 31 May 1964, p. 484
=================================================================
Instructions for joining and leaving this list and remarks about
the problem of INAPPROPRIATE MESSAGES are available at
http://jse.stat.ncsu.edu/
=================================================================
