Rich Ulrich wrote:
>
> (1) There is (something like) "Is the right answer given by someone
> with a good IQ?" I think that we are all agreed that (C) should meet
> that requirement. Further, I imagine that the item was validated
> *statistically* by this standard -- marking "C" goes along with
> higher scores on other test items.
Unless IQ is what you're trying to test, it's not the IQ, it's the
knowledge and understanding that's important.
>
> (2) There is a narrower approach -- which, indeed, was the question
> specified when this item was posted. "Does the item show whether the
> student understands rounding?" Will it be answered correctly by
> everyone who does, or could naive respondents be led astray?
Does the idea of "a naive respondent who nonetheless understands
rounding" really mean anything? Somebody who is naive *about rounding*
does not truly understand it. Whether somebody is naive about (say)
taking candy fron strangers is irrelevant here.
> "If you can imagine a way that someone would misread the item,
> then someone will." This is a mild version of Murphy's law. It is
> practically a truism when you are designing items or forms -- the hard
> part of your judgement is, figuring how much "problem" is too-much
> problem. In the recent Florida election, we learned that "punched
> cards" have an inherent error rate of over 1%. And a "butterfly
> ballot" has a rate over 5%. How much does it matter that most of
> these errors should befall that 15% of the voters in Florida who were
> voting for the first time? - well, it means that our subjective
> account should not assume that every voter is cool and experienced.
>
> "Professionally speaking," the butterfly punch-ballot has to be
> regarded as awful, no matter how much Jay Leno, etc., make fun
> of the Florida voters instead.
The purposes are very different. The purpose of the ballot is to
determine somebody's intention, not their understanding. If one *wanted*
a government chosen by the most intelligent, a ballot form that would
probably be spoiled by the uneducated voter would be the way to go. Of
course, you would need to use it in all districts and randomize which
candidate gets the easy-to-read spot at the top... rather than giving it
to the Governor's brother.
-Robert Dawson
=================================================================
Instructions for joining and leaving this list and remarks about
the problem of INAPPROPRIATE MESSAGES are available at
http://jse.stat.ncsu.edu/
=================================================================
