dennis roberts wrote:
> At 01:34 AM 6/16/01 -0500, Jay Warner wrote:
>
> >but Doug Sawyer did not ask exactly that question. What he _may_ be asking
> >is whether a bank of questions could be developed, that are more or less
> >independent of one another,
>
> you mean uncorrelated??? across the same set of examinees when
> administered?? if so ... you don't have a collection of items that can form
> a scale ... what would they be measuring?
Sorry, should have been more specific. By independence, I mean that the order
of the questions does not seriously influence the scoring, for a given student.
Sometimes one question will jog a student's mind so as to remind them of what is
needed to answer another question. Sometimes one questions will be the giveaway
for another. Gotta prevent that(!)
> > and when administered to a group of students,
> >produce a mostly Normal distribution. I can't say about the effects of
> >question order, but I will bet that if you test a group of students (measure
> >their X - capability, accomplishment, memorization...) the grades will
> >fall more or less on a Normal dist.
>
> it depends on how difficult you make these items and the groups you give
> them to ... if all these items are very simple ... or very hard ... then
> your distribution will probably be seriously skewed on way or the other ...
>
> the fact that you CAN make a test score distribution come out to be
> basically normal is a combination of knowing the difficulty of the pool of
> items you have and knowing the examinees to whom you will be administering
> the test
OK, so if you give certain types of tough questions to a group that just
received the material, you may get a bimodal dist., one hump for those who
picked it up right away or already knew it, and one hump for those who didn't
listen the first time, or didn't get it the way it was presented. (those who
need multiple exposures to get a message).
Excepting that, if I now administer a stiff exam to a group, will I not see a
monomodal dist., skewed on the right, with most of the grades near the bottom?
(I once received a 25/100 on an exam, and was told I did very well - near the
top of the class.) It might actually look pretty close to a Poisson, or log
normal. Assuming it is a _very_ tough exam, or the students are very similar
learning levels, I may see a small skew - is this not getting back toward a
Normal? I don't say it is very close, but getting there.
If I administer an 'easy' test, will I not see a monomodal distribution with a
left skew? Ha! An inverse log normal! If you can do analyses with that,
untransformed, more power to you.
IN view of the precision of measurement (reliability?), and the huge debate over
what is actually underneath the score that is supposed to be measured, I'd say
that 'close' counts here as well as in horse shoes.
When they report percentile ranking, does that work from the actual histogram of
the test scores, or from a previous test with a known mean and standard
deviation? IN the latter case, we've just forced a Normal onto the data,
whether or not it is there.
When they convert a raw score to a test report score, do they transform the
number so as to force a Normal distribution on the reported score?
Jay
--
Jay Warner
Principal Scientist
Warner Consulting, Inc.
4444 North Green Bay Road
Racine, WI 53404-1216
USA
Ph: (262) 634-9100
FAX: (262) 681-1133
email: [EMAIL PROTECTED]
web: http://www.a2q.com
The A2Q Method (tm) -- What do you want to improve today?
=================================================================
Instructions for joining and leaving this list and remarks about
the problem of INAPPROPRIATE MESSAGES are available at
http://jse.stat.ncsu.edu/
=================================================================
