Dear Dr. Henry
(cc Edstat-L, as invited)
You pointed out to me that the word "percentile", as used by
Rothstein
and Rogosa, does not mean "percentile" either of a population or sample,
but is a "score in a standard reference distribution", unrelated to
actual scores of other students. (Whether Richard Rothstein was aware of
this rather peculiar usage - and chose not to elucidate, despite writing
for a general readership - or not, I cannot tell.) Therefore, I presume
that the cutoff "percentile" was set by examining the material and
deciding that somebody scoring below a certain level was not ready to
proceed, not from an intent to hold back (say) 30% of the students.
If, as you say, "deserving" promotion means that "the student's
actual
degree of knowledge is above the mandated level set by the state", this
clearly has no genuine ethical implications, and the choice of such a
loaded word is empty rhetoric. The state's choice of mandated level may
be determined by various factors, including allowing an appropriate
margin of error. Thus, Rothstein's question
Are Americans prepared to require large numbers of
students to repeat a grade when they deserve promotion?
might be answered "why not, if 'deserving promotion' has nothing to do
with their preparedness for the next level, but is an unmeasurable
abstraction related to the testing system?"
The distinction between single and multiple tests is somewhat
specious.
One large test may have an accuracy comparable to the combination of
several smaller tests. The important question is whether the accuracy of
the test is sufficient that, with an appropriate cutoff score, it will
pass most students who are ready for the next level, while failing most
of those who would benefit more from repeating the year.
I have no idea whether that is the case. The article (which I
have
read) says nothing that would give any idea whether that is the case -
and makes me suspect that Rogosa has failed to consider whether it is
the case:
About half of fourth-grade students held back
for scores below the 30th percentile on a typical
reading test will actually have "true" scores
above that point. On any particular test, nearly
7 percent of students with true scores at the 40th
percentile will likely fail, scoring below the 30th
percentile.
If, in fact, the 40th percentile corresponds to a level of
reading
ability that is still not really adequate for progress to the next
level, this is not a problem. It may be a problem that, in order to
leave this margin or error, many semiliterate students must be promoted.
Or it may not be, depending upon how fast the benefit of promotion is
changing at that point.
Let us take it as understood that any student who has reasonable mastery
of the material will only benefit from promotion. The question is, how
does the benefit change up to that level? If the hypothetical curve is
like
benefit from
promotion
1 *******************
*
*
0 *********
0 10 20 30 40 50 60 70 80 90 100 "percentile"
then obviously a great deal of effort in testing (at least in marginal
cases) would be justified.
If,in the other hand,it is more like
benefit from
promotion
1 *************
*****
**
**
**
****
0 **
0 10 20 30 40 50 60 70 80 90 100 "percentile"
then the students with marginal scores would benefit comparatively
little from very accurate testing.
My point is that Rogosa appears not to take any of these
considerations
into account, and Rothstein certainly does not; and that any conclusion
reached _without_ doing so is probably valueless. Rothstein asks, of the
tests:
How well do they identify students who are
truly below a cutoff point like the 30th
percentile?
A better question to ask would be: how well do they distinguish students
who will probably benefit from promotion from those who will probably
benefit from repeating the year? To answer that question, one needs to
have - and use - information about students and reading, not just about
standardized tests _in_vacuo_.
-Robert Dawson
=================================================================
Instructions for joining and leaving this list and remarks about
the problem of INAPPROPRIATE MESSAGES are available at
http://jse.stat.ncsu.edu/
=================================================================
