Re: The bad multiple choice test

[Jonathan Mueller]

A more recent test of the SAT reading comprehension items   Katz, S. & Lautenschlager, G.J. (2001). The contribution of passage and no-passage factors to item performance on the SAT reading task.  Educational Assessment, 7, 165-176.   which corrected for some earlier methodological flaws found essentially the same thing: Performance on the reading comprehension items could be better explained by non-passage factors than by passage factors.    Test-question writing is hard, and most teachers have not received much training in it.  For example, the reason most experts recommend limiting alternatives on MC items to three or four is not primarily because more than four alternatives begins to be more confusing for students, but because teachers have difficulty writing more than two good distractors ( plausible wrong answers).   Anyway, I can't point you to another bad MC test, but I can let readers know that I have just added a new chapter on constructing good tests (primarily MC) to my assessment website at     http://jonathan.mueller.faculty.noctrl.edu/toolbox/tests.htm   Feedback is always welcome.   Jon     =============== Jon Mueller Professor of Psychology North Central College 30 N. Brainard St. Naperville, IL 60540 voice: (630)-637-5329 fax: (630)-637-5121 [EMAIL PROTECTED] http://jonathan.mueller.faculty.noctrl.edu >>> [EMAIL PROTECTED] 03/01/06 11:49 AM >>> I found the reference I was thinking of: Katz, S., Lautenschlager, G. J., Blackburn, A. B. & Harris, F. H. (1990). Answering reading comprehension items without passages on the SAT. Psychological Science, 1, 122-127. Here is the abstract: Examined whether the Reading Comprehension (RC) task on the Scholastic Aptitude Test (SAT) measures factors that are unrelated to RC. Two experiments were conducted with 197 college students. Performance of Ss on the RC task was well above chance when reading passages were deleted. Moreover, Ss and test items performed similarly with or without the passages: Individual performance correlated with verbal SAT score, and the difficulty of items belonging to a passage correlated with a normative measure based on equated delta. Thus, performance on the RC task appears to depend substantially on factors having nothing to do with understanding the passages normally accompanying test questions. (PsycINFO Database Record (c) 2005 APA, all rights reserved) Annette Annette Kujawski Taylor, Ph. D. Department of Psychology University of San Diego 5998 Alcala Park San Diego, CA 92110 [EMAIL PROTECTED] --- You are currently subscribed to tips as: [EMAIL PROTECTED] To unsubscribe send a blank email to [EMAIL PROTECTED] --- You are currently subscribed to tips as: archive@jab.org To unsubscribe send a blank email to [EMAIL PROTECTED]