Thanks for all your responses on the item difficulty post. John, can you tell me a little more about the OpScan program? Sounds interesting...
______________________________________________ Roderick D. Hetzel, Ph.D. Department of Psychology LeTourneau University Post Office Box 7001 2100 South Mobberly Avenue Longview, Texas 75607-7001 Office: Education Center 218 Phone: 903-233-3893 Fax: 903-233-3851 Email: [EMAIL PROTECTED] Homepage: http://www.letu.edu/people/rodhetzel > -----Original Message----- > From: John W. Nichols, M.A. [mailto:[EMAIL PROTECTED]] > Sent: Friday, February 14, 2003 12:41 PM > To: Teaching in the Psychological Sciences > Subject: Re: Test item difficulty > > > It is certainly a difficult item for the students in the > class. Your difficulty measure clearly shows that to be the > case. That however, does not necessarily mean that there is > a test construction problem or that the item should be > eliminated. It could simply be a difficult item that few > students studied well enough to do more than guess at. > > Without a discrimination measure, it cannot be determined who > answered the question correctly. Was it the best prepared > student(s) who answered it correctly? Was it the poorly > prepared student(s) who knew that one thing, or just guessed > correctly? > > In my judgment, at least some high difficulty/high > discrimination items should make up part of the exam or quiz. > If it is a high difficulty/low discrimination item, I would > try to rework it or toss it. Lucky me! I use an OpScan > program that makes it very easy to measure both. > > I doubt that there are any statistical measures that will > discriminate between inadequate instruction and inadequate > preparation, but my years of experience have provided a lot > more cases of inadequate preparation by the student than > inadequate instruction by the prof. > > I used a series of similar questions on my exams until most > Intro authors quit covering more than one or two types of > validity and reliability. My own Intro students usually > wound up with around a .45 or .50 difficulty value and > discrimination level of around .70 or better. In other > words, those who knew the rest of the material very well > usually knew that item, too. Those who did not, did not. > > > "Hetzel, Rod" wrote: > > > > Hi everyone: > > > > Here's a scenario for your consideration. > > > > I gave a multiple-choice quiz today with ten items. Each item has > > four response options, so the optimum difficulty level for any item > > would be about .625. For one question, most of the class got the > > question wrong and the actual item difficulty was .08. > Does this mean > > that item itself was a difficult item (which would be a test > > construction issue and suggest that the item should be > discarded from > > the test), or does it mean that the students were not prepared to > > answer the question (which would suggest either inadequate > instruction > > by the professor or inadequate preparation by the students)? I'm > > looking at this because the question, in my estimation, was > a simple > > question. Here it is: > > > > A student confronts his psychology professor and says, "You > assigned > > Chapters 7 through 10, but nearly all of the items came > from Chapter > > 7. How can you evaluate whether we know anything about the other > > material we were supposed to read?" The student is challenging the > > test on the basis of: > > > > A. Face validity > > B. Content validity > > C. Criterion validity > > D. Construct validity > > > > This to me seems like a straightforward question. Students chose > > equally from the three distractors. The topic was covered > > substantially in class through lecture and activities. The > book also > > provides very easy coverage of this topic. I'm trying to > decide why > > this question posed such a challenge to the students. > > > > Rod > > > > ______________________________________________ > > Roderick D. Hetzel, Ph.D. > > Department of Psychology > > LeTourneau University > > Post Office Box 7001 > > 2100 South Mobberly Avenue > > Longview, Texas 75607-7001 > > > > Office: Education Center 218 > > Phone: 903-233-3893 > > Fax: 903-233-3851 > > Email: [EMAIL PROTECTED] > > Homepage: http://www.letu.edu/people/rodhetzel > > > > --- > > You are currently subscribed to tips as: [EMAIL PROTECTED] To > > unsubscribe send a blank email to > > [EMAIL PROTECTED] > > -- > > ----------==========>>>>>>>>>> ��� <<<<<<<<<<==========---------- > Sometimes you just have to try something, and see what happens. > > John W. Nichols, M.A. > Assistant Professor of Psychology > Tulsa Community College > 909 S. Boston Ave., Tulsa, OK 74119 > (918) 595-7134 > > Home: http://www.tulsa.oklahoma.net/~jnichols > MegaPsych: http://www.tulsa.oklahoma.net/~jnichols/megapsych.html > > --- > You are currently subscribed to tips as: [EMAIL PROTECTED] > To unsubscribe send a blank email to > [EMAIL PROTECTED] > --- You are currently subscribed to tips as: [EMAIL PROTECTED] To unsubscribe send a blank email to [EMAIL PROTECTED]
