It is certainly a difficult item for the students in the class. Your difficulty measure clearly shows that to be the case. That however, does not necessarily mean that there is a test construction problem or that the item should be eliminated. It could simply be a difficult item that few students studied well enough to do more than guess at.
Without a discrimination measure, it cannot be determined who answered the question correctly. Was it the best prepared student(s) who answered it correctly? Was it the poorly prepared student(s) who knew that one thing, or just guessed correctly? In my judgment, at least some high difficulty/high discrimination items should make up part of the exam or quiz. If it is a high difficulty/low discrimination item, I would try to rework it or toss it. Lucky me! I use an OpScan program that makes it very easy to measure both. I doubt that there are any statistical measures that will discriminate between inadequate instruction and inadequate preparation, but my years of experience have provided a lot more cases of inadequate preparation by the student than inadequate instruction by the prof. I used a series of similar questions on my exams until most Intro authors quit covering more than one or two types of validity and reliability. My own Intro students usually wound up with around a .45 or .50 difficulty value and discrimination level of around .70 or better. In other words, those who knew the rest of the material very well usually knew that item, too. Those who did not, did not. "Hetzel, Rod" wrote: > > Hi everyone: > > Here's a scenario for your consideration. > > I gave a multiple-choice quiz today with ten items. Each item has four > response options, so the optimum difficulty level for any item would be > about .625. For one question, most of the class got the question wrong > and the actual item difficulty was .08. Does this mean that item itself > was a difficult item (which would be a test construction issue and > suggest that the item should be discarded from the test), or does it > mean that the students were not prepared to answer the question (which > would suggest either inadequate instruction by the professor or > inadequate preparation by the students)? I'm looking at this because > the question, in my estimation, was a simple question. Here it is: > > A student confronts his psychology professor and says, "You assigned > Chapters 7 through 10, but nearly all of the items came from Chapter 7. > How can you evaluate whether we know anything about the other material > we were supposed to read?" The student is challenging the test on the > basis of: > > A. Face validity > B. Content validity > C. Criterion validity > D. Construct validity > > This to me seems like a straightforward question. Students chose > equally from the three distractors. The topic was covered substantially > in class through lecture and activities. The book also provides very > easy coverage of this topic. I'm trying to decide why this question > posed such a challenge to the students. > > Rod > > ______________________________________________ > Roderick D. Hetzel, Ph.D. > Department of Psychology > LeTourneau University > Post Office Box 7001 > 2100 South Mobberly Avenue > Longview, Texas 75607-7001 > > Office: Education Center 218 > Phone: 903-233-3893 > Fax: 903-233-3851 > Email: [EMAIL PROTECTED] > Homepage: http://www.letu.edu/people/rodhetzel > > --- > You are currently subscribed to tips as: [EMAIL PROTECTED] > To unsubscribe send a blank email to [EMAIL PROTECTED] -- ----------==========>>>>>>>>>> ��� <<<<<<<<<<==========---------- Sometimes you just have to try something, and see what happens. John W. Nichols, M.A. Assistant Professor of Psychology Tulsa Community College 909 S. Boston Ave., Tulsa, OK 74119 (918) 595-7134 Home: http://www.tulsa.oklahoma.net/~jnichols MegaPsych: http://www.tulsa.oklahoma.net/~jnichols/megapsych.html --- You are currently subscribed to tips as: [EMAIL PROTECTED] To unsubscribe send a blank email to [EMAIL PROTECTED]
