Hi Carol's query and people's responses got me thinking about an issue I've noticed with tests over the years. Every once in a while I have to put my item results into spss to correct an error on the answer key. But when I run reliabilities, I generally get relatively low values for alpha given the number of items ... e.g., .68 or so for a 50 item test (with 2 items removed because of perfect performance). With far fewer items (less than 20), I can easily get .80 or higher for measuring psychological traits.
I wondered if item difficulty, which several people mentioned, might be a factor. So I correlated across 48 items in the above example the difficulty (proportion correct) and the item-total correlation from Cronbach's alpha. r**2 was about 8% for a quadratic relationship. And it was clear in the scattergram that item-total rs were highest in the middle and approached 0 at either end. But there was still lots of variation in the middle. So it appears (perhaps obvious?) that measuring knowledge in a course is quite different than measuring psychological traits. Which should have some implications, I guess for how we design our tests, such as how we sample from the material in the course. What would happen, for example, if I had enough items to get reliabilities by chapters or sections? Would I find better alphas at that level? Take care Jim James M. Clark Professor of Psychology 204-786-9757 204-774-4134 Fax [email protected] >>> Carol DeVolder <[email protected]> 04-May-12 11:34 pm >>> Hi, As I sit here trying to do anything but grade or write exams, a thought occurred to me. Often, when one constructs an exam over several chapters, the questions are mixed up so that those over the same chapter aren't grouped together. Is this really necessary? It seems that it merely serves to add one more layer of confusion to the process. Or am I the only one who does this? Carol -- Carol DeVolder, Ph.D. Professor of Psychology St. Ambrose University 518 West Locust Street Davenport, Iowa 52803 563-333-6482 --- You are currently subscribed to tips as: [email protected]. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=13251.645f86b5cec4da0a56ffea7a891720c9&n=T&l=tips&o=17629 or send a blank email to leave-17629-13251.645f86b5cec4da0a56ffea7a89172...@fsulist.frostburg.edu --- You are currently subscribed to tips as: [email protected]. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=13090.68da6e6e5325aa33287ff385b70df5d5&n=T&l=tips&o=17653 or send a blank email to leave-17653-13090.68da6e6e5325aa33287ff385b70df...@fsulist.frostburg.edu
