Forget questions. That's so western. Publish the tasks they will have to perform, of which they'll have to really know how to do stuff, and pepper them with questions as a guide to what matters most. The weight should be on successfully accomplishing the dynamic implementation of system features in a live environment. David Weeks On Mon, 12 Feb 2001, you wrote: > From: Michael Dowling <[EMAIL PROTECTED]> > > > : Please do elaborate. It seems to me that making the items public allows > for > : greater numbers of editors and testers, since there is no longer any need > to > : keep them secret while doing so. > > I don't have time to elaborate fully (nor spell-check) and this is a > hypothetical discussion because of the other difficulties I outlined in my > long previous email. But, hypothetically, let's take one example. What if > I only memorized a few questions and they appear on my exam. for those > questions, my exam score reflects my Linux ability and some error > (regardless of my ability, I got those items right). And notice that the > error is always "positive". Under this model, there is no "downward" force > so the net effect is always to get a little higher score. So this would > introduce a bias into the test scores. Even if it were small it would be > bad, and I see no reason to assume it would always be small. In fact, I > think it would vary, some people would be much advantaged and others not at > all. This is a psychometrician's nightmare... tests should measure what you > test as much as is possible excluding other factors. A test publisher's job > first and foremost, is to reduce bias and error in the scores of it's exams. > > Furthernore, one cannot randomly select items unless they are equivalent. > If there is variation in difficulty, for example, then random selection > would give some people harder tests and other people easier tests. But you > would hold them to the same cut-score. Does that seem fair? > > No, it seems ametuerish. What you have to do is to pre-calibrate the items > so you know how easy or hard they are. And that means intensive study of > each item.. so an enormous pool would be enormously costly (in resources, > not necssarily money). I described this in my last post. > > Besides, even if training companies were not involved, don't you think that > publishing our items would turn our test into a test of one's ability to > memorize (instead of on'e Linux skills)? Isn't that exactly what we DO NOT > want certification tests to do? I certainly hear people complain about this > influence all the time about all certification exams. I think, again, it's > our job to remove this component as much as possible. > > Finally, for what purpose? I don't think we are constraioned about who can > provide items. Didn't we decide that there was no prohibition? So this is > a solution seeking a problem. I think it flows more from a sense that > things should be open (like Linux). I agree whole-heartedly with that > attitude and I love open-source; that's why I volunteer for LPI. But that > analogy simply doesn't make for exams like the one we're discussing. > > So those are a few quick thoughts. I invite couter-arguments if I've missed > something. > > -Alan Mead > > > -- > This message was sent from the lpi-examdev mailing list. > Send `unsubscribe lpi-examdev' in the subject to [EMAIL PROTECTED] > to leave the list. -- This message was sent from the lpi-examdev mailing list. Send `unsubscribe lpi-examdev' in the subject to [EMAIL PROTECTED] to leave the list.
