Dru wrote:
I don't see a problem with this "open" system.
I would daresay that a comparison is different because few peoples'
careers or pay scales depends on their ability to operate an amateur
radio. You don't have the active trade in cheating (by both candidates
and proctors) which is a by-product of the "high stakes" nature of most
IT certification exams.
Who makes and enforces the rules for the amateur service in the
United States?
A. The Congress of the United States
B. The Federal Communications Commission (FCC)
C. The Volunteer Examiner Coordinators (VECs)
D. The Federal Bureau of Investigation (FBI)
I know the answer is B, whether I read the book and remember it or I
read the question and see the answer is B. Either way I learn the FCC
is in charge.
The questions are not reproduced exactly (i.e., Congress is always
answer A, FCC, is B, etc.), so test-takers still need to know the
right answer.
That's not the point. By knowing the questions, anyone involved in
training for the exam need only teach to those questions, not to the
underlying subject matter. For the Americans within this group, that's
the major complaint of educators regarding the "No Child Left Behind"
program -- that people are teaching to the questions, and exam scores
are acceptable, but the students' real understanding hasn't improved at all.
The benefit of this system is that lousy questions are immediately
recognized by anyone inspecting the pool. Contrast the FCC pool
questions with those in the lousy CISSP exam that never see the light
of day, and you'll know what I mean.
If you have proper feedback, any candidate can comment on the questions
at the time they take the exam. This is an effective way to identify
problems with the help of those with a stake in it (the test takers),
and it doesn't matter if the whole pool is open or not.
Would we be able to sell this approach to the corporate/HR community
or would they just think us BSD people are still on LSD?
The whole point of the exam is that people who know their stuff should
pass and the people who don't know their stuff should fail. If it's too
easy to have false positives (by allowing people to pass who are good at
memorization but couldn't administer a system) then credibility fall
sharply.
If you want to determine the respect for an easily compromised
certification exam, have a look at the Brainbench series of unproctored
web-based exams. How many job postings ask for it? How many HR people
have _any_ respect for it?
Having studied the question pool, how does it work? Is there like
20,000 questions in the pool and you get asked 100 of them?
I think LPI's psychometrician said that the sweet spot for an item pool
was at about 3,500 for a 75-question exam. If someone can actually
memorize all those thousands of items, chances are they're actually
pretty close to knowing the subject matter. But making many more items
than that wouldn't really affect things much.
- Evan
PS: A little introduction... my name is Evan Leibovitch, and for the
last few years I've been President of LPI. While I'm eager to see a BSD
certification succeed, perhaps I can offer some experience-borne reality
checks that may prevent mistakes and build upon the work of others. I've
been exceptionally impressed by the work this group has done to date --
and am somewhat flattered at the positive references to LPI -- but it
also appears that this group has become extremely proficient at one
aspect of a cert program while _seeming_ to have bypassed the many
others. Then again, I'm newly subscribed here and may surely have missed
much. I hope to catch on quickly.
_______________________________________________
BSDCert mailing list
[email protected]
http://lists.nycbug.org/mailman/listinfo/bsdcert
