Richard Bejtlich wrote:
So, if I memorize that the FCC is in charge, and the underlying
material says the FCC is in charge, how is that bad?
It's not bad, it just means that you know one snippet of information. If
the exam's objective is to test you for specific snippets of information
and you know those snippets, then you pass. But most IT certifications
are geared to far more than that.
Similarly, if I
memorize an answer that says "rpcinfo shows RPC services on a remote
system", and that is true, how is that bad?
It's bad because it means that instead of studying and being aware of
the whole realm of inter-process communications (which is likely the
intent of the certification), you don't bother to learn anything except
what rpcinfo does, because you know that that's the only question on the
exam of that realm.
A 75-item LPI exams requires 90 minutes to complete (I've proctored
exams and most people need every minute). Even a grueling, three hour
exam will only let you ask about 150 questions, which will enable you to
usually test only a very small proportion of the whole realm of subject
matter being certified.
When the questions are known then you allow the candidate to _not_ study
subjects that you want to certify but won't be asking about on one
particular instance of the test. They only need to study snippets, not
the whole realm you're certifying. And that _is_ bad.
If you have proper feedback, any candidate can comment on the questions
at the time they take the exam. This is an effective way to identify
problems with the help of those with a stake in it (the test takers),
and it doesn't matter if the whole pool is open or not.
That's nice in theory. Try sitting through a multi-hour exam and then
decide to critique the questions.
It's nice in practice. Thousands of LPI exams have had comments added at
the end (both PBT and CBT) and many have emailed in comments soon after
sitting the exams. I've proctored exams where people point out perceived
problems as they're handing in the answer sheet. The volume of feedback
has been such that we've had to automate it using the "request tracker"
package.
IMO it's a matter of community. If you're just seen as a vendor peddling
exams then candidates won't do much to help. If you're seen to be part
of your community -- that your exam takers see themselves having a stake
in the cert -- it's surprising how much unsolicited advice comes in.
("Be careful what you wish for..." and all that :-) ).
Then again, maybe some of it's because we made a conscious decision to
use multiple 90-minute exams rather than cram the whole cert into a
single multi-hour marathon; I think we made the right choice. Consider
factors such as "the ability of candidates to provide feedback" when
choosing the number and duration of exams required to complete a cert.
- Evan
_______________________________________________
BSDCert mailing list
[email protected]
http://lists.nycbug.org/mailman/listinfo/bsdcert
