On Jun 13, 2006, at 8:21 PM, Alan McKinnon wrote:
As I said in the earlier mail, completing the question on hands-on
only shows that the candidate can perform THAT action. You can't
extrapolate it to mean anything more than that.
Answering a question on a multiple choice exam only tests just that
action or situation too, it's back to my point about how you have to
structure the exam and objectives so that they HAVE to prepare for
ALL the objectives, since they don't know what they will get. You
can't tell me that all the domains/sections that are hit in an LPI
exam currently test the candidate on ALL the objectives, they use the
same logic as a hands on, you must prepare for all since you can get
any combination.
On a subject I don't know well I always perform much better on
hands-on tests, mainly because I can read man pages much faster than
I recall dimly remembered facts.
Whereas I have observed in a hands-on environment that anyone who is
substantively in the man pages in the first 10 min of the exam
invariably fails. My method is to have them know enough about all
the topics to be functional, hit the man page a little if they need
to for the --whatever switch and get it done and on to the next. If
you're depending on the man page to refresh a nearly forgotten topic,
it's completely arbitrary as to what man page it is. They range from
extremely useful and filled with examples to essentially "here's all
the 58 switches in a large run on sentence, good luck with that..."
2. We all know about braindumps, testkings, boson, etc, where
some people just memorize the answers.
Don't forget that hands on exams handle with a very limited
number of different. If you know that you will be asked to
complete one of, lets say, 10 scenarios on a running machine then
it becomes rather easier...
Yes, I know. In hands on exams you have limited time too, so who
just knows what will be asked and tested at home, if anything is
different in the exam will fail, different from the experts one.
Mark Miller raises an interesting point in this regard a year ago, I
wish I still had the mail to cite the source. The gist of it is that
there is a statistical trend with hands-on where the results are
affected by the candidates mood (nervous, freak out in exams) whereas
it is largely absent with multiple choice.
Hmm, I think there is definitely a level of stress that ANY exam and
the resulting pass/fail will incur, and it's got to be CRYSTAL clear
in the examinees mind that the proper way to avoid this debilitating
stress is to be properly prepared. You have a definite point, but I
don't know that the delta of stress between multiple choice and hands-
on is that great a gulf.
So mood is an artificial
wild variable thrown into the exam affecting results where it doesn't
belong. (That means that the candidate is not normally nervous and on
edge when doing his daily job)
Oho, I am not in agreement with this, if the topic is
troubleshooting, there is a GREAT amount of nervousness and stress,
the damn server is down and people can't work and they are all over
you the poor sysadmin, and you can't find the backup tape, and the
boss is shouting and the CFO can't check the stock price, ad
nauseum. I would submit that everyone is stressed to some degree,
learning stress is necessary in the classroom and the stress of daily
sysadmin work can be anything from little to enough to cause a
meltdown. Your point that we should not incur unnecessary stress in
the examinee is very valid, but there has to be SOME stress, or it
wouldn't really test people's ability to perform the task, it would
just be click, type, click, type, yawn, done.
A major disadvantage of hands-on is that questions usually follow on
from each other - you can only add new users if you perform the
install right. making a mistake in the first step affects the rest of
the scores, which it shouldn't as failing one step should affect only
the score for THAT step
The CLP breaks it up into 5-7 sections, each is non-dependent on the
other but there are items in the section that are critical that you
perform that will affect the rest of the section. One of the things
about the LPI exams now is that they are very modular and questions
are completely independent of the other. Very little of real life in
a sysadmin role is as simple as that. The real hidden treasure of a
hands-on exam is that it teaches people that tasks must be performed
in an order to make something work, and it more closely mirrors real
life in that regard.
Brian is right, though this is a fun discussion, but it ain't writing
objectives for the exams. Sigh.
Ross
--
If only me, you and dead people understand hex,
how many people understand hex?
Alan McKinnon
alan at linuxholdings dot co dot za
+27 82, double three seven, one nine three five
_______________________________________________
lpi-examdev mailing list
[email protected]
http://list.lpi.org/mailman/listinfo/lpi-examdev
_______________________________________________
lpi-examdev mailing list
[email protected]
http://list.lpi.org/mailman/listinfo/lpi-examdev
