On Tuesday 13 June 2006 19:19, Felipe Salum wrote: > Well, let me try to explain what I think about hands on exams. > > 1. L3 is to be the senior certification for LPI, so I think that to > it get more credibility, a hands on exam is perfect. (I'm not > saying that L1 and L2 has no credibility, or any other paper based > exam).
This is the "hands-on must be better" meme. It's a common thread found all over the net, but let's look closely at it: is hands-on really better than multiple choice? And if so, why is it better? I have been searching for answers to this for years now and never found a satisfactory answer. In fact, it is becoming clear to me that it is simply not true and is a fallacy, an assumption based purely on opinion. Hands-on tests are valuable when done correctly. I define them as a test to see if a candidate can perform a pre-determined series of steps to produce a known result. You get this from drilling the steps till you can do it naturally, like a test to get a driver's license. To do this at all, you need: a. A well-defined framework in which to work. "Linux" is not a well-defined framework, it encompasses too much. "Red Hat Enterprise Linux version 4" is a well-defined framework, because if you install packages A, B and C you can predict with certainty what will be on the machine afterwards. b. An exactly defined series of steps, such as "these are the valid syntax for iptables, tcp wrappers and /etc/exports. Practice using them." c. An exactly defined end result, such as "192.168.1.12 and 192.168.0.0/24 must be able to mount /usr over nfs, all other hosts must be denied. Use a and b above to do this". Notice how this is all practical and somewhat narrowly defined, with a minimum requirement for deep cognitive understanding of the subject matter. It is simply not true that this accurately portrays real life so while it is valuable I don't buy the idea that it is representative of anything other than extensive drilling. LPI on the other hand, tests the cognitive domain which is a whole different thing. It does not test if you can do a limited range of actions, it tests if you understand how those actions are done, and why. So the two methods are very different, the main (only?) common element is a pass/fail grade at the end! On is better than the other in the same way as an apple is better than an orange. > 2. We all know about braindumps, testkings, boson, etc, where some > people just memorize the answers. This is a factor, but there are ways to minimize it. > 3. Just for example, I was taking a look at Sans GSE certification, > which they did too much difficult to have just a little people > certified. I don't want that for LPIC3, but looking at GSE, you see > that people have to demonstrate really that they own what they are > being tested. It is what I see in a hands on exam. Just memorizing > the solution is not sufficient to put your hands there and make it > working. I don't see that what you assert for hands-on is actually true, I think you are extrapolating too much from the method. It tests what it tests, no more and no less. A pass on a hands-on means that the candidate can accomplish what he was tested on and you can't really derive more information than that. Remember, a hands-on test can be brain-dumped just as easily as a multiple choice (perhaps not at the most senior levels, but definitely at junior/middle levels. > 4. RHCE, CCIE and others are successfull examples of hands on > exams. You see many people with CCNA (paper based with some hands > on simulator) and just a few with CCIE. I think that speaks more that RH and CCIE built a good exam and test environment and less about the testing methodology used. Don't get me wrong, I'm not anti hands-on. They have their place, as does multiple choice. I do believe we should get misconceptions about their relative merits out of the way before discussing the subject in earnest. -- If only me, you and dead people understand hex, how many people understand hex? Alan McKinnon alan at linuxholdings dot co dot za +27 82, double three seven, one nine three five _______________________________________________ lpi-examdev mailing list [email protected] http://list.lpi.org/mailman/listinfo/lpi-examdev
