Alan & Susan Mead
Thu, 9 Sep 1999 00:58:01 -0700
I meant to send this to the list. It is typical Alan, a little long-winded and academic but perhaps a bit of food for thought. -Alan -----Original Message----- From: Alan & Susan Mead <[EMAIL PROTECTED]> To: Stephen Holcomb <[EMAIL PROTECTED]> Date: Tuesday, September 07, 1999 1:54 PM Subject: Re: Revised: A Proposed Concensus for Recertification and Renewal >We have definitely been discussing written tests. I think there is a lot of >interest in psychometricians and others in more performance-based testing >and a lot of certification folks (i.e., sys admins) have mentioned it in >discussions but I don't know anyone that is doing it validly and cost >effectively. > >One problem is that performance tests are often driven by an educational >syllabus rather than an analysis of job elements. So that makes these tests >more valid as an indicator of training proficiency than of job proficiency. >I think the sort of certification LPI wants to produce would be tried to a >broad family of jobs rather than any course of study. For example, they >have so far avoided offering or endorsing training. > >A second problem is scoring. If the scoring is "does it work" then it seems >easier but take my computer as an example: my TCP/IP is screwed up now that >I took my computer off the work LAN and make PPP connections from home. It >connects to everywhere BUT work (something about the routing is broken). So >I don't know if you would score me pass or fail (or if a grader would even >recognize that my solution only half works). Unreliability in the grading >of performance assessments is a major obstacle to their use (unreliability, >besides generating dissent among test-takers, lowers the test's validity). > >And the expense of having testing labs and human graders means that the >tests could not be made widely accessible. You could develop simulations >but I think that would be an enormous undertaking to simulate a Linux >environment (say, from the web) and then automate the scoring. There is >also the practical problem of where you could securely administer these >exams (i.e., I don't think the existing test administration infrastructure >will support it). > >I know Microsoft is a few four letter words to you, I feel the same way. >But in terms of IT certification testing, they are doing about as well or >better than anyone. They are actually moving towards more complex item >types which have *aspects* of performance in them. One I saw demoed >required the test-taker to assign IP numbers to a small subnet by dragging >the addresses onto the icons of the clients, gateway, server, etc. In >doing so, they are pushing the envelope and forcing the testing companies to >accommodate programmed software modules which integrate into the test. The >catch is that these have to be coded (AFAIK) in a Windows language... Who >wants to learn Visual C to code Linux simulations? > >SO, as I see it, there is a market for it. There are problems which might >be surmountable. But this is a longer-term, less certain project than the >currently developing LPI tests. > >-Alan > > > >-----Original Message----- >From: Stephen Holcomb <[EMAIL PROTECTED]> >To: [EMAIL PROTECTED] <[EMAIL PROTECTED]> >Date: Tuesday, September 07, 1999 1:06 PM >Subject: Re: Revised: A Proposed Concensus for Recertification and Renewal > > >>Greetings, >> >>This issue likely has previously been addressed, but is the testing format >>"written" only or are practical tests an integral component? I am asking >>this as someone who would like to see a robust credential emerge from this >>process. I recently completed Red Hat's RHCE course/exam and was pleasantly >>surprised at the level or rigor that I encountered. This is true especially >>when compared with the other written test based credentials that I hold >>(perhaps save Cisco). >> >>Also, if there is an informational document that I can review to become >more >>informed of progress/consensus and decision points thus far, I would >>appreciate a reference to the document so I can make more informed >>commentary. >> >>Stephen Holcomb ~ RHCE (in progress), CCNA, MCSE, MCT, CNA, A+, Net+ >>[EMAIL PROTECTED] >>602.541.0463 >>----- Original Message ----- >>From: Jared Buckley <[EMAIL PROTECTED]> >>To: <[EMAIL PROTECTED]> >>Sent: Tuesday, September 07, 1999 8:49 AM >>Subject: Re: Revised: A Proposed Concensus for Recertification and Renewal >> >> >>> Ok, I've taken a few days to digest the feedback on the second round of >>> concensus building. I'm glad we were able to get some wall-flowers to >>> participate! ;) I'll tackle the issue of recertification in a second, >>> but before we do that I'd like to move that we accept the second >>> concensus point on exam renewal. >>> >>> The only critical reply that I saw was Alan's. Alan, I think your >>> points are valid, but I don't think they fall outside the wording of the >>> second concensus point. All the concensus point requires is that update >>> at least once every two years. From Scott's emails, it sounded like >>> revising at least every two years would be a minimal psychometric >>> technical requirement. There's nothing prohibiting more frequent >>> revisions however. Alan if you or anyone else has further discussion on >>> this one, then let's branch it off into a seperate email. Otherwise I >>> say let's accept it as originally written. >>> >>> So, onto the topic of recertification. I was glad to see some strong >>> opinions on this topic because it's definitely an issue we'll encounter >>> down the road; better to settle it now. Let's chalk up the second >>> revision of the recert concensus point to a _looong_ week on my part and >>> try something a little more in line with the feedback I've seen: >>> >>> --------- >>> >>> Recertification: >>> >>> LPI will not require certificate holders to renew or recertify. LPI >>> will provide to third parties, at the certificate holder's request, >>> information pertaining to the history of test(s) passed, the date the >>> test(s) were passed, and the revision date/level(s) of the passed >>> test(s). In providing this information, LPI reserves the right to >>> indicate the current revision level of any or all of the test(s) passed >>> by the certificate holder and to issue public advisories concerning >>> changes in the content or objectives of the test(s). >>> >>> --------- >>> >>> Basically, what we're trying to do here is strike a balance between >>> protecting the long term viability of LPI while delivering real value to >>> the certificate holder. As I see it, we're trying to create a policy >>> that: >>> >>> o Minimizes LPI's Liability -- People that we say are certified >>> really can do the things outlined in the objectives. >>> o Maximizes the Value of the Certificate -- Part of which >>> includes not allowing the value to be diluted by continuing >>> innovations in the industry. (Several people pointed out that >>> basic *nix hasn't changed in more than 20 years. This is true >>> but keep in mind at higher levels we're going to be >>> certifing content with a shorter shelf life. ) >>> o Recognizes that Unused Knowledge Fades -- Yes, it's the >>> employer's job to realize that someone with a 3 year old cert >>> who hasn't touched a computer since, is probably not the >>> best candidate. OTOH, since we're not requiring recert we do >>> need some way to protect ourselves against lawsuits (however >>> frivolous they may be) from clueless consumers of our product. >>> (product = certificate holders) >>> o Recognizes that Even Current Practitioners' Knowledge Looses >>> Value Over Time -- Let's face it, even someone working w/Linux >>> every day isn't likely to use 100% of the skills they >>> certified on a regular basis. Even then, there's always the >>> problem with current practitioners sticking to legacy >>> solutions to problems. (How many people do you know who still >>> refuse to use anything but vi?) Its real effect may be >>> small but it does get back to the credibility issue. >>> o Provides Reassurance to _All_ of LPI's Consumers -- Consumers >>> of LPI's end product (certified Linux professionals) must have >>> the confidence in the system (a prerequisite) but must also >>> find it easier to utilize than the alternatives. (Other certs, >>> doing it themselves, outsourcing to a recruiting agency, etc.) >>> The harder we make it for the consumer to rely on LPI as a >>> tool, the less likely we make it that we're going to succeed. >>> My point being, the more work we make an employer do to >>> determine the value of the certificate, the less attractive >>> the whole process becomes. >>> >>> At the same time, we need a system that doesn't unduly burden >>> the certificate holders (also consumers) with needless >>> recertification costs (time and money) or devalue their >>> skills based simply on the time elasped since they last >>> certified (which has been pointed out can be a bad >>> predictor of ability.). >>> >>> >>> So let's hear what you think. >>> >>> Jared >>> >>> Jared Buckley wrote: >>> > >>> > Here's draft number two of the concensus: >>> > >>> > Recertification: >>> > >>> > LPI will not require certificate holders to renew or recertify. LPI >>> > will keep records of the test(s) passed and the revision date/level(s) >>> > of the passed test(s). LPI reserves the right to expire (cease to >>> > recognize) specific certifications that are more than two years old. >>> > >>> > Exam Renewal Consensus: >>> > >>> > LPI will revise the content of its exams in order to provide for new >>> > material, test validity, security, and to incorporate feedback from >>> > experience as deemed necessary, but not less frequently than every two >>> > years. >>> > >>> > >________________________________________________________________________ >>> > This message was sent by the linux-cert-program mailing list. To >>unsubscribe: >>> > echo unsubscribe | mail -s '' [EMAIL PROTECTED] >>> >>> >>> ________________________________________________________________________ >>> This message was sent by the linux-cert-program mailing list. To >>unsubscribe: >>> echo unsubscribe | mail -s '' [EMAIL PROTECTED] >>> >> >> >> >>________________________________________________________________________ >>This message was sent by the linux-cert-program mailing list. To >unsubscribe: >>echo unsubscribe | mail -s '' [EMAIL PROTECTED] >> > ________________________________________________________________________ This message was sent by the linux-cert-program mailing list. To unsubscribe: echo unsubscribe | mail -s '' [EMAIL PROTECTED]