I can see that what you are calling FITB questions are causing some confusion. My last post may have helped that confusion.
First off different test items are best used for different levels of understanding. Multiple choice tends to test the lowest levels of learning (in Bloom's Taxonomy that would be Knowldege, Comprehension, and Application). "Supply items" such as FITB and essay questions tend to work best at the higher levels of the taxonomy (Bloom's Analysis, Synthesis, and Evaluation). Short answer (or FITB) questions test concepts well but are very hard to write so that only one phrase or word is the only correct answer. My response to the question of language is that answers should be in English for command answers (both one word like "fsck" and multiple words like "uname -s" or "ls -al". Local languages would work if you are going for a concept like "File systems that allow rollback are called _______" with the answer being "journaling file systems". Sorry the question is for example only and is off the top of my head but I hope you see the point. That said, I think your psychometrician is correct. Short answer questions should be used for "concept questions". As I have shown, these questions can have either a concept answer or a command answer. Both answers need to be highly restricted in order to work. in addition, the question shuold be phrased as a direct question and only use the incomplete statement form if it produces greater conciseness. For example "What is another name for true-false items? ___________ is preferred over True-false items are also called ____________ . The answer btw is "alternative-response items". I have no background in other languages but I can see how this grammatical structure may not work in other languages. It may make more sense to have a rule that is opposite of what I have said if it makes the questions more concise in a specific language to so. Rather than prefer FITB to multiple choice I would recommened that we look at what level of learning is being tested for and use the item style most appropriate to testing that level of understanding. For Level 1 type tests I beleive that you will find that large portions of it are still at the Knowledge/Comprehension/Application levels best suited to multiple choice with some FITB needed for variety and to test some of the knowledge that approaches Level 2 in difficulty. If you wish to move away from multiple choice I would suggest matching items, and interpretive items. Matching itmes could get around some of the issues of too much freedom to answer in a FITB by giving specific phrases to match to specific statements. They are easier to write than FITB as well. Interpretive items are based on a scenario usually given in a paragraph and consist of multiple questions measuring various levels of learning. Note that the introduction scenario can be a chart, picture, table or whatever. An example might be a snippet of a log file with questions about it's interpretation. > It was actually by request of our last psychometrician. As I recall him > explaining it when it was talked about with Level 2, and we began our > rewrite process for Level 1, we needed to do whatever possible to stay > away from "trivia" questions. The best way to demonstrate understanding > of concepts (as opposed to memorization of files, commands, etc) was in > concept FITB questions. Sounds like he means that the test should not simply be at the level of knowledge but should also test higher levels of understanding. I agree completely. You must look at the level of the job being testined for however. A person with a level 1 cert is mainly going to be applying basic knowledge under the watchful eye of a more experienced person. Asking her to explain the deep workings of the kernel when all she needs to do is understand how and when to use fsck is overkill. Personally, I think that a lot of multiple choice to establish the foundation knowledge with FITB and matching mainly for variety and interpretive sections for testing application level concepts is a bout right on level 1 and would limit the language issues significantly. Sorry for the wide ranging post but the issues have far more depth than just shoud we use English or not. Mark Miller _______________________________________________ lpi-examdev mailing list [EMAIL PROTECTED] http://list.lpi.org/mailman/listinfo/lpi-examdev
