For those of you who teach Statistics or Research Methods or who like to think or teach about critical thinking about the use of statistics, there is a good article by Joel Best in the Chronicle of Higher Education that can be accessed at: http://chronicle.com/free/v47/i34/34b00701.htm I think it provides a very useful brief introduction to the value of learning statistics from a liberal arts perspective that could be given to a class early in the semester. There is a very good discussion of the balance I try to maintain somewhere between cynicism and credulity towards statistics in many of my classes but it is most apparent in my Psychological Testing class. In Psych Testing, I try to make the point over the semester that, simply put, psychological tests are not perfect but are, in many cases, better than the alternatives (personal biases, unstructured interviews, nepotism, etc.) Imagine my frustration when one of the best students in the department wrote the following, at the end of the semester, on a discussion thread about the future of testing. "It may be that the people who are horrible test takers could also be the perfect individual that a company might need but they won't hire them based on their test score. If the false positives and false negatives could ever be totally eliminated then it would be fair but its really not. They don't use lie detectors as evidence in court b/c they are not 100% accurate so why base a judgment of whether a person should pass or not or get a job or not on something that is not 100% accurate?" The article approaches this from a very balanced perspective. Best writes, "Being critical means appreciating the inevitable limitations that affect all statistics, rather than being awestruck in the presence of numbers. It means not being too credulous, not accepting every statistic at face value. But it also means appreciating that statistics, while always imperfect, can be useful. Instead of automatically discounting every statistic, the critical reserve judgment. When confronted with an interesting number, they may try to learn more, to evaluate, to weigh the figure's strengths and weaknesses." I use my own version of Tipster Beth Benoit's very useful Baseless Science Detector in my Research Methods classes and I think the following paragraphs from this article could make a very additional tool for thinking critically about the use of statistics. In fact, the two have many points of agreement. "It would be nice to have a checklist, a set of items we could consider in evaluating any statistic. The list might detail potential problems with definitions, measurements, sampling, mutation, and so on. These are, in fact, common sorts of flaws found in many statistics, but they should not be considered a formal, complete checklist. It is probably impossible to produce a complete list of statistical flaws -- no matter how long the list, there will be other possible problems that could affect statistics. The goal is not to memorize a list, but to develop a thoughtful approach. Becoming critical about statistics requires being prepared to ask questions about numbers. When encountering a new statistic in, say, a news report, the critical try to assess it. What might be the sources for this number? How could one go about producing the figure? Who produced the number, and what interests might they have? What are the different ways key terms might have been defined, and which definitions have been chosen? How might the phenomena be measured, and which measurement choices have been made? What sort of sample was gathered, and how might that sample affect the result? Is the statistic being properly interpreted? Are comparisons being made, and if so, are the comparisons appropriate? Are there competing statistics? If so, what stakes do the opponents have in the issue, and how are those stakes likely to affect their use of statistics? And is it possible to figure out why the statistics seem to disagree, what the differences are in the ways the competing sides are using figures?" Ironically, I think such a list would make a pretty good checklist and I may create an assignment around this for my stat class similar to the Baseless Science Detector assignment in my Research Methods class. Rick Dr. Richard L. Froman Psychology Department John Brown University Siloam Springs, AR 72761 e-mail: [EMAIL PROTECTED] http://www.jbu.edu/sbs/psych/froman.htm
