Chris et al I have the same general reaction to high-stakes ability tests in general. Plus I get the comment that there is a population of people who are "not good test takers" - which implies there is some identifiable group for whom the high-stakes tests are NOT valid (as opposed to saying, correctly, that validity and/or reliability coefficients are less than 1). Ok, that's my anecdote. My antidote has been three-fold:
I have them read "High stakes testing in higher education and employment: Appraising the evidence for validity and fairness" Sackett, Paul R.; Borneman, Matthew J.; Connelly, Brian S.; American Psychologist, Vol 63(4), May-Jun, 2008. pp. 215-227. [Journal Article]. Their abstract: The authors review criticisms commonly leveled against cognitively loaded tests used for employment and higher education admissions decisions, with a focus on large- scale databases and meta-analytic evidence. They conclude that (a) tests of developed abilities are generally valid for their intended uses in predicting a wide variety of aspects of short-term and long-term academic and job perfor- mance, (b) validity is not an artifact of socioeconomic status, (c) coaching is not a major determinant of test performance, (d) tests do not generally exhibit bias by underpredicting the performance of minority group mem- bers, and (e) test-taking motivational mechanisms are not major determinants of test performance in these high- stakes settings. I also spend a little time showing them an outdated version a WISC, and they seem at least get into the test and their initial negativity fades. Or maybe they just like to peer inside the test? Finally, I spend time distinguishing test fairness (i.e. UNfairness) from test bias, and I created an Excel spreadsheet with RAND() generated IQ data for different subgroups that clusters around a regression line that can either show slope/intercept bias or no bias depending on values I enter. We then move into the non-technical issue of "fairness" which has no simple definition .... I am not sure this gets rid of the problems completely, but at least they clarify what some of these terms mean. Interestingly, I heard a NPR piece a few weeks ago (didn't catch the names of the researcher) which claimed that simply NOT calling an IQ test a test of ability or intelligence made major group differences virtually disappear. I am somewhat familiar with a few articles on stereotype threat, but this piece seemed to imply that sub-group differences totally disappear ... did you catch this? I would bet that the data shows lessened group differences with the name change, but would be surprised if all differences disappear. If it was that simple, they would have changed the name years ago! ========================== John W. Kulig Professor of Psychology Plymouth State University Plymouth NH 03264 ==================================================================== GALILEO GALILEI: I do not feel obligated to believe that the same God who has endowed us with sense, reasons, and intellect has intended us to forgo their use. ==================================================================== ----- Original Message ----- From: "Jim Clark" <[email protected]> To: "Teaching in the Psychological Sciences (TIPS)" <[email protected]> Sent: Wednesday, April 28, 2010 12:21:08 PM Subject: [tips] Student conceptions of IQ Hi Do people find their students extremely receptive to anything that might challenge the usefulness, validity, fairness, ... of standard IQ measures? I think I give a fairly balanced presentation on positive (e.g., validity) and negative (e.g., stereotype threat) aspects of IQ tests, but student answers appear to most heavily weight the negative. Any antidote? Take care Jim James M. Clark Professor of Psychology 204-786-9757 204-774-4134 Fax [email protected] Department of Psychology University of Winnipeg Winnipeg, Manitoba R3B 2E9 CANADA --- You are currently subscribed to tips as: [email protected]. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=13338.f659d005276678c0696b7f6beda66454&n=T&l=tips&o=2282 or send a blank email to leave-2282-13338.f659d005276678c0696b7f6beda66...@fsulist.frostburg.edu --- You are currently subscribed to tips as: [email protected]. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=13090.68da6e6e5325aa33287ff385b70df5d5&n=T&l=tips&o=2290 or send a blank email to leave-2290-13090.68da6e6e5325aa33287ff385b70df...@fsulist.frostburg.edu
