Hi James M. Clark Professor of Psychology 204-786-9757 204-774-4134 Fax [EMAIL PROTECTED]
>>> Ken Steele <[EMAIL PROTECTED]> 30-May-08 12:11 PM >>> Mike Palij wrote: > On Thu, 29 May 2008 12:50:40 -0700, Ken Steele writes: >> A review article of these issues has appeared in timely fashion- >> >> Sackett, P. R., Borneman, M. J., & Connelly, B. J. (2008). >> High-stakes testing in higher education and employment: Appraising >> the evidence for validity and fairness. American Psychologist, 63 (4), >> 215-227. > > But I'd like to raise a larger issue and get feedback on the it. > One of the key problems in analyzing the relationship between > the SAT and measures of academic performance, such as GPA > at the end of the first year, is that of range restriction: colleges > usually do not admit people with SAT scores belows a certain > value so the range of values for both the SAT and GPA are both > restricted. However, consider the following two situations: > I agree with Mike that this is an issue that is poorly understood. I just created a data set for a colleague to simulate this effect. The set was a simulation of the relationship between SAT and HS GPA. The original set began with N = 100 and r = 0.8. I started reducing the set size based on hypothetical college admission admission cutoffs. By the last slice (OldIvy standards, SAT > 1399, N = 6), r = 0.19. JC: Ken, did you plug the observed values into the formula for correction to see how well it reproduced the .80? My suspicion is that the adjustment would at least be "technically" correct in this sense, although more subjects in the simulation might have made agreement even more likely. I guess I do not understand the concerns about this correction (or the mention of a similar concern about ANCOVA) raised by Mike. IF all students were accepted to university and IF all other things (e.g., standards) remained constant, then this is the correlation we would observe. So the number represents how well a test would predict in the absence of restriction of range and without other changes occurring. Someone posted an example of the GRE and grad studies that exactly fit this scenario. That strikes me as an appropriate indicator of the impact of restriction of range on the correlations. Certainly any number of other changes could impact the correlation (e.g., giving grades and degrees on the basis of height would presumably have a marked effect on the relationship between test scores and grades because the grades would no longer represent what they did previously). But what does that have to do with the relationship between test scores and grades, as currently defined, absent a restricted range? Take care Jim --- To make changes to your subscription contact: Bill Southerly ([EMAIL PROTECTED])
