On Mon, 14 Jun 2010 15:16:07 -0700, Scott O Lilienfeld wrote: >This study, which I haven't read, raises another question (in addition to the <correlation-causation question) that I always find interesting to ponder...when >does detecting an effect in one group (e.g., African-Americans) but not >another >(e.g., Hispanics) constitute a legitimate statistical interaction vs. a failed >replication?
There is a problem in Scott's use of the term "replication". In the study by Sharkey, there are actually two groups of subjects, both composed of African-Americans and Hispanics, but the first group was used for analysis and the second group (from a different source) was used as a "replication sample" on which the statistical model derived from the first group was "validated". It may be best to view race/ethnic group as a between-subject factor in his design, not as a design that tests a model on African-Americans and tries to replicate on Hispanics (Sharkey's design is actually fairly complicated, examining groups of subjects at the "city block" level, census tract level, and neighborhood level). So, ethnic group enters into an interaction with the homicide effect, that is, present for in one group but not the other. This interaction is then replicated in the second sample of data that Sharkey analyses. More relevant concerns include estimating the power for the effect in Hispanics and White subjects. I've just done a quick scan of his article and can't seem to find the relevant sample sizes. A closer reading of the paper might reveal this. Another concern is selection bias though Sharkey claims that this study overcomes this objection. Again, a close reading might substantiate this. However, he is using data based on subjects that were enrolled in studies in Chicago and it is not clear to me that these are random samples from the larger population. Whites in the studies did not experience enough homicides to be included in the analyses but surely there has to be cities with large enough White populations where Sharkey's analyses could be conducted. Sharkey's analysis is a starting point, not a conclusion. As compelling and newsworthy the results are, there may be less here than meets the eye. >And is the author justified in interpreting it as the former >rather than latter, as he seems to be? It's something I struggle with, and >I'd be curious to hear others' thoughts. ....Scott By the way, though Sharkey is affiliated with NYU I don't know him and hadn't heard about him prior to this article. It should be noted that Sharkey is co-author on a previous article in PNAS and he has worked with and had notable researchers review his paper (e.g., Stephen Raudenbush of multilevel analysis fame). -Mike Palij New York University [email protected] --- You are currently subscribed to tips as: [email protected]. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=13090.68da6e6e5325aa33287ff385b70df5d5&n=T&l=tips&o=3088 or send a blank email to leave-3088-13090.68da6e6e5325aa33287ff385b70df...@fsulist.frostburg.edu
