Hi Mike et al. - Mike, thanks for the helpful clarification. I have to confess that I was initially confused by your message until I later realized that you we referring not to the press story described in David Hogberg's message, but to the actual study itself, which I hadn't read. So yes, if they did indeed derive the model in one African-American sample and cross-validate it in a second sample (I trust your eagle eye on this one), then I cheerfully withdraw my initial comment about a lack of replication (although I'm not sure I understand, Mike, what you mean by a "problem" with using the term replication in the context in which I was using it, which was the story as reported by the media...sounds instead to me that what you're saying is that the news story neglected to mention the replication) . All the best.....Scott
________________________________________ From: Mike Palij [[email protected]] Sent: Monday, June 14, 2010 8:44 PM To: Teaching in the Psychological Sciences (TIPS) Cc: Mike Palij Subject: RE: [tips] a recent corr/causation example On Mon, 14 Jun 2010 15:16:07 -0700, Scott O Lilienfeld wrote: >This study, which I haven't read, raises another question (in addition to the <correlation-causation question) that I always find interesting to ponder...when >does detecting an effect in one group (e.g., African-Americans) but not another >(e.g., Hispanics) constitute a legitimate statistical interaction vs. a failed >replication? There is a problem in Scott's use of the term "replication". In the study by Sharkey, there are actually two groups of subjects, both composed of African-Americans and Hispanics, but the first group was used for analysis and the second group (from a different source) was used as a "replication sample" on which the statistical model derived from the first group was "validated". It may be best to view race/ethnic group as a between-subject factor in his design, not as a design that tests a model on African-Americans and tries to replicate on Hispanics (Sharkey's design is actually fairly complicated, examining groups of subjects at the "city block" level, census tract level, and neighborhood level). So, ethnic group enters into an interaction with the homicide effect, that is, present for in one group but not the other. This interaction is then replicated in the second sample of data that Sharkey analyses. More relevant concerns include estimating the power for the effect in Hispanics and White subjects. I've just done a quick scan of his article and can't seem to find the relevant sample sizes. A closer reading of the paper might reveal this. Another concern is selection bias though Sharkey claims that this study overcomes this objection. Again, a close reading might substantiate this. However, he is using data based on subjects that were enrolled in studies in Chicago and it is not clear to me that these are random samples from the larger population. Whites in the studies did not experience enough homicides to be included in the analyses but surely there has to be cities with large enough White populations where Sharkey's analyses could be conducted. Sharkey's analysis is a starting point, not a conclusion. As compelling and newsworthy the results are, there may be less here than meets the eye. >And is the author justified in interpreting it as the former >rather than latter, as he seems to be? It's something I struggle with, and >I'd be curious to hear others' thoughts. ....Scott By the way, though Sharkey is affiliated with NYU I don't know him and hadn't heard about him prior to this article. It should be noted that Sharkey is co-author on a previous article in PNAS and he has worked with and had notable researchers review his paper (e.g., Stephen Raudenbush of multilevel analysis fame). -Mike Palij New York University [email protected] --- You are currently subscribed to tips as: [email protected]. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=13509.d0999cebc8f4ed4eb54d5317367e9b2f&n=T&l=tips&o=3088 or send a blank email to leave-3088-13509.d0999cebc8f4ed4eb54d5317367e9...@fsulist.frostburg.edu This e-mail message (including any attachments) is for the sole use of the intended recipient(s) and may contain confidential and privileged information. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this message (including any attachments) is strictly prohibited. If you have received this message in error, please contact the sender by reply e-mail message and destroy all copies of the original message (including attachments). --- You are currently subscribed to tips as: [email protected]. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=13090.68da6e6e5325aa33287ff385b70df5d5&n=T&l=tips&o=3097 or send a blank email to leave-3097-13090.68da6e6e5325aa33287ff385b70df...@fsulist.frostburg.edu
