On Tue, 15 Jun 2010 04:11:28 -0700, Scott O Lilienfeld wrote: >Hi Mike et al. - Mike, thanks for the helpful clarification. I have to >confess that I was initially confused by your message until I later >realized that you we referring not to the press story described in >David Hogberg's message, but to the actual study itself, which I >hadn't read.
I apologize for not making clear that I was using Sharkey's article as the basis for my comments. >So yes, if they did indeed derive the model in one African-American >sample and cross-validate it in a second sample (I trust your eagle >eye on this one), then I cheerfully withdraw my initial comment about >a lack of replication To reiterate what Sharkey did: (1) He computed a model/equation on the basis of the African-American sample and then applied it to the White and Hispanic students in his sample. There was too little data for White children for the model to be tested and though there was enough data for Hispanic children, the equation did not apply to them. (2) Sharkey had access to another group of subjects who were participating in another study and replicated the finding that his equation held in the African-American group but not the other groups. (3) It should be noted that without info on the design and analysis that Sharkey used, which was not provided in the news article, it is easy to misinterpret what Sharkey actually did and what objections might have greater weight. >(although I'm not sure I understand, Mike, what you mean by a >"problem" with using the term replication in the context in which >I was using it, which was the story as reported by the media...sounds >instead to me that what you're saying is that the news story neglected >to mention the replication) . The new article provides the following info on the analysis of the different groups: |The effects wear off after a week to nine days, Sharkey found. But |in areas with a lot of crime, this does not provide much relief. | |"When one takes into account the prevalence of homicide in the |most violent neighborhoods in cities like Chicago, these results |mean that some children spend about one week out of every month |functioning at a low level as they navigate the home or school environment," |he said in a statement. | |In general, black U.S. children score about one standard deviation |lower on standardized tests than white children. This finding accounts |for half that difference, Sharkey said. | |He was unable to find enough murders in predominately white |neighborhoods to see if white children were affected. | |Curiously, there were enough murders in Hispanic neighborhoods |but Latino children seemed unaffected. | |"I just didn't find the same effect," Sharkey said. | |It could be the Hispanic children did not identify with the violence, |Sharkey added. "Most of the victims, even in the Hispanic |neighborhoods, were black." | |It is well documented that blacks are far more likely to be murdered |than members of any other U.S. ethnic group -- murder is the most |common cause of death for young black men. The article doesn't mention replication at all, just that the effect seen in the African-American sample was not found in the Hispanic group. The jounral article explicitly distinguishing between the initial sample on which the equation was developed and the second sample where this finding was "replicated". In your original post you said: On Mon, 14 Jun 2010 15:16:07 -0700, Scott O Lilienfeld wrote: >This study, which I haven't read, raises another question >(in addition to the correlation-causation question) that I always >find interesting to ponder...when does detecting an effect in one >group (e.g., African-Americans) but not another (e.g., Hispanics) >constitute a legitimate statistical interaction vs. a failed replication? In Sharkey's study, African-Americans, Hispanics, and Whites constitute a single sample because they were enrolled in the study whose data Sharkey used. The "effect" that Sharkey is testing is actually an equation relating homicide and other variables to cognitivie performance. If this effect/equation does not hold across groups, then one would say that there is an interaction present not that one failed to replicate the result in the other groups. The second sample "replicates" the first sample's findings. If it did not, then one could claim a failure to replicate. Another way to view this situation is to use the concepts of internal validity and external validity. If one has established a real effect (such as Sharkey believes he has done for African-American children), then one can question its internal validity (is the causal relationship correctly specified, no confounding exists, other variables are appropriately controlled for, etc.) and ask how generalizable the effect is, that is, is it found in other groups, a question of external validity. If an effect is found in one group (i.e., African-Americans) but subsequent replications with groups from the same population do not demonstarte the effect, then a failure to replicate has occurred. That is, the initial effect was spurious, a Type I error, or something similar. The failure to replicate identifies the problem as one involving internal validity, not external validity (where the effect is assumed to be reliably found under specific conditions). -Mike Palij New York University [email protected] --- You are currently subscribed to tips as: [email protected]. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=13090.68da6e6e5325aa33287ff385b70df5d5&n=T&l=tips&o=3100 or send a blank email to leave-3100-13090.68da6e6e5325aa33287ff385b70df...@fsulist.frostburg.edu
