On Tue, 13 Sep 2011 22:50:30 -0700, Mike Wiliams wrote: >Hello All, >Its interesting how Scott and Mike P. dismiss the threat to internal validity >as if a meta-analysis balances out the defects. All a meta-analysis >does is add up the defects.
Clearly, you did not understand what I wrote and the above is a mischaracterization of my position. In a previous post I wrote: |In summary, many research studies are flawed, some fatally flawed |and useless while the flaws in others are not fatal. One can always |check the systematic reviews on the Cochrane Collaboration website |(www.cochrane.org ) to see how many crappy treatment/intervention |studies have been published because the researchers did not address |the different types of threats to the validity of the study.It appears that I made the mistake of assuming that Mike W. were familiar with what a systematic review was. Briefly, it consists of the following: (1) Locating all sources of data relevant to a specific questions (e.g., does this treatment produce beneficial results or does X cause systematic changes in Y). Both published and unpublished results should be obtained -- comprehensiveness in range of coverage is important. (2) Evaluation of the study quality: in medical research a double-blind placebo-control design with random assignment of cases is seen as one "gold standard" because it allows one to infer whether a causal relationship exists between a treatment and an outcome. Threats to internal validity as well as the other validities (e.g., statistical conclusion validity) are identified and studies are then scored on degree to which it allows one to claim that a causal relationship is present. Exclusion/inclusion criteria are identified and used to rule out studies from the meta-analysis or rule them in. It is not unusual to find systematic reviews on Cochrane where all of the available studies were fatally flawed and the analysis stopped at this point. (3) After assuring that one has a group of results where meta-analysis can be meaningfully applied, one can do a meta-analysis and more complex analyses to assess the degree of variability in effect sizes and, if there is significant heterogeneity in effect sizes, using regression to determine if study characteristics (e.g., the age of participants) is related to effect size. One can find more information about this on the Cochrane Collaboration website I cite above. For the Reader's Digest crowd, one can read the Wikipedia entry (yadda-yadda) on Systematic Review; see: http://en.wikipedia.org/wiki/Systematic_review One text that I like because it is well-organized and the writing is clear and accessible is Egger et al's "Systematic Reviews in Health Care" which is available in preview mode on books.google.com; see: http://tinyurl.com/eggersmetaanalysis And one can purchase a copy on Amazon; see: http://www.amazon.com/Systematic-Reviews-Health-Care-Meta-Analysis/dp/072791488X -Mike Palij New York University [email protected] P.S. Thanks to the Tipster who in email pointed out that I was right about Tulving's "Elements of Episodic Memory" -- I missed it because I remembered the difference in fonts as being more pronounced. Maybe it's time for me to get reading glasses. --- You are currently subscribed to tips as: [email protected]. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=13090.68da6e6e5325aa33287ff385b70df5d5&n=T&l=tips&o=12654 or send a blank email to leave-12654-13090.68da6e6e5325aa33287ff385b70df...@fsulist.frostburg.edu
