Good points!
There's a lot of faith based practice out there
('the problem is serious, therefore the treatment must be effective')On Sep 14, 2011, at 12:50 AM, Mike Wiliams wrote: > Hello All, > > Its interesting how Scott and Mike P. dismiss the threat to internal validity > as if a meta-analysis balances out the defects. All a meta-analysis > does is add up the defects. > > The Meta-analyses can only present the data that is collected in the > individual studies. As far as I know, no blinded subject has ever been asked > whether they were in the treatment condition or not. No blinded investigator > has ever been asked if they could identify the treated subjects. I > definitely could identify them from their report of side effects. All the > investigators could have done the same by examining the adverse event > reports. > > The observer ratings that Scott refers to can all be influenced by the same > internal defect. All the observers, including parents, know that the > children are taking a medication because of the side effects. All these same > people know when a child is being treated with a behavior intervention > because it appears very different than a waiting list control or other > control conditions. > > All the drug companies and the psychotherapy outcome investigators need to do > is survey the subjects and the investigators to verify the > blinding. They don't do this because they know that these studies can never > be blinded. They interpret the results as if they are. > > Until a genuinely blinded treatment study is conducted, all the effect sizes > in all these studies could be the result of the internal bias that > Campbell, Stanley and Shadish so eloquently present. > > If anyone can present a study that was correctly blinded, or even present a > way this could be done, it would advance the field 100% since > all the studies done up to this point have presented noise. > > The research defect I described doesn't exist among studies in which the > blinding isn't threatened by side effects and other clear indications of the > treatment condition. > > Until a genuinely blind treatment study is conducted, these drugs and > psychotherapy interventions have no empirical validation. > > No insurance company should pay for these treatments until they are > empirically validated. > > Isn't anyone but me curious about why placebos are sugar pills? Why not try > a salt pill? The control condition must be similar to the treatment > condition for humans or they quickly figure out which condition they are in > and they are very influenced by the social setting of research. > > Mike Williams > > > > On 9/14/11 1:00 AM, Teaching in the Psychological Sciences (TIPS) digest > wrote: >> Mike W. is right to raise useful questions regarding the internal >> validity of psychotherapy designs, but I agree with Mike P. that he is wrong >> to categorically dismiss all of them simply as "invalid." Surely, no study >> is perfect, but many of them yield highly useful inferences. In addition to >> Mike P.'s endorsement of Don Campbell's writings on internal validity, I'd >> like to add Campbell's helpful principle of the heterogeneity of >> irrelevancies. The most helpful inferences derive the consilience of >> multiple independent studies, all with largely offsetting flaws. > > > --- > You are currently subscribed to tips as: [email protected]. > To unsubscribe click here: > http://fsulist.frostburg.edu/u?id=13438.3b5166ef147b143fedd04b1c4a64900b&n=T&l=tips&o=12633 > or send a blank email to > leave-12633-13438.3b5166ef147b143fedd04b1c4a649...@fsulist.frostburg.edu --- You are currently subscribed to tips as: [email protected]. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=13090.68da6e6e5325aa33287ff385b70df5d5&n=T&l=tips&o=12650 or send a blank email to leave-12650-13090.68da6e6e5325aa33287ff385b70df...@fsulist.frostburg.edu
