Hello All,
Its interesting how Scott and Mike P. dismiss the threat to internal
validity as if a meta-analysis balances out the defects. All a
meta-analysis
does is add up the defects.
The Meta-analyses can only present the data that is collected in the
individual studies. As far as I know, no blinded subject has ever been
asked whether they were in the treatment condition or not. No blinded
investigator has ever been asked if they could identify the treated
subjects. I definitely could identify them from their report of side
effects. All the investigators could have done the same by examining
the adverse event
reports.
The observer ratings that Scott refers to can all be influenced by the
same internal defect. All the observers, including parents, know that
the children are taking a medication because of the side effects. All
these same people know when a child is being treated with a behavior
intervention
because it appears very different than a waiting list control or other
control conditions.
All the drug companies and the psychotherapy outcome investigators need
to do is survey the subjects and the investigators to verify the
blinding. They don't do this because they know that these studies can
never be blinded. They interpret the results as if they are.
Until a genuinely blinded treatment study is conducted, all the effect
sizes in all these studies could be the result of the internal bias that
Campbell, Stanley and Shadish so eloquently present.
If anyone can present a study that was correctly blinded, or even
present a way this could be done, it would advance the field 100% since
all the studies done up to this point have presented noise.
The research defect I described doesn't exist among studies in which the
blinding isn't threatened by side effects and other clear indications of the
treatment condition.
Until a genuinely blind treatment study is conducted, these drugs and
psychotherapy interventions have no empirical validation.
No insurance company should pay for these treatments until they are
empirically validated.
Isn't anyone but me curious about why placebos are sugar pills? Why not
try a salt pill? The control condition must be similar to the treatment
condition for humans or they quickly figure out which condition they are
in and they are very influenced by the social setting of research.
Mike Williams
On 9/14/11 1:00 AM, Teaching in the Psychological Sciences (TIPS) digest
wrote:
Mike W. is right to raise useful questions regarding the internal validity of
psychotherapy designs, but I agree with Mike P. that he is wrong to categorically dismiss
all of them simply as "invalid." Surely, no study is perfect, but many of them
yield highly useful inferences. In addition to Mike P.'s endorsement of Don Campbell's
writings on internal validity, I'd like to add Campbell's helpful principle of the
heterogeneity of irrelevancies. The most helpful inferences derive the consilience of
multiple independent studies, all with largely offsetting flaws.
---
You are currently subscribed to tips as: [email protected].
To unsubscribe click here:
http://fsulist.frostburg.edu/u?id=13090.68da6e6e5325aa33287ff385b70df5d5&n=T&l=tips&o=12633
or send a blank email to
leave-12633-13090.68da6e6e5325aa33287ff385b70df...@fsulist.frostburg.edu
