Just a few points: (1) I guess that before going to bed folks should read Campbell and Stanley and/or Cook and Campbell and/or Shadish's updating of their material because the issues raised by Mike Williams fall under the topic of "threats to internal validity". Specifically, (a) diffusion or imitation of treatments and (b) compensatory rivalry or resentful demoralization. In both of these situations, knowledge of the different treatments used in a between-subjects design, can affect how subjects respond or fail to respond to treatments. It should be noted that drug treatment studies can be conducted with within-subject designs such as crossover designs where one group receives a drug treatment first and, after a washout period, receives a placebo treatment. Another group has placebo first and drug later. In any event, a competent researcher will make sure that the design they use addresses threats to the different types of validity involved in the study and try to make sure that their effect is negated or minimized. Here is link to Pedhazur & Schmelkin's treatment of validity issues in their text "Measurement, Design, and Analysis": http://tinyurl.com/PedhazurIntValid
(2) It might be a somewhat useful to follow the research heuristic that "all treatment/medication studies involving human are invalid" but, as with all heuristics, there will be situations where it fails and situations where it is right but for the wrong reasons. In the example that Williams uses below, it is clear that in an inpatient/ward study, one has to guard against the threats I identify above. But if one uses an outpatient population where the participants have no contact with each other, it is hard to see the merit in Williams' critique. There are many different ways to conduct a drug/treatment/intervention study and the quality of that research is dependent upon what the person in charge knows about research design, the different types of validities involved as well as threats to them, and the degree of control one has over the research situation. (3) A minor point: I would assert that though one's own personal experience is, perhaps, a useful guide to think about things, it does not necessarily constitute a valid guide. It is helpful to remember that "anecdotes do not constitute data" (unless, of course, one is doing a qualitative study where subject's statements are the data). Williams does not provide the results of the study he was involved which is odd because he suggests that the study should have produced statistically significant results. If the study did, the logical follow-up question would be was the effect obtained with other types of designs (e.g., out patient studies where subjects did not have contact with each other). The objection that Williams makes is limited to a particular research situation and does not hold for other designs where the threats are not relevant. This one point invalidates the "universality" of his claims that all drug studies are all invalid. (4) I have conducted the statistical analysis for a few drug studies as represented in the following publications: (a) The effectiveness of antianxiety medications (after participating receiving a sodium lactate infusion to see if it provoked a panic attack): http://tinyurl.com/Liebowitz1984 (b) Whether fenfluramine positively affects behavior and attention in young children with autism: http://www.springerlink.com/content/t152543684578671/ (c) The use of bromocriptine in the treatment of cocaine addiction: http://onlinelibrary.wiley.com/doi/10.1111/j.1521-0391.1997.tb00392.x/abstract One of the things that one learns from analyzing the data from such studies is that subjects can have very different responses to treatment, sometimes contrary to what "commonsense" would lead one to expect, such as finding that the placebo condition has MORE side effects than the drug condition. One should also keep in mind that though one can engage in a "guessing game" about which treatment one is receiving, people are not 100% accurate in making such predictions and there probably are drugs where the effects are not perceptible or the effects build up slowly over time and the day to day changes are not noticeable (think of the frog in pot on the stove where the heat is low but eventually comes to a boil). In summary, many research studies are flawed, some fatally flawed and useless while the flaws in others are not fatal. One can always check the systematic reviews on the Cochrane Collaboration website (www.cochrane.org ) to see how many crappy treatment/intervention studies have been published because the researchers did not address the different types of threats to the validity of the study. Trying to claim that all studies are invalid or all studies are valid is logically invalid from an inductive perspective -- it is as foolish as claiming that "All swans are white". Those without experience with black swans will swear the "all swans are white" if that has been their lifelong experience. -Mike Palij New York University [email protected] ------------------ Original Messages ------------------------------ On Mon, 12 Sep 2011 23:24:29 -0700, Mike Williams wrote: Hello All. When I was a grad student, we were conducting a clinical trial of Imipramine vs Xanex in the treatment of severe depression. The study was conducted on an inpatient research unit in the hospital. The patients lived there and I noticed that they would sit in the day room in the evenings and discuss their treatment. Although the medications were assigned randomly and the researchers did not know the assignment, the patients with dry mouth and constipation knew they were taking the medications. Those given placebo knew this because they did not suffer constipation and dry mouth (the anticholinergic side effects). The patients knew which treatment they were receiving and they communicated this to the investigators because the investigators constantly monitored the side effects. The constant monitoring of side effects unblinds the study. This happens in every clinical trial of psychotropic medications. This problem is even more obvious in every clinical trial of psychotherapy. All these studies are invalid. I could explain why they are invalidated by referring to the gigantic literature on expectation biases. Since all the dependent measures involve a judgement by the patient or the investigator that the disorder got better or worse, they are all influenced by the expectation bias that the treatment worked. I think many subjects want to help the researchers and they endorse small positive changes on the dependent measures. The people who get placebo behave consistent with this because they know they never got treatment. All the investigators have to do is anonymously survey the subjects. The results will blow their minds. To my knowledge, this obvious, simple assessment has never been made. Now you may be able to understand why the treatment effect size today for antidepressants is the same as the placebo effect for some studies in the past - its all noise. Mike Williams ______________________________________________________ Hi Mike: This is a very interesting point but I am not sure that I follow the argument completely. Please expand your argument, dotting the 'i's and crossing the 't's. Ken On 9/12/2011 3:00 AM, Mike Wiliams wrote: Clinical Psychology psychotherapy and psychotropic medication therapies will never have sufficient empirical support simply because the subjects are never blind to the treatment condition. ************************* All the investigators are doing is training the subjects to endorse change on the dependent measures. ************************** That's why the meta-analyses conclude that any therapy is effective. I have never seen an analysis that addressed this research problem. It's similar to the obesity researchers who never notice that their entire field is based on the dieting behavior of young women. Mike Williams Drexel University --------------------------------------------------------------- --- You are currently subscribed to tips as: [email protected]. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=13090.68da6e6e5325aa33287ff385b70df5d5&n=T&l=tips&o=12615 or send a blank email to leave-12615-13090.68da6e6e5325aa33287ff385b70df...@fsulist.frostburg.edu
