As promised in my last post in this thread, I provide below a brief review of the Boutron et al JAMA paper for the lay public. In doing so, I will answer the questions that needed to be answered in the first place. These answers should make it clear to those who have followed this thread why the article by Mark Hyman on Huffington Post was a misrepresentation of many aspects of that paper. I have already given you examples of the misrepresentations in the following prior post in this thread:
http://lists.goanet.org/pipermail/goanet-goanet.org/2010-October/200051.html The title of the paper that I am referring to below is: "Reporting and Interpretation of Randomized Controlled Trials With Statistically Nonsignificant Results for Primary Outcomes". It was published in the Journal of American Medical Association by Isabelle Boutron, Susan Dutton, Philippe Ravaud and Douglas G. Altman on May 26, 2010. 1. What was the paper about? The paper was an attempt to examine the validity of the prior tacit understanding that subjective bias enters into the reporting of negative results of drug treatment trials. This understanding was supported to some extent by earlier studies of others. 2. What did the authors do? They selected 72 out of 616 papers dealing with what are known as randomized controlled trials (RCTs) published in December 2006. Only 72 papers were selected because these contained non-significant or negative primary results, which was what the authors were focusing on. Two researchers independently read these 72 papers, and subjectively assessed whether the authors had spun the interpretation of the negative results. Reading the description of the methods it is clear that the following excerpts from Mark Hyman's article were misrepresentations: "....analyzed in detail 72 of those they considered to be of the highest quality." ......Mark Hyman "The authors of this report did not just read the abstracts and conclusions of the studies they reviewed, but independently analyzed the raw data." ......Mark Hyman 3. What did the authors find? They found that what they defined as spin in the interpretation of negative results was present in as many as 42 of these papers in one of their sections, and more than 40% of them in at least 2 sections in the main text. One caveat/limitation of their findings, which they mention is that the two researchers did not always agree on the presence of spin in the different sections. Accordingly, they state that their reproducibility was moderate. Reading these findings it is clear that the following statement of Mark Hyman is highly misleading: "They found that 40 percent of the articles misrepresented the data in the abstract or in the main text of the study. Furthermore they uncovered that in cases where studies had negative outcomes--in other words, the treatment studied DID NOT work--the scientists authoring the studies created a "spin" on the data that showed the treatments DID work." .....Mark Hyman 4. What did the authors infer from their findings? They inferred that in reporting of negative or non-significant treatment outcomes the authors of many studies with such negative results consciously or subconsciously introduce distortion or spin to make the most of those outcomes. They also made the following observation: "Our results are consistent with those of other related studies showing a positive relation between financial ties and favorable conclusions stated in trial reports." ....Boutron et al. However, when they were challenged to substantiate the above statement with actual data by two other authors of a subsequent comment on their paper, they had to retract the statement. Here is what they wrote in their retraction: "The statement in our "Comment" section that was noted by Allison and Cope was too strong. Because of small numbers and missing data, we cannot draw any clear conclusion on the relation between funding source and the presence of spin." .....Boutron et al. 5. What are the limitations of their approach, findings and conclusions? In addition to the above retraction, and the already mentioned caveat, the authors themselves stated the following limitations of their paper: i) That their assessment is subjective, and there may be disagreements between different researchers/authors on their conclusions. ii) That they cannot say whether the spin was deliberate or because of lack of knowledge or both. iii) That they cannot tell whether the spin had any effect on readers and peer reviewers. Reading this and the above there should be no doubt that the following statement of Mark Hyman is a gross misrepresentation: QUOTE In plain language, 40 percent of the studies we count on to make medical decisions are authored by scientists who act as "spin doctors" distorting medical research to suit personal needs or corporate economic interests. "Spin" can be defined as specific reporting that could distort the interpretation of results and mislead readers. If the conclusions in 40 percent of the papers published in medical journals are being spun toward independent interests, how can we consider the medicine we are practicing "evidence based?" UNQUOTE .....Mark Hyman So, if you count the number of words that Hyman has spent on describing this paper in the Huffington Post, you will find that out of 262 words, not counting his direct quotes from the paper, 179 words make up blatantly false or misleading statements. Cheers, Santosh --- On Sun, 10/17/10, Gilbert Lawrence <[email protected]> wrote: > > Any person with or without scientific background can read > for themselves my post > analyzing the JAMA paper. My post was quoting the relevant > parts of the data > from the JAMA paper; that the PAPER ITSELF had analyzed > and presented. > > Santosh's blah .. blah of the JAMA paper and his comments > of my review (of the > same paper) does not quote a single factoid of > the analyzed data. So in > claiming that I am unable to do scientific blah ... blah > ... blah, Santosh is > really smearing the authors of the JAMA paper (for those > who need spoon- > feeding). The JAMA paper corroborates what Dr. Hyman > wrote. Or, Dr. Hyman's > article corroborated what the JAMA paper reported. That > is again spoon-feeding > to those who needed it. > > > Please read Santosh's past bogus comments that Dr. Hyman > was misrepresenting the > JAMA paper. From my reading and presentation, there was > nothing in the JAMA > paper which contradicted the conclusions of Hyman's > article. Thus Santosh's > claim (when HE recommended we read the JAMA paper), is > totally baseless that any > reader, "should be able to recognize quite well that Hyman > has misrepresented > many methodological and technical aspects of the JAMA paper > in his article. In > fact, he has resorted to more spin than the targets of the > paper." > > > With sadness in my heart, I have to reiterate that Santosh > once again reaffirms > that he does not read, understand and digest what is > written. With technical > words, Santosh is faking his expertise in analyzing > clinical papers. > > > No longer is Dr. Hyman's paper the topic of discussion. > Santosh has changed the > focus of arguments and discussion (if one can dignify his > writings) to > EVERYTHING BUT the original paper on "medical 'Science > for Sale'" - once again > spoon-feeding is necessary on the topic of the thread. He > continues to do what > he does best - smear, distortions and bogus comments. > > Santosh, keep up your reputation as Goa's 'premier' Master > at smear, distortion > and bogus comments. You know how Dilbert says we should > deal with such > writings. > > Regards, GL > >
