Posted by Todd Zywicki:
SOFT DRINKS AND STEVEN MILLOY:

   Daniel over on Crooked Timber takes a [1]whack at Steven Milloy's
   [2]critique of the soft drink study that was discussed last week.
   (Sorry, I had my first week of classes this week, so I'm just getting
   back to all this now). Obviously, Steven Milloy can defend himself if
   he wants, but I figured a few points of elaboration on [3]my original
   post regarding his essay are warranted. For the record, I don't know
   Milloy or have any other contact with him except for having read the
   essay in question.
   The testable hypothesis in the study is the null hypothesis that an
   increase in soft drink consumption like that in the study (less than
   one per week to one or more per day) will dramatically increase the
   risk of weight gain and type 2 diabetes. Milloy's piece critiques the
   study in two ways. First, it argues that there were major
   methodological problems with the study that undermine its reliability,
   and that once corrected for, many of the findings of the study are
   much more questionable than touted. Second, that whatever the findings
   of the study, the authors have oversold their conclusions, in that the
   conclusions that they draw are not supportable by what the study
   actually found. I characterized the overall critique as "devastating,"
   a characterization with which Crooked Timber took issue.
   First, Crooked Timber states that Milloy is "a bit of a hack, who got
   his starat with abit part towards the end of the single largest and
   most impressive work of intelelctual dishonesty of the previous
   century, the effort to discredit the scientific work on the linke
   between tobacco and lung cancer." [4]Others open their responses to
   Milloy in a similar fashion.
   Now, Milloy's article is a critique--he is saying that the authors of
   the soft drink study have failed to carry the burden of rebutting the
   null hypothesis that soft drink consumption causes diabetes. I have no
   idea why Milloy's position on smoking would make the JAMA article on
   soft drinks more or less persuasive. Either it is methodologically
   correct or it isn't--the article stands or falls on its own merits.
   Any defects that Milloy may or may not have does not increase or
   decrease the JAMA article a whit.
   So the real issue is the merits of the critique. First, Milloy says
   that the once the researchers "statistically adjusted their results
   for bodyweight (a risk factor for dabetes) and for caloric intake (a
   proxy measure of consumption of sweetened foods other than soda), the
   83 percent increase [in type 2 diabetes prevalence] dropped to an even
   more statistically dubious (and soft-pedaled) 32 percent increase."
   Now it seems to me that Milloy is obviously correct here--bodyweight
   and non-soda caloric intake seem to me to obviously relevant to trying
   to isolate the marginal effect of the increased soda consumption. So
   the 83 percent figure is really an irrelevant number--nonetheless, the
   [5]Washington Post reported on page 1, in the second paragraph of the
   article, that those who drank more than one serving a day "had more
   than an 80 percent increased risk of developing Type 2 diabetes" than
   those who had less than one a month. Then a few days later the Post
   stated in an [6]editorial, "those who had one or more drinks
   containing sugar or corn syrup per day were 83 percent more likely to
   develop Type 2 diabetes than those who drank less than one such drink
   per month."
   But, the reader might object, the researchers cannot be responsible
   for how the Washington Post characterizes their research. Well, it
   turns out that the Post story was lifted directly from the [7]press
   release from the Harvard School of Public Health, which states in
   paragraph 2, "Those who reported drinking sugar-sweetened sodas more
   than once per day showed an increased risk for type 2 diabetes of more
   than 80 percent compared to women in the study who drank less than one
   per month...." the 32 percent figure, by contrast, appears nowhere in
   the press release.
   Ok, so assume for the moment that Milloy is correct that there is a 32
   percent increase in the prevalence of type 2 diabetes in the subject
   population. Here's the excerpt from Milloy that Daniel quotes, "When
   the researchers statistically adjusted their results for bodyweight (a
   risk factor for diabetes) and for caloric intake (a proxy measure for
   consumption of sweetened foods other than soda), the 83 percent
   increase dropped to an even more statistically dubious (and
   soft-pedaled) 32 percent increase. That result is of the same
   magnitude as the study's reported 21 percent increase in diabetes
   among consumers of more than one diet soft drink per day."
   This is where Daniel gets to the heart of his critique, writing about
   this passage: "Think about this for a second. According to Milloy, the
   correct (even the ethical) thing to have done in presenting the
   results of this study would have been to have headlined the '32 per
   cent increase' (1.32 relative risk) that one gets in a model which
   controls for body mass and caloric intake. This is equivalent to
   suggesting that the correct way to think about the health risks
   associated with soft drinks is to deal with a model under which
   somebody goes from drinking one can of Coke a week to more than one
   per day, but reduces their consumption of other foods so as to
   maintain a constant total caloric intake. Given that the entire reason
   why people worry about soft drink consumption is the sugar in the
   drinks, does this make any sense at all?"
   Perhaps I have misread Milloy's point, but I don't read the claim at
   all the same way as Daniel does. Now what I understand Milloy to be
   saying is that those who drank more than one REGULAR soft drink per
   day had a 32% increase in diabetes prevalence, whereas those who drank
   more than one DIET soft drink per day had a 21% increase. The point,
   as I understood it, is that there was a noticeable increase in
   diabetes prevalence REGARDLESS of whether a person drank alot of
   regular or diet soda. Moreover, there is little difference in the
   increased prevalence between the two different types (its not clear to
   me if the 32% versus 21% is a statistically significant
   difference--Milloy's point seems to be the more colloquial one that
   the difference is not very large). Milloy's conclusion, therefore, is
   that the study has not shown that it is the sugar in the regular soft
   drinks that is causing the increased diabetes prevalence, because
   there is an increased diabetes prevalence from soft drink consumption
   regardless of whether it is sugared or non-sugared. If so, Milloy
   states that this suggests that some other independent factor is to
   blame, such as lifestyle or genetics, such that those who drink a lot
   of soda (regardless of which type) to be prone to contracting type 2
   diabetes. A hypothesis would be something like those who drink alot of
   soda don't exercise as much or don't eat as healthy. I have no idea
   whether this is true or not, but if Milloy is correct then it suggests
   that the study misses the point by focusing on sugared drinks. It
   seems to me this is also why Milloy says that it matters what people's
   other caloric consumption is--if soda consumption is corellated with
   and is a proxy for other lifestyle habits, we would want to know what
   people's other caloric consumption (note that the reference point is
   non-soft drink calorie consumption).
   Now that's how I understand the Milloy critique, which seems quite
   powerful to me if correct. To be honest--and I'm not being rhetorical
   here--I honestly can't make sense out of how Daniel is characterizing
   how he understands the claim made by Milloy here. I understand the
   point of Milloy's critique to be the comparison between regular and
   diet soda consumption and what that says about the role of sugar in
   the study. Daniel interprets Milloy as making some othe point about
   increasing soda consumption and decreasing other foods and the proper
   statement of the diabetes prevalence, and I don't read Milloy that
   way. Daniel says that the model (as he summarizes it) is a "stupid
   model." If Daniel is correct in the way he characterizes Milloy's
   point, then as I noted, it doesn't really make sense to me either. But
   it seems to me that what Milloy is intending to say, and actually says
   as I read it, is not actually stupid. As a general practice, if we
   assume that an author is trying to make a serious claim, it seems to
   me that a more logical intellectual policy is to adopt the reading
   that construes Milloy's claim in a manner that makes sense (leaving
   aside whether it is right or wrong), rather than to construct it in a
   manner that does not make sense or is "stupid." In other words, given
   the choice between construing a claim as a straw man versus a serious
   claim, it is a better practice to adopt the latter reading on the
   assumption that is what the author intended.
   In addition, other credible commentators have made critiques of the
   study that are similar to Milloy's. A [8]story on a Bay Area TV
   website referred to the comments of Karmeen Kulkarni of the American
   Diabetes Association who said that similar results might be found if
   researchers studied another food with little nutritional value, such
   as chips, cakes or cookies. Kulkarni, like Milloy, stressed that
   lifestyle factors are critically important in determining the effect
   of soft drink consumption. The story states, "She [Kulkarni] said
   women in the study who drink more sugary beverages tended to live a
   less healthy lifestyle -- smoking more, working out less, eating more
   calories and less fiber and protein. The research also relied on the
   women to write down what they ate, making it less reliable."
   Milloy similarly notes that the study does not control for genetics or
   lifestyle issues--although he goes on to add that the "real
   explanation for the reported weight gain more likely lies in teh
   women's geneteics and their overall lifestyles." Now, I don't know
   whether that last part is true or not--but then again, all I have ever
   claimed is that Milloy's critique of the weaknesses in the study and
   its conclusions are very strong. I have never said that in this piece
   Milloy offers a compelling alternative hypothesis or support for an
   alternative hypothesis. But it seems obvious to me that lifestyle
   matters alot (just as Kulkarni observes) and that it would be pretty
   easy to control for that. It is less obvious to me how much genetics
   matter (could be a little or a lot) and it might be much more
   difficult to control for genetics. Nonetheless, it is obvious that
   these factors should be controlled for.
   The quality of this study stands or falls on its own merits. I think
   that Milloy has identified some very serious methodological flaws in
   the construction of this study and the conclusions that are drawn from
   it. The authors of the study appear to be promoting the most expansive
   and least-supportable interpretation of the study and seem to have
   done little to address serious alternative interpretations of the data
   reported in the study. What Milloy may have said about other issues
   such as smoking, or what his own hypothesis is on this issue, is
   largely beside the point for purposes of my assessment that this
   remains a very serious critique of the study.

References

   1. http://www.crookedtimber.org/archives/002420.html
   2. http://www.foxnews.com/story/0,2933,130263,00.html
   3. http://volokh.com/archives/archive_2004_08_28.shtml#1093701222
   4. http://www.johnquiggin.com/archives/001909.html#more
   5. http://www.washingtonpost.com/wp-dyn/articles/A29434-2004Aug24.html
   6. http://www.washingtonpost.com/wp-dyn/articles/A42713-2004Aug28.html
   7. http://www.hsph.harvard.edu/press/releases/press08242004.html
   8. http://www.kron4.com/Global/story.asp?S=2214898

_______________________________________________
Volokh mailing list
[EMAIL PROTECTED]
http://highsorcery.com/cgi-bin/mailman/listinfo/volokh

Reply via email to