Re: Bonferroni Correction

[Simon, Steve, PhD]

Michael L. Keaser writes:   > I have two data sets: A and B > > For data set A, I conduct 25 significant tests. Moreover I > conduct 30 tests on data set B. I then create third data set, called C, > by subtracting A from B. Thus: > > Data set C = (Data Set B scores) - (Data Set A raw scores). I > then conduct another 15 tests on this data set C. My question is, > when doing the Bonferroni correction, would the significance > level at p = 0.05 be   Wow, your computer must be working overtime to do all those statistical tests.   The Bonferroni correction is highly controversial and no matter what you do, someone will complain. I list some of these issues on my web page:   http://www.childrens-mercy.org/stats/ask/bonferroni.asp   and you should definitely review some of the references and web pages that I list there.   I suspect that it is more important to decide whether to use Bonferroni at all and that the actual denominator that you use in these corrections is just quibbling about details.   The first question that I have to ask is "Why do you have to run so many tests?"  Whether you use a Bonferroni correction or not, you end up diluting the strength of your research findings. In medicine, we often define two or three primary outcome measures. These are considered the most definitive measures of the efficacy of a new therapy. All other measures are secondary and any results that are significant for these variables would be considered exploratory or would just offer support for understanding how a therapy works.   Do you have a global hypothesis that says something like "The treatment will be considered superior to the control if it is better on any of these 25 or 30 outcomes"? Most of the time, a global hypothesis makes no sense and you would, therefore, not be interested in a Bonferroni adjustment.   Furthermore, most research is already underpowered and Bonferroni makes this ten times worse.   Is the goal of your research to prove a hypothesis or to explore relationships? Exploratory studies rarely use a Bonferroni correction.   Is there some sort of composite variable that you could use that would summarize in general whether your treatment is better than your control?   I can think of several scenarios where a Bonferroni correction makes a lot of sense, but I suspect that none of them apply here. It would help a lot if you would be willing to share some more details about your research, especially what the overall goal of this research is. I've made several assumptions (such as the use of a treatment and control) that may be incorrect.   Steve Simon, [EMAIL PROTECTED], Standard Disclaimer. The STATS web page has moved to http://www.childrens-mercy.org/stats.     ....................