On 16 Jan 2003 10:23:59 -0800, [EMAIL PROTECTED] (Conn, Judith) wrote: > I don't know the specifics but I thought you had to take into account that > having multiple tests of hypotheses on the same data and would get false > positives cause multiple tests. Or am I thinking of something else? Judy > Conn [snip: my comment, which quoted another comment. Neither comment reveals to me the immediate salience of this comment about multiple tests.]
Judy, and everyone, I occasionally top-post when I write e-mail, but it does not work well in writing to net-groups. What works most often, it seems to me, is the Quote followed by the Comment. It also does not work very well when the original poster has substance written into the Subject line which is not repeated in the text -- the Subject line was easy to overlook or to ignore, in my experience, even when my Newsreader wrote it out with the message. These days, I have it an inch or two above the text that is supposed to be relevant. Anyway, it seems to me that Judy's comment is directed to the Subject line. > Subject: Re: validity of secondary outcomes with very large sample size Yes, one ought to take multiple testing into account. But if I piggyback my Question onto your expensive data collection, and my 'study' questions are largely unrelated to yours, can't I start with a fresh 5%? - folks do this, and I can't object. And, taking it further, each of us 'piggybackers' (including the original investigator) may want to use a fresh 5% level for two or three other questions that are "unrelated" to some extent .... I think it is okay for a first one, "totally unrelated", but it is not okay for #100 if I don't say *something* about multiplicity. One attempt to control for *excessive* multiplicity is to insist that the *real* questions have to be documented in the original grant application. (Especially, it is hazardous to are safe-inference making if the hypotheses are generated by the peeks at 1000s of data items.) The expectations should be laid out in advance, along with (if possible) the hierarchy of decision making: For instance, if Overall-test #1 does not 'reject', then the Subtests 1a, 1b, 1c are going to be regarded as exploratory and doubtful. (This strategy can be undermined if interim research changes the scientific priorities, in the years between funding and publication.) (I suspect that PhDs are more successful than MDs or other clinicians, when it comes to expressing their *doubts* about 'marginal' clinical findings.) -- Rich Ulrich, [EMAIL PROTECTED] http://www.pitt.edu/~wpilib/index.html . . ================================================================= Instructions for joining and leaving this list, remarks about the problem of INAPPROPRIATE MESSAGES, and archives are available at: . http://jse.stat.ncsu.edu/ . =================================================================
