well, this is interesting indeed ... for let's say that you did adopt a .1
level for a pilot AND, you just happened to reject the null IN the pilot
... is THAT sufficient justification for committing more time and resources
TO a large main study?? the implication from this pilot result is that ...
this means that the trend would continue in a more dramatic or noticeable
way IN a main study ... and you have no justification for that based on the
p value you have adopted in your pilot study ...
or, think about it this way ... what IF your p value had been .001 based on
your pilot data ... why do any more? well, there are lots of reasons ...
the importance of the p value is so minimal even in the best of
circumstances ... to use it THIS way is even worse.
the main thing you should be doing in a pilot study ... is to first ...
iron out the bugs of the methods and procedures ... after all, someone
should have already approved of the 'idea' ... and here is a chance to get
your ducks in a row with protocols, instrumentation, times for doing
things, etc. the second main purpose would be to see if there is ANY
evidence at all in the generally predicted direction ... FORget the
application of the fancy inferential tests and worrying about alpha ... or
power at this stage ...
At 01:21 PM 03/15/2000 -0800, Andy Avins wrote:
>We proposed a pilot clinical trial that was shot down by a local review
>committee.
are you saying that it was shot down BECAUSE you proposed to use a pilot
alpha of .25??? this is hard to believe ... but, it could be true ... if it
were and that were the ONLY problem ... is this not SO simple to fix???
just change it to .1! i would suspect that there are other more important
issues that were considered ... but since we are not privy to these ... it
is difficult to comment here
> Lacking any other guidance, we arbitrarily chose an alpha of 0.25 for
> doing the power calculations (reasoning that we didn't want to set too
> stringent a standard for rejecting the null and not proceeding with a
> more definitive trial). We were criticized for not adopting a more
> conventional standard of alpha=0.10. I've never heard that there was any
> convention for this sort of calculation.
the basic idea is ... in a pilot ... don't make it AS hard to reject the
null as in a full blown study ... but, the use of .1 as THE value for a
pilot is just as arbitrary as it is to say that we will use .05 or .01 for
the MAIN study ... focusing on alpha is only half the deal ... there is a
type II error too that could be FAR more critical in a given setting than
alpha ...
WHEN WILL WE GET OVER OUR FIXATION ON ALPHA!!!
>Does anyone have any thoughts or references for sample size calculations
>for pilot studies?
>Thanks much in advance!
>--Andy
>--Andy Avins, MD, MPH
>Assistant Professor
>Department of Medicine
>Department of Epidemiology & Biostatistics
>University of California, San Francisco
>E-mail: [EMAIL PROTECTED]
>Tel: 415-597-9196
>
>
>
>
>===========================================================================
>This list is open to everyone. Occasionally, less thoughtful
>people send inappropriate messages. Please DO NOT COMPLAIN TO
>THE POSTMASTER about these messages because the postmaster has no
>way of controlling them, and excessive complaints will result in
>termination of the list.
>
>For information about this list, including information about the
>problem of inappropriate messages and information about how to
>unsubscribe, please see the web page at
>http://jse.stat.ncsu.edu/
>===========================================================================
===========================================================================
This list is open to everyone. Occasionally, less thoughtful
people send inappropriate messages. Please DO NOT COMPLAIN TO
THE POSTMASTER about these messages because the postmaster has no
way of controlling them, and excessive complaints will result in
termination of the list.
For information about this list, including information about the
problem of inappropriate messages and information about how to
unsubscribe, please see the web page at
http://jse.stat.ncsu.edu/
===========================================================================
