If the expectation of the empirical estimator for mean p-value is a
function of the number iterations that becomes asymptotically unbiased,
then this would explain the simulation results (attachment in the original
posting), since for small number of iterations, some bias would remain, and
only disappear (hence the "stabilization" feeling) when number of
iterations is large.
Thanks again for all for clarifying the issue.
On Tuesday, June 9, 2015 at 7:04:52 AM UTC+8, dcadams wrote:
>
> Yes that is more precise.
>
> In my post to the query I only noted that the variance in significance
> levels across multiple permutation tests decreases as the number of
> iterations increases. Joe's post provides the equation for the expected
> value of that variance; mine provided reference to an empirical example
> (Adams and Anthony, 1996).
>
> Dean
>
> Dr. Dean C. Adams
> Professor
> Department of Ecology, Evolution, and Organismal Biology
>Department of Statistics
> Iowa State University
> www.public.iastate.edu/~dcadams/
> phone: 515-294-3834
>
> -Original Message-
> From: R-sig-phylo [mailto:r-sig-phy...@r-project.org ] On
> Behalf Of Joe Felsenstein
> Sent: Monday, June 8, 2015 1:29 AM
> To: Dennis E. Slice; r-sig-phylo mailman
> Subject: Re: [R-sig-phylo] [MORPHMET] Re: Stability of p-values (physignal
> and testing for morphological integration)
>
> A number of people have suggested that P values should stabilize after a
> number of samples (in a permutation test) that depends on the data set.
>
> I suspect that these were unintended misstatements. As Dennis Slice has
> mentioned, one can regard each permutation in the permutation test as a
> random sample from a distribution. Comparing a test statistic X to its
> value in the data (say, Y), each permutation draws from a distribution in
> which there is a probability P that X exceeds Y.
>
> So each permutation is (to good approximation) a coin toss with
> probability P of Heads. There obviously no number of tosses beyond which
> the fraction of Heads "stabilizes". The fraction of heads after N tosses
> will depart from the true value P by an amount which has expectation 0, and
> variance P(1-P)/N. This is a fairly slow approach of the fraction of Heads
> to the true value.
>
> So to get twice as close to the true P value, one needs 4 times as many
> permutations. And this need for more and more samples continues
> indefinitely. There is no sudden change as one reaches a threshold number
> of permutations.
>
> But that's what you really meant, right?
>
> Joe
> ---
> Joe Felsenstein j...@gs.washington.edu
>
> [[alternative HTML version deleted]]
>
> ___
> R-sig-phylo mailing list - r-sig...@r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-phylo
> Searchable archive at
> http://www.mail-archive.com/r-sig-phylo@r-project.org/
>
--
" PENAFIAN: E-mel ini dan apa-apa fail yang dikepilkan bersamanya ("Mesej")
adalah ditujukan hanya untuk kegunaan penerima(-penerima) yang termaklum di
atas dan mungkin mengandungi maklumat sulit. Anda dengan ini dimaklumkan
bahawa mengambil apa jua tindakan bersandarkan kepada, membuat penilaian,
mengulang hantar, menghebah, mengedar, mencetak, atau menyalin Mesej ini
atau sebahagian daripadanya oleh sesiapa selain daripada
penerima(-penerima) yang termaklum di atas adalah dilarang. Jika anda telah
menerima Mesej ini kerana kesilapan, anda mesti menghapuskan Mesej ini
dengan segera dan memaklumkan kepada penghantar Mesej ini menerusi balasan
e-mel. Pendapat-pendapat, rumusan-rumusan, dan sebarang maklumat lain di
dalam Mesej ini yang tidak berkait dengan urusan rasmi Universiti Malaya
adalah difahami sebagai bukan dikeluar atau diperakui oleh mana-mana pihak
yang disebut.
DISCLAIMER: This e-mail and any files transmitted with it ("Message") is
intended only for the use of the recipient(s) named above and may contain
confidential information. You are hereby notified that the taking of any
action in reliance upon, or any review, retransmission, dissemination,
distribution, printing or copying of this Message or any part thereof by
anyone other than the intended recipient(s) is strictly prohibited. If you
have received this Message in error, you should delete this Message
immediately and advise the sender by return e-mail. Opinions, conclusions
and other information in this Message that do not relate to the official
business of University of Malaya shall be understood as neither given nor
endorsed by any of the forementioned. "
--
MORPHMET may be accessed via its webpage at http://www.morphometrics.org
To unsubscribe from this group and stop receiving emails from it, send an email
to morphmet+unsubscr...@morphometrics.org.