If the expectation of the empirical estimator for mean p-value is a
function of the number iterations that becomes asymptotically unbiased,
then this would explain the simulation results (attachment in the original
posting), since for small number of iterations, some bias would remain, and
only disappear (hence the stabilization feeling) when number of
iterations is large.
Thanks again for all for clarifying the issue.
On Tuesday, June 9, 2015 at 7:04:52 AM UTC+8, dcadams wrote:
Yes that is more precise.
In my post to the query I only noted that the variance in significance
levels across multiple permutation tests decreases as the number of
iterations increases. Joe's post provides the equation for the expected
value of that variance; mine provided reference to an empirical example
(Adams and Anthony, 1996).
Dean
Dr. Dean C. Adams
Professor
Department of Ecology, Evolution, and Organismal Biology
Department of Statistics
Iowa State University
www.public.iastate.edu/~dcadams/
phone: 515-294-3834
-Original Message-
From: R-sig-phylo [mailto:r-sig-phy...@r-project.org javascript:] On
Behalf Of Joe Felsenstein
Sent: Monday, June 8, 2015 1:29 AM
To: Dennis E. Slice; r-sig-phylo mailman
Subject: Re: [R-sig-phylo] [MORPHMET] Re: Stability of p-values (physignal
and testing for morphological integration)
A number of people have suggested that P values should stabilize after a
number of samples (in a permutation test) that depends on the data set.
I suspect that these were unintended misstatements. As Dennis Slice has
mentioned, one can regard each permutation in the permutation test as a
random sample from a distribution. Comparing a test statistic X to its
value in the data (say, Y), each permutation draws from a distribution in
which there is a probability P that X exceeds Y.
So each permutation is (to good approximation) a coin toss with
probability P of Heads. There obviously no number of tosses beyond which
the fraction of Heads stabilizes. The fraction of heads after N tosses
will depart from the true value P by an amount which has expectation 0, and
variance P(1-P)/N. This is a fairly slow approach of the fraction of Heads
to the true value.
So to get twice as close to the true P value, one needs 4 times as many
permutations. And this need for more and more samples continues
indefinitely. There is no sudden change as one reaches a threshold number
of permutations.
But that's what you really meant, right?
Joe
---
Joe Felsenstein j...@gs.washington.edu javascript:
[[alternative HTML version deleted]]
___
R-sig-phylo mailing list - r-sig...@r-project.org javascript:
https://stat.ethz.ch/mailman/listinfo/r-sig-phylo
Searchable archive at
http://www.mail-archive.com/r-sig-phylo@r-project.org/
--
PENAFIAN: E-mel ini dan apa-apa fail yang dikepilkan bersamanya (Mesej)
adalah ditujukan hanya untuk kegunaan penerima(-penerima) yang termaklum di
atas dan mungkin mengandungi maklumat sulit. Anda dengan ini dimaklumkan
bahawa mengambil apa jua tindakan bersandarkan kepada, membuat penilaian,
mengulang hantar, menghebah, mengedar, mencetak, atau menyalin Mesej ini
atau sebahagian daripadanya oleh sesiapa selain daripada
penerima(-penerima) yang termaklum di atas adalah dilarang. Jika anda telah
menerima Mesej ini kerana kesilapan, anda mesti menghapuskan Mesej ini
dengan segera dan memaklumkan kepada penghantar Mesej ini menerusi balasan
e-mel. Pendapat-pendapat, rumusan-rumusan, dan sebarang maklumat lain di
dalam Mesej ini yang tidak berkait dengan urusan rasmi Universiti Malaya
adalah difahami sebagai bukan dikeluar atau diperakui oleh mana-mana pihak
yang disebut.
DISCLAIMER: This e-mail and any files transmitted with it (Message) is
intended only for the use of the recipient(s) named above and may contain
confidential information. You are hereby notified that the taking of any
action in reliance upon, or any review, retransmission, dissemination,
distribution, printing or copying of this Message or any part thereof by
anyone other than the intended recipient(s) is strictly prohibited. If you
have received this Message in error, you should delete this Message
immediately and advise the sender by return e-mail. Opinions, conclusions
and other information in this Message that do not relate to the official
business of University of Malaya shall be understood as neither given nor
endorsed by any of the forementioned.
--
MORPHMET may be accessed via its webpage at http://www.morphometrics.org
To unsubscribe from this group and stop receiving emails from it, send an email
to morphmet+unsubscr...@morphometrics.org.
