Forgive this horribly un-sourced response, but there is (and has been, nearly from the get-go) a debate about whether p(rep)'s assumptions are correct in its original formulation. The problem has to do with (iirc) the estimation of effect sizes, which are required for computing p(rep). (Those are the lower-case deltas, I believe, in Gat's blog post.) Gat adds in a later post <http://probonostats.wordpress.com/2007/09/16/p-rep-and-the-myth-of-rati onal-science/> that the error has been pointed out already.
I'm not statistician, but I suspect that this is going to work itself out. The idea is fundamentally sound, and if it's possible to make reasonable assumptions about effect sizes (or the distributions of that variable), then one *should* be able to compute the probability of a replication. I'm waiting for things to calm down a bit and for people far smarter than me to figure it out. Again, I'm not a statistician, but I don't see immediately why that difficulty kills the idea. I'm talking to my stats classes about it so that they'll be aware that it's probably something they're going to see in the future (and if they read APS journals, they'll see now), but I'm not teaching it, per se. It doesn't seem to me that anyone has launched a devastating critique of the idea of it, but rather have found problems with the computation of it. m ------ "There is no power for change greater than a community discovering what it cares about." -- Margaret Wheatley -----Original Message----- From: Christopher D. Green [mailto:[EMAIL PROTECTED] Sent: Friday, November 16, 2007 9:28 AM To: Teaching in the Psychological Sciences (TIPS) Cc: Robert A. Cribbie; David Flora; Michael Friendly; Doba Goodman Subject: [tips] p-rep, mathematical error, pure and simple < Pro Bono Statistics There has been a lot of excitement around a statistic being used in APS journals of late called p-rep. It was developed by Peter Killeen, published in a 2005 issue of Psychological Science. The primary advantage claimed for it is that it gives the average probability of replicating a given effect in future studies, rather than giving the probability, under the null hypothesis, of finding data at least as improbable as that actually found (which is roughly what good ol' p-values give). But now I have found a blog posting by a California statistician named Yoram Gat that claims that Killeen's derivation is based on a "mathematical error, pure and simple." You can find the posting at: http://probonostats.wordpress.com/2007/09/15/p-rep-mathematical-error-pu re-and-simple/ He details what the error is, but I am not statistically sophisticated enough to know whether or not he is correct. Has anyone else come across this? Is Gat right? And, if he is, is the error as devastating to p-rep as he claims? Regards, Chris Green York U. Toronto, Canada --- To make changes to your subscription contact: Bill Southerly ([EMAIL PROTECTED]) ---
