On Wed, May 15, 2013 at 5:08 PM, Jed Rothwell <[email protected]> wrote:
> Joshua Cude <[email protected]> wrote: > > >> As I wrote, it represents the probability that ALL of the replications >>> were the result of error. >>> >> >> > >> No it doesn't. That is true only if all the attempts give replications. >> Look up the binomial distribution, and find someone to explain it to you. >> > > I believe that would only apply if success or failure was random. > You're not following the argument. The claim is that *If* the positive results are from random errors, then the number of positive hits would be very unlikely. But that's not true using 1/3 for the chance of a false positive, because that fits pretty well with the success rate reported by Hubler for example. If the probability of a false positive is 1/3, and you run N experiments, you should expect something close to N/3 false positives. O'Malley's calculation determines the probability of getting N hits in N tries. It's just wrong. > When a cathode fails in a properly equipped lab, they always know why it > failed. They can spot the defect. When there are no defects and all control > parameters are met, it always works. So you need only look at the positive > results, and estimate the likelihood that every one of them was caused by > incompetent researchers making mistakes. > Storms himself says positive results depend on nature's mood. He said here: "Of course it's erratic… created by guided luck." If you're calculating the likelihood of a certain number of hits from errors, you have to consider all the attempts, not just the successful ones. It's elementary. I'm not saying Cravens' bayesian analysis is wrong, though I suspect the assumptions are, but O'Malleys' simplistic analysis teaches us nothing.
