--- Phil Steitz <[EMAIL PROTECTED]> wrote: > Mark R. Diggory wrote: > > Yes, unfortunately, this is a poor test which can fail randomly, it > > think it needs some more work so that it doesn't cause building to fail > > > when it does fail. > > As the output says, the test will fail with probability 1 in 1000, even > if > there code is working properly. What is being tested is the stochastic > independence of randomly generated hex strings, using a statistical test > (chi-square). I could > > (a) modify the test to just log / systout the anomoly instead of failing > (b) remove this and other similar tests (bad idea, IMHO) > (c) omit all of the tests from the build target used by the nightly build > > (what I was planning to do for gump eventually) > (d) reduce the sensitivity of the test (so that is is, say 1 in 1000000 > failure -- also a bad idea, IMHO) > (e) don't worry, be happy, the build will succeed tomorrow :-) > > I guess that I favor (a) for the nightlies, but (c) for gump. > > Phil >
Just had a better idea: (f) repeat the test if it fails and "really" fail only if two consecutive failures occur. This will reduce the probability of a false failure to miniscule without trashing sensitivity. I will do this. Phil > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > __________________________________ Do you Yahoo!? Yahoo! Search - Find what you�re looking for faster http://search.yahoo.com --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
