On Sun, Feb 21, 2010 at 3:12 PM, Phil Steitz <phil.ste...@gmail.com> wrote: > Phil Steitz wrote: >> Niall Pemberton wrote: >>> The last release of Lang was in March 2008 and there have been quite a >>> few bug fixes and enhancements since then. Many of the changes in the >>> current trunk which are compatible have been backported to the Lang >>> 2.x branch to create this release. >>> >>> [ ] +1 Yes go ahead an release based on RC1 >>> [ ] -1 No, because... >>> >>> Lang 2.5 RC1 is available for review here: >>> http://people.apache.org/~niallp/lang-2.5-rc1/ >>> >>> Maven artifacts are here: >>> http://people.apache.org/~niallp/lang-2.5-rc1/maven/commons-lang/ >>> >>> Details of changes since 2.4 are in the release notes: >>> http://people.apache.org/~niallp/lang-2.5-rc1/RELEASE-NOTES.txt >>> http://people.apache.org/~niallp/lang-2.5-rc1/site/changes-report.html >>> >>> I have tested this with JDK 1.3 & 1.4 using ant and JDK 1.5 & 1.6 using >>> maven2. >>> (note JVMRandom tests failed for JDK 1.3 & 1.4 - but I re-ran a couple >>> of times and both passed) >>> >>> >>> The tag is here: >>> http://svn.apache.org/viewvc/commons/proper/lang/tags/LANG_2_5_RC1/ >>> >>> Site: >>> http://people.apache.org/~niallp/lang-2.5-rc1/site/ >>> (note some *relative* links are broken and the 2.5 directories are >>> not yet created - these will be OK once the site is deployed) >>> >>> Clirr Report (compared to 2.4): >>> http://people.apache.org/~niallp/lang-2.5-rc1/site/clirr-report.html >>> >>> RAT Report: >>> http://people.apache.org/~niallp/lang-2.5-rc1/site/rat-report.html >>> >>> tia >>> >>> Niall >> >> Sigs, hashes are good >> Clirr is good >> Tag matches src >> Tested build on Apple JDKs 1.4.2_18, 1.5.0_19, 1.6.0_17 >> Sun JDKs 1.4.2_19, 1.5.0_22, 1.6.0_17 (using Ant for 1.4 JDKs). >> >> I did see one RandomUtilsTest failure (Apple JDK 1.6). These tests >> are failing at too high a frequency, which I am investigating; but I >> do not see this as a showstopper, so >> >> +1 >> >> Phil >> > > I hate to do this; but I would like to change my vote to +0. The > random data tests are failing at a high enough frequency to be > annoying / alarming to users. > > I investigated the high incidence of test failures and found nothing > wrong with what the tests are doing and nothing to indicate > systematic bias in the data being generated; but the addition of 40+ > chi-square tests in the test methods added in r907159 makes the > probability of failure in a given run > 1/25. This is why there is > a high incidence of test failures. > > I verified that failures appear to be evenly distributed (too many, > too few even/edd, too many, too few above/below range midpoints) and > that the chisquare statistics are being computed correctly, with the > right critical values applied. > > If you do cut another RC, I would recommend one of the following: > > 1) Grab / copy and extend [math]'s RetryTestCase (will cut incidence > of failure in half) > > 2) Disable the stochastic test cases for the release > > 3) Reduce sensitivity of the chi-square test (change to e.g., .0005 > level of significance) > > 4) Reduce the number of tests > > My recommendation is 2) - leave in the source but comment out. The > tests are valuable as they would fail regularly and miserably if > there were systematic bias (as there used to be on odd/even); but > without reducing significantly the number of tests or the > sensitivity (or limiting to a single "successful" PRNG sequence), > there is no way to leave them all in without generating an > annoyingly high rate of random failures.
Thanks for looking at this Phil. I've gone which your suggestion and excluded the tests from the normal execution in the Ant and Maven builds - but they can be run using special profile/target. I logged this as a JIRA ticket for future reference: https://issues.apache.org/jira/browse/LANG-592 As well as this issue, I also fixed another bug (LANG-76) since RC1 was cut so I'd like to cancel this VOTE and cut another RC. Niall > Phil --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org For additional commands, e-mail: dev-h...@commons.apache.org
