Rick Hillegas <[email protected]> writes: > I think that the laudable goal of error-free tests for 7 consecutive > nights may prove to be overly aggressive for the 10.8.2 release > manager. I couldn't testify that we have ever seen more than a couple > consecutive nights of clean runs.
Error-free tests would be nice, yes. :) Looking at the test results from the last week at http://dbtg.foundry.sun.com/derby/test/, and disregarding the Java 7 noise for a moment, it looks like a normal day has from 1 to 3 failures. Considering that the tests run on 25 different platforms, I think the number is not that bad. What typically fails, is one of the replication tests and AutomaticIndexStatisticsTest on Windows. Disabling AutomaticIndexStatisticsTest on Windows until we've figured out DERBY-5108 sounds like a reasonable way to reduce the noise. Not sure what we should do with the replication tests until we've stabilized them. Disabling them all may be a bit drastic. Although most of us probably have developed a mental filter by now and ignore the failures in the replication tests, they still have value since I'm sure we'll notice once they detect a regression and start failing consistently, as opposed to the intermittent failures we see currently. -- Knut Anders
