On 10-Oct-07, at 3:41 PM, Ryan McKinley wrote:
Would it be appropriate to exclude this test, since a test that
fails
every night is like the boy who cried wolf? At least until it's
worked
out -- I'm not really familiar with the setup on the nightly
build box.
+1! Never-working tests are worse than no tests.
I agree crying wolf is BAD BAD BAD, but I'm not sure removing them
is the best solution.
The tests in question are:
org.apache.solr.update.AutoCommitTest
and all the Jetty based ones
The only place I have seen them fail (incorrectly) is in the
automated test setup. They pass in all my dev environments and I
have assumed they pass in everyone elses...
We have messed with the AutoCommitTest increasing the times
dramatically, but passing still seems dependent on system
resources. Maybe we take that out?
It is failing quite rarely these days--about once every two weeks.
I'm reluctant to give up on this completely since it is a rather
important test.
For the jetty based tests, perhaps we should just give a warning if
it tries to bind a port and can't -- in that case the tests are not
called, but we still have them for the normal dev case where we can
bind the port.
thoughts?
Perhaps we could move some tests to an alternate ant target. 'sanity-
test'? It wouldn't be part of the nightly build.
-Mike
