On Jan 27, 2012, at 4:50 AM, Simon Willnauer wrote: > mark | yonik, can you open issues for all these failures. > > I see lots of failures for tests like: > > ChaosMonkeySafeLeaderTest > RecoveryZkTest > TermVectorComponentTest > LeaderElectionTest > FullSolrCloudDistribCmdsTest > SpatialFilterTest
It appears to be one issue that is falling into other tests rather than an issue for each of these tests. > > > The last two days are crazy and I don't see any issues ie. reactions > to these failures. Please open issues for these test. I've been in reaction mode pretty much every moment I've been awake - take a look at my commit history. I haven't sent additional email because I assumed a concerned committer might notice my action trail. > I am consider > disabling the tests since the apparently rarely pass. Almost every > build fails see the jenkins stats: > > this is the first build with solr cloud: > https://builds.apache.org/job/Lucene-Solr-tests-only-trunk/12258/ > > since this commit not a single run passed: > > https://builds.apache.org/job/Lucene-Solr-tests-only-trunk/ > > maybe it was too early to merge this in? Should we back it out and > create a branch jenkins build until this stabelizes. I mean trunk you > should be able to go crazy with features but this seems ridiculous. I > really fear that this stuff is moving us 6 month back with respect to > Lucene / Solr 4.0 :( These tests pass over an over on a ton of machines - there is some issue with hudson that we have not yet been able to suss out but that we are working on. Looking at the lucene tests lately I see a lot of red too. Perhaps stuff is too unstable there? They are passing like 50% at best on Lucene-Trunk most of the time it looks. Or perhaps good random tests can pass a lot in some envs but not in others without some investigation and hardening. Having been looking into this issue non stop for the past many many hours, I can say I don't think its a stability issue with the cloud code at all - its very likely a test/env issue. And we are working non stop to figure out what it is. The slow feedback loop with hudson makes it so I can only work so fast - like I said, I don't see this failure on multiple other machines. I only ask for a little patience - every failure is being looked at, and every attempt is being made to get hudson happy. If we don't make progress we will obviously take another route. > > simon > On Fri, Jan 27, 2012 at 10:08 AM, Apache Jenkins Server > <[email protected]> wrote: >> Build: https://builds.apache.org/job/Solr-trunk/1746/ >> >> 3 tests failed. >> FAILED: >> junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeySafeLeaderTest >> >> Error Message: >> ERROR: SolrIndexSearcher opens=166 closes=159 >> >> Stack Trace: >> junit.framework.AssertionFailedError: ERROR: SolrIndexSearcher opens=166 >> closes=159 >> at >> org.apache.solr.SolrTestCaseJ4.endTrackingSearchers(SolrTestCaseJ4.java:151) >> at >> org.apache.solr.SolrTestCaseJ4.afterClassSolrTestCase(SolrTestCaseJ4.java:76) >> >> >> FAILED: junit.framework.TestSuite.org.apache.solr.cloud.RecoveryZkTest >> >> Error Message: >> ERROR: SolrIndexSearcher opens=13 closes=12 >> >> Stack Trace: >> junit.framework.AssertionFailedError: ERROR: SolrIndexSearcher opens=13 >> closes=12 >> at >> org.apache.solr.SolrTestCaseJ4.endTrackingSearchers(SolrTestCaseJ4.java:151) >> at >> org.apache.solr.SolrTestCaseJ4.afterClassSolrTestCase(SolrTestCaseJ4.java:76) >> >> >> FAILED: >> junit.framework.TestSuite.org.apache.solr.handler.component.TermVectorComponentTest >> >> Error Message: >> ERROR: SolrIndexSearcher opens=2 closes=3 >> >> Stack Trace: >> junit.framework.AssertionFailedError: ERROR: SolrIndexSearcher opens=2 >> closes=3 >> at >> org.apache.solr.SolrTestCaseJ4.endTrackingSearchers(SolrTestCaseJ4.java:151) >> at >> org.apache.solr.SolrTestCaseJ4.afterClassSolrTestCase(SolrTestCaseJ4.java:76) >> >> >> >> >> Build Log (for compile errors): >> [...truncated 49188 lines...] >> >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > - Mark Miller lucidimagination.com --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
