Looking at the output, it seems this likely failed due to running out of RAM - it says "java.lang.OutOfMemoryError: unable to create new native thread" multiple times.
Seems like a problem with the machines/OS's state at the time - that's gonna be off heap. On Jul 24, 2012, at 8:23 AM, Apache Jenkins Server wrote: > Build: https://builds.apache.org/job/Lucene-Solr-tests-only-4.x/325/ > > 1 tests failed. > REGRESSION: org.apache.solr.cloud.RecoveryZkTest.testDistribSearch > > Error Message: > expected:<1936> but was:<1932> > > Stack Trace: > java.lang.AssertionError: expected:<1936> but was:<1932> > at > __randomizedtesting.SeedInfo.seed([98FA251234CC1277:191CAB0A4393724B]:0) > at org.junit.Assert.fail(Assert.java:93) > at org.junit.Assert.failNotEquals(Assert.java:647) > at org.junit.Assert.assertEquals(Assert.java:128) > at org.junit.Assert.assertEquals(Assert.java:472) > at org.junit.Assert.assertEquals(Assert.java:456) > at org.apache.solr.cloud.RecoveryZkTest.doTest(RecoveryZkTest.java:107) > at > org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:679) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:616) > at > com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1995) > at > com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:132) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:818) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:877) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:891) > at > com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) > at > org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) > at > org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:32) > at > org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) > at > com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) > at > org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68) > at > org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) > at > org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70) > at > org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) > at > com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:825) > at > com.carrotsearch.randomizedtesting.RandomizedRunner.access$700(RandomizedRunner.java:132) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$3$1.run(RandomizedRunner.java:671) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:697) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:736) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:747) > at > com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) > at > org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) > at > org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68) > at > org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38) > at > org.apache.lucene.util.TestRuleIcuHack$1.evaluate(TestRuleIcuHack.java:51) > at > com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) > at > org.apache.lucene.util.TestRuleNoInstanceHooksOverrides$1.evaluate(TestRuleNoInstanceHooksOverrides.java:53) > at > org.apache.lucene.util.TestRuleNoStaticHooksShadowing$1.evaluate(TestRuleNoStaticHooksShadowing.java:52) > at > org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:36) > at > org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) > at > org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70) > at > org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) > at > com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:605) > at > com.carrotsearch.randomizedtesting.RandomizedRunner.access$400(RandomizedRunner.java:132) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:551) > > > > > Build Log: > [...truncated 30385 lines...] > [junit4:junit4] 2> 34567 T3510 C73 P13581 REQ /update > {update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 > [junit4:junit4] 2> 34567 T1644 C72 P13579 /update {wt=javabin&version=2} > {add=[11921 (1408376209096572928)]} 0 2 > [junit4:junit4] 2> 34569 T3506 C73 P13581 REQ /update > {update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 > [junit4:junit4] 2> 34569 T1630 C71 P13577 /update {wt=javabin&version=2} > {add=[11922 (1408376209100767232)]} 0 0 > [junit4:junit4] 2> 34569 T1645 C72 P13579 /update {wt=javabin&version=2} > {delete=[1868 (-1408376209098670080)]} 0 2 > [junit4:junit4] 2> 34573 T1630 C71 P13577 /update {wt=javabin&version=2} > {add=[1918 (1408376209102864384)]} 0 2 > [junit4:junit4] 2> 34573 T3506 C73 P13581 REQ /update > {update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 > [junit4:junit4] 2> 34573 T1644 C72 P13579 /update {wt=javabin&version=2} > {add=[11922 (1408376209102864384)]} 0 2 > [junit4:junit4] 2> 34576 T1630 C71 P13577 /update {wt=javabin&version=2} > {delete=[11878 (-1408376209108107264)]} 0 0 > [junit4:junit4] 2> 34577 T3506 C73 P13581 REQ /update > {update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 > [junit4:junit4] 2> 34577 T1645 C72 P13579 /update {wt=javabin&version=2} > {add=[1918 (1408376209107058688)]} 0 2 > [junit4:junit4] 2> 34579 T1630 C71 P13577 /update {wt=javabin&version=2} > {delete=[1871 (-1408376209111252992)]} 0 0 > [junit4:junit4] 2> 34580 T3506 C73 P13581 REQ /update > {update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 > [junit4:junit4] 2> 34580 T1644 C72 P13579 /update {wt=javabin&version=2} > {delete=[11878 (-1408376209110204416)]} 0 2 > [junit4:junit4] 2> 34582 T1630 C71 P13577 /update {wt=javabin&version=2} > {add=[11923 (1408376209114398720)]} 0 0 > [junit4:junit4] 2> 34583 T3506 C73 P13581 REQ /update > {update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 > [junit4:junit4] 2> 34583 T1645 C72 P13579 /update {wt=javabin&version=2} > {delete=[1871 (-1408376209113350144)]} 0 2 > [junit4:junit4] 2> 34585 T1630 C71 P13577 /update {wt=javabin&version=2} > {add=[1919 (1408376209117544448)]} 0 0 > [junit4:junit4] 2> 34586 T3506 C73 P13581 REQ /update > {update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 > [junit4:junit4] 2> 34586 T1644 C72 P13579 /update {wt=javabin&version=2} > {add=[11923 (1408376209116495872)]} 0 2 > [junit4:junit4] 2> 34588 T1630 C71 P13577 /update {wt=javabin&version=2} > {delete=[11880 (-1408376209120690176)]} 0 0 > [junit4:junit4] 2> 34591 T3506 C73 P13581 REQ /update > {update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 > [junit4:junit4] 2> 34592 T1645 C72 P13579 /update {wt=javabin&version=2} > {add=[1919 (1408376209119641600)]} 0 5 > [junit4:junit4] 2> 34592 T3510 C73 P13581 REQ /update > {update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 > [junit4:junit4] 2> 34592 T1644 C72 P13579 /update {wt=javabin&version=2} > {delete=[11880 (-1408376209122787328)]} 0 2 > [junit4:junit4] 2> 34594 T1630 C71 P13577 /update {wt=javabin&version=2} > {delete=[1873 (-1408376209126981632)]} 0 0 > [junit4:junit4] 2> 34594 T1629 C71 P13577 /update {wt=javabin&version=2} > {add=[11924 (1408376209126981633)]} 0 0 > [junit4:junit4] 2> 34598 T3510 C73 P13581 REQ /update > {update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 > [junit4:junit4] 2> 34598 T1644 C72 P13579 /update {wt=javabin&version=2} > {delete=[1873 (-1408376209129078785)]} 0 2 > [junit4:junit4] 2> 34598 T1617 oascc.ZkStateReader.updateCloudState > Updating cloud state from ZooKeeper... > [junit4:junit4] 2> 34598 T3506 C73 P13581 REQ /update > {update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 > [junit4:junit4] 2> 34598 T1645 C72 P13579 /update {wt=javabin&version=2} > {add=[11924 (1408376209129078784)]} 0 2 > [junit4:junit4] 2> added docs:1924 with 9 fails deletes:955 > [junit4:junit4] 2> 34600 T1629 C71 P13577 /update {wt=javabin&version=2} > {add=[1920 (1408376209133273088)]} 0 0 > [junit4:junit4] 2> 34604 T3506 C73 P13581 REQ /update > {update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 > [junit4:junit4] 2> 34604 T1645 C72 P13579 /update {wt=javabin&version=2} > {add=[1920 (1408376209135370240)]} 0 2 > [junit4:junit4] 2> added docs:1920 with 7 fails deletes:953 > [junit4:junit4] 2> 34606 T1629 C71 P13577 oasu.DirectUpdateHandler2.commit > start > commit{flags=0,version=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false} > [junit4:junit4] 2> 34812 T670 oaz.ClientCnxn$SendThread.run Client session > timed out, have not heard from server in 12001ms for sessionid > 0x138b8e637650010, closing socket connection and attempting reconnect > [junit4:junit4] 2> 35259 T1629 C71 P13577 oasc.SolrDeletionPolicy.onCommit > SolrDeletionPolicy.onCommit: commits:num=2 > [junit4:junit4] 2> > commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x/checkout/solr/build/solr-core/test/J1/org.apache.solr.cloud.RecoveryZkTest-1343132183890/control/data/index > lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5bd904; > maxCacheMB=48.0 > maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1] > [junit4:junit4] 2> > commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x/checkout/solr/build/solr-core/test/J1/org.apache.solr.cloud.RecoveryZkTest-1343132183890/control/data/index > lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5bd904; > maxCacheMB=48.0 > maxMergeSizeMB=4.0),segFN=segments_2,generation=2,filenames=[_68.si, > _c4_MockRandom_0.tim, _c2_MockVariableIntBlock_0.pos, _c1_MockSep_0.tib, > _c1_MockSep_0.pyl, _c2_MockRandom_0.tii, _c1_MockSep_0.tii, > _c4_MockRandom_0.tip, _c6_MockRandom_0.pyl, _c5_MockRandom_0.sd, > _c2_MockRandom_0.doc, _c2_MockRandom_0.tib, _68_MockRandom_0.sd, > _c1_MockRandom_0.tim, _c4_nrm.cfs, _c1_MockRandom_0.tip, > _c6_MockRandom_0.skp, _c4_MockSep_0.frq, _c4.fdt, _c4_nrm.cfe, > _c2_Memory_0.ram, _c2_MockSep_0.doc, _c4.fdx, _c5_MockRandom_0.frq, > _c3_MockSep_0.frq, _c3_MockSep_0.pos, _68.fnm, > _c6_MockVariableIntBlock_0.tib, _c2.fdt, _68_MockSep_0.pos, > _c1_MockRandom_0.pyl, _c1.fdt, _c5_MockRandom_0.tib, _c1_MockRandom_0.doc, > _c1_MockRandom_0.sd, _c2.fdx, _c6_MockVariableIntBlock_0.tii, _c1.fdx, > _c5_MockRandom_0.tiv, _c2_MockVariableIntBlock_0.frq, _c6_MockSep_0.pos, > _c5_MockVariableIntBlock_0.pos, _c4_MockVariableIntBlock_0.frq, _c6.fdx, > _c2_MockRandom_0.frq, _c1.fnm, _c6.si, _c6.fdt, _c6_MockRandom_0.sd, > _c4_MockRandom_0.pyl, _c5_MockVariableIntBlock_0.skp, > _c5_MockVariableIntBlock_0.doc, _68_Memory_0.ram, _c3.fdt, > _c4_MockRandom_0.frq, _c4_MockSep_0.doc, _c3.fdx, > _c6_MockVariableIntBlock_0.frq, _c6_MockVariableIntBlock_0.doc, _c3.fnm, > _68_nrm.cfs, _c1_Memory_0.ram, _c5_Memory_0.ram, > _c2_MockVariableIntBlock_0.pyl, _68_MockRandom_0.skp, _c6_MockRandom_0.pos, > _c4_MockRandom_0.sd, _68_MockSep_0.doc, _68_MockVariableIntBlock_0.pyl, > _c6_MockVariableIntBlock_0.pyl, _c4_MockVariableIntBlock_0.pos, > _c3_MockVariableIntBlock_0.pos, _68_MockSep_0.skp, > _c2_MockVariableIntBlock_0.skp, _c3_MockRandom_0.sd, _c4_MockSep_0.pyl, > _c5_MockSep_0.doc, _c3_MockSep_0.doc, _c2_MockRandom_0.skp, _c3.si, > _c2_MockSep_0.pos, _c3_MockRandom_0.tii, _c1_MockSep_0.skp, > _c1_MockRandom_0.frq, _c3_MockRandom_0.tib, _c5_MockVariableIntBlock_0.frq, > _c6_MockSep_0.tii, _c1_MockVariableIntBlock_0.pos, > _c4_MockVariableIntBlock_0.skp, _68_MockRandom_0.pos, _c6_MockSep_0.tib, > _c2_MockVariableIntBlock_0.tib, _c1_MockSep_0.pos, _c4_MockSep_0.tii, > _c2_MockRandom_0.pyl, _68_MockVariableIntBlock_0.tii, > _c2_MockVariableIntBlock_0.tii, _c4_MockSep_0.tib, > _68_MockVariableIntBlock_0.tib, _68_nrm.cfe, _68_MockSep_0.tib, > _68_MockSep_0.frq, _c5.fnm, _68_MockSep_0.tii, _c6_MockSep_0.frq, > _c2_MockSep_0.tib, _c5_MockVariableIntBlock_0.tib, > _c4_MockVariableIntBlock_0.doc, _c4_MockRandom_0.doc, _c2_MockSep_0.tii, > _c5_MockVariableIntBlock_0.tii, _c4_Memory_0.ram, > _c3_MockVariableIntBlock_0.skp, _c3_MockVariableIntBlock_0.tii, > _c5_MockSep_0.frq, _c1_MockRandom_0.pos, _68_MockRandom_0.frq, > _c6_Memory_0.ram, _c1_nrm.cfs, _c3_MockVariableIntBlock_0.tib, > _c1_MockVariableIntBlock_0.frq, _c2_MockRandom_0.sd, _c5_MockRandom_0.prx, > _c3_MockRandom_0.prx, _c6_MockRandom_0.doc, _c6_MockSep_0.pyl, _c1_nrm.cfe, > _68_MockSep_0.pyl, _c6_nrm.cfs, _c4_MockSep_0.skp, _c2_MockRandom_0.pos, > _68_MockVariableIntBlock_0.skp, _c6_nrm.cfe, _68.fdt, _c1_MockRandom_0.skp, > _68.fdx, _68_MockVariableIntBlock_0.doc, _c6_MockVariableIntBlock_0.pos, > _c1_MockSep_0.doc, _c5_MockSep_0.pos, _c4_MockRandom_0.skp, > _68_MockVariableIntBlock_0.pos, _c5_MockVariableIntBlock_0.pyl, > _c5_MockSep_0.skp, _c3_MockVariableIntBlock_0.doc, _c5_MockSep_0.pyl, > _68_MockRandom_0.doc, _c3_MockSep_0.tib, _c2_MockSep_0.pyl, _68_1.del, > _c3_MockSep_0.tii, _c2.fnm, _c1_MockVariableIntBlock_0.doc, > _c3_MockSep_0.pyl, _c2_MockSep_0.frq, _c1_MockVariableIntBlock_0.skp, > _c4_MockVariableIntBlock_0.tii, _c1_1.del, _c6.fnm, _c5_nrm.cfe, _c2.si, > _c4_MockVariableIntBlock_0.tib, _c1.si, _c6_MockSep_0.skp, _c6_MockSep_0.doc, > _c5_nrm.cfs, _c3_MockRandom_0.frq, _c6_MockRandom_0.frq, > _c1_MockVariableIntBlock_0.pyl, _c5_MockSep_0.tii, > _c4_MockVariableIntBlock_0.pyl, _c5_MockSep_0.tib, > _c2_MockVariableIntBlock_0.doc, _c4_MockRandom_0.pos, _c6_MockRandom_0.tib, > _68_MockVariableIntBlock_0.frq, _c5.si, _c4.fnm, _c6_MockRandom_0.tii, > _c1_MockSep_0.frq, _c3_Memory_0.ram, _c2_MockSep_0.skp, > _c3_MockVariableIntBlock_0.frq, _c4.si, _68_MockRandom_0.tim, _c2_nrm.cfs, > _c1_MockVariableIntBlock_0.tib, _c3_MockVariableIntBlock_0.pyl, _c3_nrm.cfs, > _68_MockRandom_0.tip, _c1_MockVariableIntBlock_0.tii, _c3_MockSep_0.skp, > _68_MockRandom_0.pyl, _c2_nrm.cfe, _c6_MockVariableIntBlock_0.skp, _c5.fdt, > _c5.fdx, segments_2, _c4_MockSep_0.pos, _c3_nrm.cfe] > [junit4:junit4] 2> 35260 T1629 C71 P13577 > oasc.SolrDeletionPolicy.updateCommits newest commit = 2 > [junit4:junit4] 2> 35283 T1629 C71 P13577 oass.SolrIndexSearcher.<init> > Opening Searcher@808d6cc main > [junit4:junit4] 2> 35283 T1629 C71 P13577 oasu.DirectUpdateHandler2.commit > end_commit_flush > [junit4:junit4] 2> 35284 T1640 oasc.SolrCore.registerSearcher [collection1] > Registered new searcher Searcher@808d6cc > main{StandardDirectoryReader(segments_2:933 _68(4.0.0.1):C1020/73 > _c1(4.0.0.1):C967/22 _c2(4.0.0.1):C10 _c3(4.0.0.1):C10 _c4(4.0.0.1):C10 > _c5(4.0.0.1):C10 _c6(4.0.0.1):C4)} > [junit4:junit4] 2> 35287 T1629 C71 P13577 /update > {waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} > {commit=} 0 681 > [junit4:junit4] 2> 35289 T1645 C72 P13579 oasu.DirectUpdateHandler2.commit > start > commit{flags=0,version=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false} > [junit4:junit4] 2> 35931 T1645 C72 P13579 oasc.SolrDeletionPolicy.onCommit > SolrDeletionPolicy.onCommit: commits:num=2 > [junit4:junit4] 2> > commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x/checkout/solr/build/solr-core/test/J1/org.apache.solr.cloud.RecoveryZkTest-1343132183890/jetty1/index > lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@51e68873; > maxCacheMB=48.0 > maxMergeSizeMB=4.0),segFN=segments_3,generation=3,filenames=[_a5.fdx, _68.si, > _a5.fdt, _a1_MockRandom_0.sd, _a2.fdt, _a3_MockRandom_0.prx, _a2.fdx, > _a4_MockSep_0.skp, _68_MockRandom_0.sd, _a1_MockVariableIntBlock_0.doc, > _a6_MockVariableIntBlock_0.skp, _a6_MockRandom_0.doc, _a1_Memory_0.ram, > _a6_MockVariableIntBlock_0.pyl, _a3_MockSep_0.tib, > _a3_MockVariableIntBlock_0.tii, _a1_MockVariableIntBlock_0.skp, > _a2_MockVariableIntBlock_0.pos, _a3_MockVariableIntBlock_0.tib, > _a3_MockSep_0.tii, _a3_MockSep_0.pyl, _a2_MockRandom_0.frq, > _a2_MockSep_0.frq, _a3.fdx, _a3_MockSep_0.doc, > _a2_MockVariableIntBlock_0.frq, _a3.fdt, _a3_MockRandom_0.frq, _68.fnm, > _68_MockSep_0.pos, _a2_Memory_0.ram, _a2_MockSep_0.pos, _a5_MockSep_0.tib, > _a4_Memory_0.ram, _a5_MockSep_0.tii, _a5_MockVariableIntBlock_0.pos, _a1.fdt, > _a1_nrm.cfs, _a1.fdx, _a2_MockRandom_0.pos, _a1_MockRandom_0.pos, > _a5_Memory_0.ram, _a1_nrm.cfe, _68_Memory_0.ram, _a5_MockSep_0.pyl, > _a5_nrm.cfe, _a1_MockSep_0.pyl, _a5_nrm.cfs, _a5_MockSep_0.frq, _a6.fnm, > _68_MockRandom_0.prx, _68_nrm.cfs, _a6_MockRandom_0.pos, > _a4_MockRandom_0.frq, _a5_MockRandom_0.frq, _a3_MockVariableIntBlock_0.frq, > _a6_MockSep_0.tib, _68_MockSep_0.doc, _68_MockVariableIntBlock_0.pyl, > _a2_MockSep_0.skp, _a4_MockSep_0.tib, _a4_MockVariableIntBlock_0.skp, > _a4_MockSep_0.tii, _68_MockSep_0.skp, _a6_MockSep_0.frq, > _a2_MockRandom_0.doc, _a6.si, _a1_MockSep_0.tii, _a1_MockSep_0.tib, > _a2_MockVariableIntBlock_0.pyl, _a3_Memory_0.ram, > _a5_MockVariableIntBlock_0.frq, _a2_nrm.cfs, _a3_MockVariableIntBlock_0.pyl, > _a4_MockRandom_0.doc, _a1_MockVariableIntBlock_0.pos, > _a2_MockVariableIntBlock_0.doc, _a1_MockRandom_0.tim, _a4.fdx, > _a1_MockRandom_0.tip, _a2_nrm.cfe, _a4_MockRandom_0.sd, _a4.fdt, > _a5_MockRandom_0.pos, _a5_MockRandom_0.sd, _a6_MockSep_0.tii, > _a4_MockVariableIntBlock_0.pyl, _a2.si, _a3_nrm.cfe, _a4_nrm.cfs, > _a2_MockVariableIntBlock_0.tii, _a6_MockSep_0.pyl, _a2_MockRandom_0.sd, > _a3.si, _68_MockVariableIntBlock_0.tii, _a3_MockRandom_0.sd, > _a2_MockVariableIntBlock_0.tib, _a3_nrm.cfs, _68_MockVariableIntBlock_0.tib, > _a4_nrm.cfe, _68_nrm.cfe, _a1.fnm, _68_MockSep_0.tib, _a5.si, > _a1_MockSep_0.skp, _68_MockSep_0.frq, _a3_MockSep_0.skp, > _a1_MockRandom_0.pyl, _a1_MockSep_0.doc, _68_MockSep_0.tii, > _a4_MockVariableIntBlock_0.tib, _a4_MockVariableIntBlock_0.tii, > _68_MockRandom_0.frq, _a3_MockRandom_0.tip, _a3_MockRandom_0.tim, > _a2_MockRandom_0.tii, _a5.fnm, _a1_MockSep_0.pos, _a2_MockRandom_0.tib, > _a4_MockVariableIntBlock_0.doc, _68_MockSep_0.pyl, _a2_MockRandom_0.pyl, > _a4_MockRandom_0.tim, _a1_MockVariableIntBlock_0.tib, > _a5_MockVariableIntBlock_0.doc, _a6_MockRandom_0.frq, > _a1_MockVariableIntBlock_0.tii, _68_MockVariableIntBlock_0.skp, > _a6_MockRandom_0.sd, _a4_MockRandom_0.tip, _68.fdt, > _a3_MockVariableIntBlock_0.doc, _68.fdx, _68_MockVariableIntBlock_0.doc, > _a4_MockVariableIntBlock_0.frq, _a1.si, _a6_nrm.cfs, _a6_MockSep_0.skp, > _a5_MockSep_0.pos, _a5_MockVariableIntBlock_0.pyl, > _a3_MockVariableIntBlock_0.skp, _a1_MockRandom_0.skp, _a2.fnm, > _a2_MockRandom_0.skp, _a3_MockSep_0.pos, _a6_MockSep_0.doc, _a3.fnm, > _a6_MockVariableIntBlock_0.doc, _a4_MockSep_0.pyl, > _a5_MockVariableIntBlock_0.skp, _a5_MockRandom_0.doc, > _68_MockVariableIntBlock_0.pos, _a1_MockRandom_0.doc, _a2_1.del, > _a4_MockSep_0.frq, _a1_MockVariableIntBlock_0.pyl, _a4_MockRandom_0.skp, > _a2_MockVariableIntBlock_0.skp, _68_1.del, _a4_MockSep_0.pos, > _a1_MockSep_0.frq, _a2_MockSep_0.tib, _a6_Memory_0.ram, _a2_MockSep_0.pyl, > _a6.fdx, _a2_MockSep_0.tii, _a4_MockRandom_0.pyl, _a5_MockRandom_0.tip, > _a6_MockRandom_0.tim, _a4.fnm, _a6_MockVariableIntBlock_0.frq, > _a5_MockRandom_0.tim, _a6_MockRandom_0.tip, _68_MockRandom_0.tib, > _a1_MockVariableIntBlock_0.frq, _a6_nrm.cfe, _a4_MockVariableIntBlock_0.pos, > _a6.fdt, _a2_MockSep_0.doc, _a4_MockRandom_0.pos, > _a5_MockVariableIntBlock_0.tib, _a5_MockVariableIntBlock_0.tii, > _a4_MockSep_0.doc, _a6_MockVariableIntBlock_0.tib, _a5_MockSep_0.skp, > _a6_MockRandom_0.skp, _a5_MockRandom_0.skp, _a6_MockVariableIntBlock_0.pos, > _68_MockVariableIntBlock_0.frq, _a5_MockRandom_0.pyl, _a3_MockSep_0.frq, > _a6_MockRandom_0.pyl, _a3_MockVariableIntBlock_0.pos, _68_MockRandom_0.tiv, > _a6_MockSep_0.pos, _a1_MockRandom_0.frq, _a4.si, segments_3, > _a5_MockSep_0.doc, _a6_MockVariableIntBlock_0.tii] > [junit4:junit4] 2> > commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x/checkout/solr/build/solr-core/test/J1/org.apache.solr.cloud.RecoveryZkTest-1343132183890/jetty1/index > lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@51e68873; > maxCacheMB=48.0 > maxMergeSizeMB=4.0),segFN=segments_4,generation=4,filenames=[_c1_MockSep_0.tib, > _c2_MockVariableIntBlock_0.pos, _a9_MockVariableIntBlock_0.frq, _c5.fnm, > _c1_MockSep_0.pyl, _c1_MockSep_0.tii, _a9_MockRandom_0.tiv, > _c2_MockRandom_0.tii, _c6_MockSep_0.frq, _c5_MockRandom_0.sd, _a9.fnm, > _a9_MockRandom_0.tib, _c5_MockVariableIntBlock_0.tib, _c2_MockSep_0.tib, > _c4_MockVariableIntBlock_0.doc, _c2_MockRandom_0.tib, _c2_MockSep_0.tii, > _c4_MockRandom_0.tib, _c5_MockVariableIntBlock_0.tii, _c4_Memory_0.ram, > _a9_MockSep_0.tii, _c5_MockSep_0.frq, _c3_MockVariableIntBlock_0.skp, > _c3_MockVariableIntBlock_0.tii, _c4_nrm.cfs, _a9_MockVariableIntBlock_0.tib, > _c4_MockSep_0.frq, _c1_MockRandom_0.tii, _c6_Memory_0.ram, > _c3_MockVariableIntBlock_0.tib, _c1_nrm.cfs, _c1_MockVariableIntBlock_0.frq, > _c4.fdt, _c4_nrm.cfe, _c2_MockRandom_0.sd, _c5_MockRandom_0.prx, > _c2_MockSep_0.doc, _c2_Memory_0.ram, _c4.fdx, _c3_MockRandom_0.prx, > _c1_MockRandom_0.tib, _c5_MockRandom_0.frq, _c6_MockSep_0.pyl, _c1_nrm.cfe, > _c3_MockSep_0.frq, _c6_nrm.cfs, _c3_MockSep_0.pos, > _c6_MockVariableIntBlock_0.tib, _c4_MockSep_0.skp, _c2.fdt, > _c5_MockRandom_0.tib, _c1.fdt, _c1_MockRandom_0.sd, _a9_nrm.cfs, > _c6_MockVariableIntBlock_0.tii, _c2.fdx, _c1.fdx, _a9.si, > _c1_MockRandom_0.prx, _a9.fdx, _c6_nrm.cfe, _c2_MockVariableIntBlock_0.frq, > _a9_MockSep_0.doc, _c6_MockVariableIntBlock_0.pos, _c6_MockSep_0.pos, > _c5_MockVariableIntBlock_0.pos, _c4_MockVariableIntBlock_0.frq, > _a9_MockVariableIntBlock_0.tii, _c5_MockRandom_0.tii, _c1_MockSep_0.doc, > _c5_MockSep_0.pos, _c6.fdx, _c4_MockRandom_0.prx, _c6.si, _c1.fnm, > _c2_MockRandom_0.frq, _c6.fdt, _c6_MockRandom_0.sd, > _c5_MockVariableIntBlock_0.skp, _c5_MockVariableIntBlock_0.doc, _a9.fdt, > _c4_MockRandom_0.tiv, _c4_MockRandom_0.frq, _c3.fdt, _c4_MockSep_0.doc, > _c5_MockVariableIntBlock_0.pyl, _a9_MockSep_0.skp, _c5_MockSep_0.skp, > _c3_MockVariableIntBlock_0.doc, _a9_MockRandom_0.prx, _c5_MockSep_0.pyl, > _c3.fdx, _c6_MockVariableIntBlock_0.frq, _a9_MockVariableIntBlock_0.doc, > _c2_MockSep_0.pyl, _c3_MockSep_0.tib, _c3_MockSep_0.tii, > _c6_MockVariableIntBlock_0.doc, _c2.fnm, _c5_Memory_0.ram, _c1_Memory_0.ram, > _a9_MockRandom_0.sd, _c3.fnm, _a9_MockVariableIntBlock_0.skp, > _c1_MockVariableIntBlock_0.doc, _c3_MockSep_0.pyl, _c2_MockSep_0.frq, > _c1_MockVariableIntBlock_0.skp, _c2_MockVariableIntBlock_0.pyl, > _c4_MockRandom_0.sd, _a9_MockSep_0.pos, _c6_MockVariableIntBlock_0.pyl, > _c4_MockVariableIntBlock_0.pos, _c4_MockVariableIntBlock_0.tii, _c1_1.del, > _c6.fnm, _c5_nrm.cfe, _c2.si, _c3_MockVariableIntBlock_0.pos, > _c4_MockVariableIntBlock_0.tib, _c1.si, _c6_MockSep_0.skp, _c6_MockSep_0.doc, > _c2_MockVariableIntBlock_0.skp, _c5_nrm.cfs, _c3_MockRandom_0.sd, > _c3_MockRandom_0.frq, _c6_MockRandom_0.frq, _c5_MockSep_0.doc, > _c4_MockSep_0.pyl, _c3_MockSep_0.doc, _a9_MockSep_0.pyl, _a9_nrm.cfe, > _a9_MockVariableIntBlock_0.pyl, _c2_MockSep_0.pos, _c3.si, > _c1_MockVariableIntBlock_0.pyl, _c5_MockSep_0.tii, _a9_MockRandom_0.frq, > _c4_MockVariableIntBlock_0.pyl, _c1_MockSep_0.skp, _c3_MockRandom_0.tii, > _c1_MockRandom_0.frq, _c5_MockSep_0.tib, _c2_MockVariableIntBlock_0.doc, > _c6_MockRandom_0.tib, _c3_MockRandom_0.tib, _c5.si, _c4.fnm, > _c5_MockVariableIntBlock_0.frq, _c6_MockRandom_0.tiv, _c1_MockSep_0.frq, > _c3_Memory_0.ram, _c2_MockSep_0.skp, _c3_MockVariableIntBlock_0.frq, _c4.si, > _a9_MockVariableIntBlock_0.pos, _c6_MockSep_0.tii, > _c1_MockVariableIntBlock_0.pos, _a9_1.del, _c4_MockVariableIntBlock_0.skp, > _c2_MockRandom_0.prx, _c2_nrm.cfs, _c1_MockVariableIntBlock_0.tib, > _c3_MockVariableIntBlock_0.pyl, _c3_nrm.cfs, _c6_MockSep_0.tib, > _a9_MockSep_0.tib, _c1_MockVariableIntBlock_0.tii, _c3_MockSep_0.skp, > _c2_MockVariableIntBlock_0.tib, _c6_MockRandom_0.prx, _c2_nrm.cfe, > _c6_MockVariableIntBlock_0.skp, _c5.fdt, _c1_MockSep_0.pos, > _c4_MockSep_0.tii, _c5.fdx, _a9_MockSep_0.frq, > _c2_MockVariableIntBlock_0.tii, _c4_MockSep_0.pos, _c4_MockSep_0.tib, > _a9_Memory_0.ram, _c3_nrm.cfe, segments_4] > [junit4:junit4] 2> 35936 T1645 C72 P13579 > oasc.SolrDeletionPolicy.updateCommits newest commit = 4 > [junit4:junit4] 2> 35965 T1645 C72 P13579 oass.SolrIndexSearcher.<init> > Opening Searcher@5e19c8ca main > [junit4:junit4] 2> 35966 T1645 C72 P13579 oasu.DirectUpdateHandler2.commit > end_commit_flush > [junit4:junit4] 2> 35966 T1652 oasc.SolrCore.registerSearcher [collection1] > Registered new searcher Searcher@5e19c8ca > main{StandardDirectoryReader(segments_4:935 _a9(4.0.0.1):C1629/37 > _c1(4.0.0.1):C321/26 _c2(4.0.0.1):C10 _c3(4.0.0.1):C10 _c4(4.0.0.1):C10 > _c5(4.0.0.1):C10 _c6(4.0.0.1):C9)} > [junit4:junit4] 2> 35975 T3506 C73 P13581 oasu.DirectUpdateHandler2.commit > start > commit{flags=0,version=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false} > [junit4:junit4] 2> 36118 T670 oaz.ClientCnxn$SendThread.startConnect > Opening socket connection to server localhost/127.0.0.1:65199 > [junit4:junit4] 2> 36364 T3506 C73 P13581 oasc.SolrDeletionPolicy.onCommit > SolrDeletionPolicy.onCommit: commits:num=2 > [junit4:junit4] 2> > commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x/checkout/solr/build/solr-core/test/J1/org.apache.solr.cloud.RecoveryZkTest-1343132183890/jetty2/index.20120724061654867 > lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@147eea71; > maxCacheMB=48.0 > maxMergeSizeMB=4.0),segFN=segments_4,generation=4,filenames=[_bn_MockVariableIntBlock_0.tii, > _bn_MockRandom_0.prx, _bq_MockSep_0.frq, _bp_MockRandom_0.doc, > _bm_MockSep_0.pos, _bm_MockSep_0.tii, _bn_MockVariableIntBlock_0.tib, > _bq_MockVariableIntBlock_0.skp, _a9_MockVariableIntBlock_0.frq, _bn.fnm, > _bo_Memory_0.ram, _a9_MockRandom_0.tib, _a9.fnm, _bm.fdt, _bm.fdx, > _bn_MockSep_0.tib, _a9_MockRandom_0.tii, _a9_MockRandom_0.pos, > _bm_MockRandom_0.tiv, _bm_MockSep_0.tib, _a9_MockSep_0.tii, > _bo_MockSep_0.pos, _bn_MockSep_0.tii, _bp_MockSep_0.frq, > _a9_MockVariableIntBlock_0.tib, _a9_MockRandom_0.skp, _bm_MockRandom_0.pyl, > _bm_MockVariableIntBlock_0.frq, _bn.fdt, _bm_MockRandom_0.tib, _bn.fdx, > _bn.si, _bq_MockVariableIntBlock_0.pyl, _bo_MockVariableIntBlock_0.tii, > _bm_MockSep_0.pyl, _bq.fnm, _bo_MockVariableIntBlock_0.tib, > _bq_MockVariableIntBlock_0.doc, _bm_Memory_0.ram, _bn_MockSep_0.frq, > _a9_MockRandom_0.pyl, _bp_MockVariableIntBlock_0.frq, > _bp_MockVariableIntBlock_0.pyl, _bm_MockRandom_0.skp, _a9_nrm.cfs, > _bm_MockVariableIntBlock_0.pyl, _a9.si, _bp_MockRandom_0.pyl, _a9.fdx, > _bp.si, _bo_MockRandom_0.frq, _bm_MockRandom_0.doc, > _bp_MockVariableIntBlock_0.skp, _bn_MockRandom_0.tiv, _bq_MockSep_0.tib, > _a9_MockSep_0.doc, _bo.si, _a9_MockVariableIntBlock_0.tii, > _bn_MockRandom_0.sd, _bq_MockSep_0.tii, _bm_MockRandom_0.pos, > _bq_Memory_0.ram, _bo_MockSep_0.frq, _bm_MockSep_0.frq, _bq_MockSep_0.pos, > _a9.fdt, _bp_MockSep_0.tii, _bm_MockVariableIntBlock_0.pos, > _bp_MockRandom_0.skp, _a9_MockRandom_0.doc, _bn_MockVariableIntBlock_0.doc, > _a9_MockSep_0.skp, _bp_MockSep_0.tib, _bm_1.del, _bo_MockSep_0.doc, > _bo_MockVariableIntBlock_0.frq, _bp_MockRandom_0.sd, _bq_MockRandom_0.sd, > _bq_MockSep_0.doc, _a9_MockVariableIntBlock_0.doc, _bp_MockSep_0.doc, > _bm_nrm.cfe, _bq_MockSep_0.skp, _a9_MockRandom_0.sd, > _a9_MockVariableIntBlock_0.skp, _bn_MockRandom_0.frq, > _bm_MockVariableIntBlock_0.doc, _bp_MockVariableIntBlock_0.tib, > _bm_MockRandom_0.sd, _bm_nrm.cfs, _bo_MockRandom_0.sd, _a9_MockSep_0.pos, > _bp_MockVariableIntBlock_0.tii, _bo_MockSep_0.pyl, _bq_MockSep_0.pyl, _bm.si, > _bn_MockVariableIntBlock_0.skp, _bq_MockVariableIntBlock_0.frq, _bo_nrm.cfe, > _bn_MockSep_0.pos, _bo_MockVariableIntBlock_0.pos, _bn_MockSep_0.skp, > _bp_MockVariableIntBlock_0.pos, _bo_MockVariableIntBlock_0.pyl, > _bp_MockSep_0.skp, _bo_nrm.cfs, _bp_MockSep_0.pyl, _bo_MockSep_0.skp, > _bp_MockRandom_0.pos, _bp_MockRandom_0.tim, _bn_MockVariableIntBlock_0.pos, > _bn_MockRandom_0.tib, _bm_MockVariableIntBlock_0.skp, _bn_MockSep_0.doc, > _a9_MockSep_0.pyl, _bp_MockRandom_0.tip, _a9_nrm.cfe, _bm_MockSep_0.doc, > _a9_MockVariableIntBlock_0.pyl, _bp_MockVariableIntBlock_0.doc, > _bq_MockRandom_0.frq, _bn_MockSep_0.pyl, _a9_MockRandom_0.frq, _bq_nrm.cfe, > _bq.si, _bn_nrm.cfs, _bq_MockRandom_0.tim, _bq_MockRandom_0.tip, > _bp_Memory_0.ram, _bq_nrm.cfs, _bq_MockVariableIntBlock_0.tib, _bp.fnm, > _bn_nrm.cfe, _bm_MockRandom_0.frq, _bo_MockRandom_0.tip, > _bq_MockVariableIntBlock_0.tii, _bo_MockVariableIntBlock_0.doc, _bm.fnm, > _bo_MockRandom_0.tim, _bn_Memory_0.ram, _a9_MockVariableIntBlock_0.pos, > _bq_MockRandom_0.prx, _bo_MockSep_0.tii, _a9_1.del, > _bm_MockVariableIntBlock_0.tib, _bo.fdx, _bo.fdt, _bo_MockSep_0.tib, > _bq_MockVariableIntBlock_0.pos, _a9_MockSep_0.tib, _bp_MockRandom_0.frq, > _bo_MockVariableIntBlock_0.skp, _bp_nrm.cfs, _bp.fdt, _bp_MockSep_0.pos, > _bq.fdx, _a9_MockSep_0.frq, _bq.fdt, _bo_MockRandom_0.prx, > _bn_MockVariableIntBlock_0.pyl, _bn_MockVariableIntBlock_0.frq, > _bm_MockVariableIntBlock_0.tii, _bp_nrm.cfe, _bo.fnm, _bp.fdx, > _a9_Memory_0.ram, _bm_MockSep_0.skp, segments_4] > [junit4:junit4] 2> > commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x/checkout/solr/build/solr-core/test/J1/org.apache.solr.cloud.RecoveryZkTest-1343132183890/jetty2/index.20120724061654867 > lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@147eea71; > maxCacheMB=48.0 > maxMergeSizeMB=4.0),segFN=segments_5,generation=5,filenames=[_c2_MockVariableIntBlock_0.pos, > _c2_MockRandom_0.tii, _c6_MockRandom_0.pyl, _a9.fnm, _a9_MockRandom_0.tib, > _c5_MockRandom_0.sd, _c7_MockSep_0.skp, _c2_MockRandom_0.doc, > _c7_MockVariableIntBlock_0.pos, _c2_MockRandom_0.tib, _c4_MockRandom_0.tib, > _a9_MockRandom_0.tii, _a9_MockSep_0.tii, _c4_nrm.cfs, _a9_MockRandom_0.skp, > _a9_MockVariableIntBlock_0.tib, _c7_MockSep_0.doc, _c6_MockRandom_0.skp, > _c4_MockSep_0.frq, _c7_MockVariableIntBlock_0.doc, _c4.fdt, _c4_nrm.cfe, > _c2_Memory_0.ram, _c2_MockSep_0.doc, _c5_MockRandom_0.pyl, _c4.fdx, > _c5_MockRandom_0.frq, _a9_MockRandom_0.pyl, _c3_MockSep_0.frq, > _c3_MockSep_0.pos, _c6_MockVariableIntBlock_0.tib, _c2.fdt, _c7.fnm, _c2.fdx, > _c6_MockVariableIntBlock_0.tii, _c2_MockVariableIntBlock_0.frq, > _c5_MockRandom_0.tip, _a9_MockSep_0.doc, _c5_MockRandom_0.tim, > _c6_MockSep_0.pos, _c5_MockVariableIntBlock_0.pos, > _c4_MockVariableIntBlock_0.frq, _a9_MockVariableIntBlock_0.tii, _c6.fdx, > _c2_MockRandom_0.frq, _c6.si, _c6.fdt, _c7_MockRandom_0.tib, > _c6_MockRandom_0.sd, _c4_MockRandom_0.pyl, _c5_MockVariableIntBlock_0.skp, > _c5_MockVariableIntBlock_0.doc, _c3.fdt, _c4_MockRandom_0.frq, > _c4_MockRandom_0.tiv, _a9_MockRandom_0.doc, _c4_MockSep_0.doc, > _a9_MockSep_0.skp, _c7_MockRandom_0.tiv, _c3.fdx, > _c6_MockVariableIntBlock_0.frq, _c7_nrm.cfs, _a9_MockVariableIntBlock_0.doc, > _c6_MockVariableIntBlock_0.doc, _c3.fnm, _c5_Memory_0.ram, _c7_nrm.cfe, > _c7_MockVariableIntBlock_0.frq, _c2_MockVariableIntBlock_0.pyl, > _c6_MockRandom_0.pos, _c4_MockRandom_0.sd, _c6_MockVariableIntBlock_0.pyl, > _c4_MockVariableIntBlock_0.pos, _c3_MockVariableIntBlock_0.pos, > _c5_MockRandom_0.doc, _c2_MockVariableIntBlock_0.skp, _c3_MockRandom_0.sd, > _c4_MockSep_0.pyl, _c5_MockSep_0.doc, _c3_MockSep_0.doc, _a9_MockSep_0.pyl, > _c2_MockRandom_0.skp, _c2_MockSep_0.pos, _c3.si, _c3_MockRandom_0.tim, > _a9_MockRandom_0.frq, _c3_MockRandom_0.tip, _c7_MockSep_0.pos, > _c7_MockRandom_0.pos, _c5_MockVariableIntBlock_0.frq, > _a9_MockVariableIntBlock_0.pos, _c6_MockSep_0.tii, _a9_1.del, > _c4_MockVariableIntBlock_0.skp, _c6_MockSep_0.tib, _a9_MockSep_0.tib, > _c7.fdx, _c2_MockVariableIntBlock_0.tib, _c7.fdt, _c4_MockSep_0.tii, > _c2_MockRandom_0.pyl, _a9_MockSep_0.frq, _c2_MockVariableIntBlock_0.tii, > _c4_MockSep_0.tib, _a9_MockVariableIntBlock_0.frq, _c3_1.del, > _c7_MockSep_0.tib, _c5.fnm, _c6_MockSep_0.frq, _c7_MockSep_0.tii, > _c2_MockSep_0.tib, _c5_MockVariableIntBlock_0.tib, _c4_MockRandom_0.doc, > _c4_MockVariableIntBlock_0.doc, _c2_MockSep_0.tii, _c7_MockRandom_0.doc, > _a9_MockRandom_0.pos, _c5_MockVariableIntBlock_0.tii, _c4_Memory_0.ram, > _c3_MockVariableIntBlock_0.skp, _c3_MockVariableIntBlock_0.tii, > _c5_MockSep_0.frq, _c6_Memory_0.ram, _c3_MockVariableIntBlock_0.tib, > _c7_MockVariableIntBlock_0.tib, _c2_MockRandom_0.sd, > _c7_MockVariableIntBlock_0.pyl, _c3_MockRandom_0.prx, > _c7_MockVariableIntBlock_0.tii, _c6_MockSep_0.pyl, _c6_MockRandom_0.doc, > _c6_nrm.cfs, _c4_MockSep_0.skp, _c7_MockRandom_0.skp, _a9_nrm.cfs, _a9.si, > _c2_MockRandom_0.pos, _a9.fdx, _c6_nrm.cfe, _c5_MockRandom_0.pos, > _c6_MockVariableIntBlock_0.pos, _c5_MockSep_0.pos, _c7_Memory_0.ram, > _c7_MockSep_0.frq, _c4_MockRandom_0.skp, _c7_MockSep_0.pyl, _c7.si, _a9.fdt, > _c5_MockVariableIntBlock_0.pyl, _c5_MockSep_0.skp, > _c3_MockVariableIntBlock_0.doc, _c5_MockSep_0.pyl, _c2_MockSep_0.pyl, > _c3_MockSep_0.tib, _c3_MockSep_0.tii, _c2.fnm, _a9_MockRandom_0.sd, > _a9_MockVariableIntBlock_0.skp, _c3_MockSep_0.pyl, _c2_MockSep_0.frq, > _c7_MockRandom_0.sd, _a9_MockSep_0.pos, _c7_MockRandom_0.frq, > _c4_MockVariableIntBlock_0.tii, _c6.fnm, _c5_nrm.cfe, _c2.si, > _c4_MockVariableIntBlock_0.tib, _c6_MockSep_0.skp, _c6_MockSep_0.doc, > _c5_nrm.cfs, _c3_MockRandom_0.frq, _c6_MockRandom_0.frq, _a9_nrm.cfe, > _a9_MockVariableIntBlock_0.pyl, _c5_MockSep_0.tii, > _c4_MockVariableIntBlock_0.pyl, _c5_MockSep_0.tib, > _c2_MockVariableIntBlock_0.doc, _c4_MockRandom_0.pos, _c6_MockRandom_0.tim, > _c5.si, _c4.fnm, _c7_MockRandom_0.pyl, _c3_Memory_0.ram, > _c6_MockRandom_0.tip, _c2_MockSep_0.skp, _c3_MockVariableIntBlock_0.frq, > _c4.si, _c2_nrm.cfs, _c3_MockVariableIntBlock_0.pyl, _c5_MockRandom_0.skp, > _c3_nrm.cfs, _c3_MockSep_0.skp, _c2_nrm.cfe, _c5.fdt, > _c6_MockVariableIntBlock_0.skp, _c5.fdx, _c7_MockVariableIntBlock_0.skp, > segments_5, _c4_MockSep_0.pos, _a9_Memory_0.ram, _c3_nrm.cfe] > [junit4:junit4] 2> 36367 T3506 C73 P13581 > oasc.SolrDeletionPolicy.updateCommits newest commit = 5 > [junit4:junit4] 2> 36391 T3506 C73 P13581 oass.SolrIndexSearcher.<init> > Opening Searcher@2ccf2697 main > [junit4:junit4] 2> 36391 T3506 C73 P13581 oasu.DirectUpdateHandler2.commit > end_commit_flush > [junit4:junit4] 2> 36392 T3589 oasc.SolrCore.registerSearcher [collection1] > Registered new searcher Searcher@2ccf2697 > main{StandardDirectoryReader(segments_5:937:nrt _a9(4.0.0.1):C1628/36 > _c3(4.0.0.1):C320/22 _c2(4.0.0.1):C10 _c4(4.0.0.1):C10 _c5(4.0.0.1):C10 > _c6(4.0.0.1):C10 _c7(4.0.0.1):C2)} > [junit4:junit4] 2> 36393 T3506 C73 P13581 /update > {waitSearcher=true&commit=true&wt=javabin&commit_end_point=true&softCommit=false&version=2} > {commit=} 0 418 > [junit4:junit4] 2> 36393 T1645 C72 P13579 /update > {waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} > {commit=} 0 1104 > [junit4:junit4] 2> 36395 T1647 C72 P13579 REQ /select > {distrib=false&wt=javabin&q=*:*&version=2} hits=1936 status=0 QTime=0 > [junit4:junit4] 2> 36397 T3550 C73 P13581 REQ /select > {distrib=false&wt=javabin&q=*:*&version=2} hits=1932 status=0 QTime=1 > [junit4:junit4] 2> 36400 T1647 C72 P13579 REQ /select > {distrib=false&wt=javabin&q=*:*&version=2} hits=1936 status=0 QTime=0 > [junit4:junit4] 2> 36402 T3550 C73 P13581 REQ /select > {distrib=false&wt=javabin&q=*:*&version=2} hits=1932 status=0 QTime=0 > [junit4:junit4] 2> 36403 T1622 oazs.PrepRequestProcessor.pRequest Processed > session termination for sessionid: 0x138b8eaad580006 > [junit4:junit4] 2> 36405 T1671 oaz.ClientCnxn$EventThread.run EventThread > shut down > [junit4:junit4] 2> 36405 T1619 oazs.NIOServerCnxn.closeSock Closed socket > connection for client /140.211.11.196:13587 which had sessionid > 0x138b8eaad580006 > [junit4:junit4] 2> 36405 T1617 oaz.ZooKeeper.close Session: > 0x138b8eaad580006 closed > [junit4:junit4] 2> 36405 T1622 oazs.PrepRequestProcessor.pRequest Processed > session termination for sessionid: 0x138b8eaad580005 > [junit4:junit4] 2> 36415 T1667 oaz.ClientCnxn$EventThread.run EventThread > shut down > [junit4:junit4] 2> 36415 T1617 oaz.ZooKeeper.close Session: > 0x138b8eaad580005 closed > [junit4:junit4] 2> 36415 T1619 oazs.NIOServerCnxn.closeSock Closed socket > connection for client /140.211.11.196:13583 which had sessionid > 0x138b8eaad580005 > [junit4:junit4] 2> 36415 T1621 oazs.SyncRequestProcessor.run > SyncRequestProcessor exited! > [junit4:junit4] 2> 36415 T1622 oazs.PrepRequestProcessor.run > PrepRequestProcessor exited loop! > [junit4:junit4] 2> 36415 T1617 oazs.FinalRequestProcessor.shutdown shutdown > of request processor complete > [junit4:junit4] 2> 37592 T1617 oazs.NIOServerCnxn.closeSock Closed socket > connection for client /140.211.11.196:13598 which had sessionid > 0x138b8eaad580007 > [junit4:junit4] 2> 37592 T1617 oazs.NIOServerCnxn.closeSock Closed socket > connection for client /140.211.11.196:13580 which had sessionid > 0x138b8eaad580003 > [junit4:junit4] 2> 37592 T3512 oaz.ClientCnxn$SendThread.run Unable to read > additional data from server sessionid 0x138b8eaad580007, likely server has > closed socket, closing socket connection and attempting reconnect > [junit4:junit4] 2> 37593 T1650 oaz.ClientCnxn$SendThread.run Unable to read > additional data from server sessionid 0x138b8eaad580003, likely server has > closed socket, closing socket connection and attempting reconnect > [junit4:junit4] 2> 37593 T1617 oazs.NIOServerCnxn.closeSock Closed socket > connection for client /140.211.11.196:13578 which had sessionid > 0x138b8eaad580002 > [junit4:junit4] 2> 37593 T1636 oaz.ClientCnxn$SendThread.run Unable to read > additional data from server sessionid 0x138b8eaad580002, likely server has > closed socket, closing socket connection and attempting reconnect > [junit4:junit4] 2> 37593 T1619 oazs.NIOServerCnxn$Factory.run NIOServerCnxn > factory exited run method > [junit4:junit4] 2> 37594 T1617 oazs.FinalRequestProcessor.shutdown shutdown > of request processor complete > [junit4:junit4] 2> 37594 T1617 oasc.ChaosMonkey.monkeyLog monkey: stop > shard! 13577 > [junit4:junit4] 2> 37594 T1617 oasc.CoreContainer.shutdown Shutting down > CoreContainer instance=945416360 > [junit4:junit4] 2> 37594 T1617 oasc.SolrCore.close [collection1] CLOSING > SolrCore org.apache.solr.core.SolrCore@3ba7b45b > [junit4:junit4] 2> 37595 T1617 oasc.SolrCore.closeSearcher [collection1] > Closing main searcher on request. > [junit4:junit4] 2> 37597 T1617 oasu.DirectUpdateHandler2.close closing > DirectUpdateHandler2{commits=1,autocommits=0,soft > autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=3844,cumulative_deletesById=1908,cumulative_deletesByQuery=0,cumulative_errors=0} > [junit4:junit4] 2> 37694 T1637 oascc.ConnectionManager.process Watcher > org.apache.solr.common.cloud.ConnectionManager@16b98a7f > name:ZooKeeperConnection Watcher:127.0.0.1:13573/solr got event WatchedEvent > state:Disconnected type:None path:null path:null type:None > [junit4:junit4] 2> 37694 T1638 oasc.Overseer$CloudStateUpdater.amILeader > WARNING org.apache.zookeeper.KeeperException$SessionExpiredException: > KeeperErrorCode = Session expired for /overseer_elect/leader > [junit4:junit4] 2> at > org.apache.zookeeper.KeeperException.create(KeeperException.java:118) > [junit4:junit4] 2> at > org.apache.zookeeper.KeeperException.create(KeeperException.java:42) > [junit4:junit4] 2> at > org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:927) > [junit4:junit4] 2> at > org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:289) > [junit4:junit4] 2> at > org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:286) > [junit4:junit4] 2> at > org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:67) > [junit4:junit4] 2> at > org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:286) > [junit4:junit4] 2> at > org.apache.solr.cloud.Overseer$CloudStateUpdater.amILeader(Overseer.java:187) > [junit4:junit4] 2> at > org.apache.solr.cloud.Overseer$CloudStateUpdater.run(Overseer.java:113) > [junit4:junit4] 2> at java.lang.Thread.run(Thread.java:679) > [junit4:junit4] 2> > [junit4:junit4] 2> 37694 T1617 oaz.ZooKeeper.close Session: > 0x138b8eaad580002 closed > [junit4:junit4] 2> 37694 T1651 oascc.ConnectionManager.process Watcher > org.apache.solr.common.cloud.ConnectionManager@4f2c0836 > name:ZooKeeperConnection Watcher:127.0.0.1:13573/solr got event WatchedEvent > state:Disconnected type:None path:null path:null type:None > [junit4:junit4] 2> 37694 T3513 oascc.ConnectionManager.process Watcher > org.apache.solr.common.cloud.ConnectionManager@4270d2e5 > name:ZooKeeperConnection Watcher:127.0.0.1:13573/solr got event WatchedEvent > state:Disconnected type:None path:null path:null type:None > [junit4:junit4] 2> 37694 T1617 oejsh.ContextHandler.doStop stopped > o.e.j.s.ServletContextHandler{/solr,null} > [junit4:junit4] 2> 37694 T1639 oasc.OverseerCollectionProcessor.run WARNING > Overseer cannot talk to ZK > [junit4:junit4] 2> 37694 T1638 oasc.Overseer$CloudStateUpdater.amILeader > According to ZK I (id=88023510803808258-127.0.0.1:13577_solr-n_0000000000) am > no longer a leader. > [junit4:junit4] 2> 37694 T1637 oaz.ClientCnxn$EventThread.run EventThread > shut down > [junit4:junit4] 2> 37747 T1617 oasc.ChaosMonkey.monkeyLog monkey: stop > shard! 13579 > [junit4:junit4] 2> 37747 T1617 oasc.CoreContainer.shutdown Shutting down > CoreContainer instance=2125074601 > [junit4:junit4] 2> 37747 T1617 oasc.SolrCore.close [collection1] CLOSING > SolrCore org.apache.solr.core.SolrCore@1822bbe7 > [junit4:junit4] 2> 37748 T1617 oasc.SolrCore.closeSearcher [collection1] > Closing main searcher on request. > [junit4:junit4] 2> 37750 T1617 oasu.DirectUpdateHandler2.close closing > DirectUpdateHandler2{commits=3,autocommits=0,soft > autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=3844,cumulative_deletesById=1908,cumulative_deletesByQuery=0,cumulative_errors=0} > [junit4:junit4] 2> 39113 T1620 oazs.SessionTrackerImpl.run > SessionTrackerImpl exited loop! > [junit4:junit4] 2> 39454 T3512 oaz.ClientCnxn$SendThread.startConnect > Opening socket connection to server localhost/127.0.0.1:13573 > [junit4:junit4] 2> 39581 T1650 oaz.ClientCnxn$SendThread.startConnect > Opening socket connection to server localhost/127.0.0.1:13573 > [junit4:junit4] 2> 46914 T670 oaz.ClientCnxn$SendThread.run Client session > timed out, have not heard from server in 12001ms for sessionid > 0x138b8e637650010, closing socket connection and attempting reconnect > [junit4:junit4] 2> 48053 T670 oaz.ClientCnxn$SendThread.startConnect > Opening socket connection to server localhost/127.0.0.1:65199 > [junit4:junit4] 2> 49695 T3512 oaz.ClientCnxn$SendThread.run Client session > timed out, have not heard from server in 12001ms for sessionid > 0x138b8eaad580007, closing socket connection and attempting reconnect > [junit4:junit4] 2> 49796 T1651 oaz.ClientCnxn$EventThread.run EventThread > shut down > [junit4:junit4] 2> 49796 T1617 oaz.ZooKeeper.close Session: > 0x138b8eaad580003 closed > [junit4:junit4] 2> 49797 T1617 oejsh.ContextHandler.doStop stopped > o.e.j.s.ServletContextHandler{/solr,null} > [junit4:junit4] 2> 49849 T1617 oasc.ChaosMonkey.monkeyLog monkey: stop > shard! 13581 > [junit4:junit4] 2> 49849 T1617 oasc.CoreContainer.shutdown Shutting down > CoreContainer instance=1755464392 > [junit4:junit4] 2> 49849 T1617 oasc.RecoveryStrategy.close WARNING Stopping > recovery for core collection1 zkNodeName=127.0.0.1:13581_solr_collection1 > [junit4:junit4] 2> 49850 T1617 oasc.SolrCore.close [collection1] CLOSING > SolrCore org.apache.solr.core.SolrCore@22c3105d > [junit4:junit4] 2> 49852 T1617 oasc.SolrCore.closeSearcher [collection1] > Closing main searcher on request. > [junit4:junit4] 2> 49853 T1617 oasu.DirectUpdateHandler2.close closing > DirectUpdateHandler2{commits=3,autocommits=0,soft > autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=1825,cumulative_deletesById=943,cumulative_deletesByQuery=0,cumulative_errors=0} > [junit4:junit4] 2> 49854 T1617 oasc.RecoveryStrategy.close WARNING Stopping > recovery for core collection1 zkNodeName=127.0.0.1:13581_solr_collection1 > [junit4:junit4] 2> 51068 T3512 oaz.ClientCnxn$SendThread.startConnect > Opening socket connection to server localhost/127.0.0.1:13573 > [junit4:junit4] 2> 59016 T670 oaz.ClientCnxn$SendThread.run Client session > timed out, have not heard from server in 12001ms for sessionid > 0x138b8e637650010, closing socket connection and attempting reconnect > [junit4:junit4] 2> 61089 T670 oaz.ClientCnxn$SendThread.startConnect > Opening socket connection to server localhost/127.0.0.1:65199 > [junit4:junit4] 2> 61898 T3513 oaz.ClientCnxn$EventThread.run EventThread > shut down > [junit4:junit4] 2> 61898 T1617 oaz.ZooKeeper.close Session: > 0x138b8eaad580007 closed > [junit4:junit4] 2> 61898 T1617 oejsh.ContextHandler.doStop stopped > o.e.j.s.ServletContextHandler{/solr,null} > [junit4:junit4] 2> 62061 T1617 oas.SolrTestCaseJ4.tearDown ###Ending > testDistribSearch > [junit4:junit4] 2> NOTE: reproduce with: ant test > -Dtestcase=RecoveryZkTest -Dtests.method=testDistribSearch > -Dtests.seed=98FA251234CC1277 -Dtests.multiplier=3 -Dtests.slow=true > -Dtests.locale=sr -Dtests.timezone=SystemV/MST7MDT > -Dtests.file.encoding=ISO8859-1 > [junit4:junit4] 2> > [junit4:junit4] > (@AfterClass output) > [junit4:junit4] 2> 62087 T1617 oas.SolrTestCaseJ4.deleteCore ###deleteCore > [junit4:junit4] 2> NOTE: test params are: codec=Lucene40: > {range_facet_l=MockVariableIntBlock(baseBlockSize=119), > text=PostingsFormat(name=Memory doPackFST= true), > _version_=PostingsFormat(name=MockSep), > rnd_b=PostingsFormat(name=MockRandom), > intDefault=PostingsFormat(name=MockSep), id=PostingsFormat(name=MockRandom), > timestamp=MockVariableIntBlock(baseBlockSize=119), > a_t=MockVariableIntBlock(baseBlockSize=119), > range_facet_sl=PostingsFormat(name=MockRandom), > range_facet_si=MockVariableIntBlock(baseBlockSize=119), > other_tl1=PostingsFormat(name=MockSep), > multiDefault=MockVariableIntBlock(baseBlockSize=119), > a_si=PostingsFormat(name=MockSep)}, > sim=RandomSimilarityProvider(queryNorm=true,coord=false): {}, locale=sr, > timezone=SystemV/MST7MDT > [junit4:junit4] 2> NOTE: FreeBSD 9.0-RELEASE amd64/Sun Microsystems Inc. > 1.6.0_32 (64-bit)/cpus=16,threads=18,free=147743512,total=354746368 > [junit4:junit4] 2> NOTE: All tests run in this JVM: > [CommonGramsFilterFactoryTest, UUIDFieldTest, TestRecovery, > QueryEqualityTest, IndexSchemaRuntimeFieldTest, FileBasedSpellCheckerTest, > TestPersianNormalizationFilterFactory, MBeansHandlerTest, > SignatureUpdateProcessorFactoryTest, TestCJKBigramFilterFactory, > FieldAnalysisRequestHandlerTest, FieldMutatingUpdateProcessorTest, > TestNGramFilters, TestCSVResponseWriter, TestRussianFilters, > TestGalicianMinimalStemFilterFactory, TestJapaneseReadingFormFilterFactory, > CSVRequestHandlerTest, JsonLoaderTest, WordBreakSolrSpellCheckerTest, > TestWriterPerf, FullSolrCloudTest, TestPortugueseLightStemFilterFactory, > TestStressLucene, DocumentBuilderTest, TestChineseTokenizerFactory, > TestCoreContainer, TestReplicationHandler, UpdateParamsTest, > TestJmxMonitoredMap, TestSolrDeletionPolicy1, > TestRussianLightStemFilterFactory, PolyFieldTest, > TestGermanNormalizationFilterFactory, IndexSchemaTest, TestArbitraryIndexDir, > TestDocumentBuilder, TestTurkishLowerCaseFilterFactory, > TestLMDirichletSimilarityFactory, TestQueryTypes, TestSolrDeletionPolicy2, > MultiTermTest, TestHindiFilters, BasicZkTest, TestRealTimeGet, TestUtils, > CloudStateTest, TestQuerySenderNoQuery, TimeZoneUtilsTest, > TestLuceneMatchVersion, SolrCoreTest, SpellCheckComponentTest, > TestPropInjectDefaults, TestXIncludeConfig, CircularListTest, > TestNumberUtils, TestHungarianLightStemFilterFactory, > TestElisionFilterFactory, DistanceFunctionTest, SolrPluginUtilsTest, > TestKeywordMarkerFilterFactory, AlternateDirectoryTest, > TermVectorComponentTest, TestWikipediaTokenizerFactory, SuggesterFSTTest, > TestSuggestSpellingConverter, TestIndexSearcher, TestGroupingSearch, > SoftAutoCommitTest, SpellingQueryConverterTest, TestBeiderMorseFilterFactory, > OverseerTest, RecoveryZkTest] > [junit4:junit4] 2> > [junit4:junit4] Completed on J1 in 62.11s, 1 test, 1 failure <<< FAILURES! > [...truncated 586 lines...] > > [...truncated 31122 lines...] > > [...truncated 31122 lines...] > > [...truncated 31122 lines...] > > [...truncated 31122 lines...] > > [...truncated 31107 lines...] > BUILD FAILED > /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x/checkout/build.xml:29: > The following error occurred while executing this line: > /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x/checkout/solr/build.xml:144: > The following error occurred while executing this line: > /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x/checkout/lucene/module-build.xml:62: > The following error occurred while executing this line: > /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x/checkout/lucene/common-build.xml:1094: > The following error occurred while executing this line: > /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x/checkout/lucene/common-build.xml:789: > There were test failures: 305 suites, 1085 tests, 1 failure, 5 ignored > > Total time: 25 minutes 24 seconds > Build step 'Execute shell' marked build as failure > Archiving artifacts > Recording test results > Publishing Clover coverage report... > No Clover report will be published due to a Build Failure > Email was triggered for: Failure > Sending email for trigger: Failure > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] - Mark Miller lucidimagination.com --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
