Couldn't create a new native thread....

On Jul 25, 2012, at 1:08 PM, Apache Jenkins Server <[email protected]> 
wrote:

> Build: https://builds.apache.org/job/Lucene-Solr-tests-only-trunk/14941/
> 
> 1 tests failed.
> REGRESSION:  org.apache.solr.cloud.RecoveryZkTest.testDistribSearch
> 
> Error Message:
> expected:<2134> but was:<2131>
> 
> Stack Trace:
> java.lang.AssertionError: expected:<2134> but was:<2131>
>       at 
> __randomizedtesting.SeedInfo.seed([464A264BD38912FE:C7ACA853A4D672C2]:0)
>       at org.junit.Assert.fail(Assert.java:93)
>       at org.junit.Assert.failNotEquals(Assert.java:647)
>       at org.junit.Assert.assertEquals(Assert.java:128)
>       at org.junit.Assert.assertEquals(Assert.java:472)
>       at org.junit.Assert.assertEquals(Assert.java:456)
>       at org.apache.solr.cloud.RecoveryZkTest.doTest(RecoveryZkTest.java:107)
>       at 
> org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:679)
>       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>       at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>       at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>       at java.lang.reflect.Method.invoke(Method.java:616)
>       at 
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1995)
>       at 
> com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:132)
>       at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:818)
>       at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:877)
>       at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:891)
>       at 
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
>       at 
> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>       at 
> org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:32)
>       at 
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>       at 
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
>       at 
> org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)
>       at 
> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
>       at 
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
>       at 
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>       at 
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:825)
>       at 
> com.carrotsearch.randomizedtesting.RandomizedRunner.access$700(RandomizedRunner.java:132)
>       at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$3$1.run(RandomizedRunner.java:671)
>       at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:697)
>       at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:736)
>       at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:747)
>       at 
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
>       at 
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>       at 
> org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)
>       at 
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
>       at 
> org.apache.lucene.util.TestRuleIcuHack$1.evaluate(TestRuleIcuHack.java:51)
>       at 
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
>       at 
> org.apache.lucene.util.TestRuleNoInstanceHooksOverrides$1.evaluate(TestRuleNoInstanceHooksOverrides.java:53)
>       at 
> org.apache.lucene.util.TestRuleNoStaticHooksShadowing$1.evaluate(TestRuleNoStaticHooksShadowing.java:52)
>       at 
> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:36)
>       at 
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>       at 
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
>       at 
> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
>       at 
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:605)
>       at 
> com.carrotsearch.randomizedtesting.RandomizedRunner.access$400(RandomizedRunner.java:132)
>       at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:551)
> 
> 
> 
> 
> Build Log:
> [...truncated 35962 lines...]
> [junit4:junit4]   2> 37397 T1857 C151 P26525 /update {wt=javabin&version=2} 
> {add=[2269 (1408484678163234816)]} 0 0
> [junit4:junit4]   2> 37398 T3647 C153 P62559 REQ /update 
> {update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
> [junit4:junit4]   2> 37398 T1871 C152 P55793 /update {wt=javabin&version=2} 
> {add=[11855 (1408484678162186240)]} 0 2
> [junit4:junit4]   2> 37401 T1857 C151 P26525 /update {wt=javabin&version=2} 
> {delete=[11789 (-1408484678167429120)]} 0 0
> [junit4:junit4]   2> 37401 T3647 C153 P62559 REQ /update 
> {update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
> [junit4:junit4]   2> 37401 T1872 C152 P55793 /update {wt=javabin&version=2} 
> {add=[2269 (1408484678165331968)]} 0 2
> [junit4:junit4]   2> 37404 T1857 C151 P26525 /update {wt=javabin&version=2} 
> {add=[2270 (1408484678170574848)]} 0 0
> [junit4:junit4]   2> 37405 T3647 C153 P62559 REQ /update 
> {update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
> [junit4:junit4]   2> 37405 T1871 C152 P55793 /update {wt=javabin&version=2} 
> {delete=[11789 (-1408484678169526272)]} 0 2
> [junit4:junit4]   2> 37407 T1857 C151 P26525 /update {wt=javabin&version=2} 
> {add=[11856 (1408484678173720576)]} 0 0
> [junit4:junit4]   2> 37408 T3647 C153 P62559 REQ /update 
> {update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
> [junit4:junit4]   2> 37408 T1872 C152 P55793 /update {wt=javabin&version=2} 
> {add=[2270 (1408484678172672000)]} 0 2
> [junit4:junit4]   2> 37410 T1857 C151 P26525 /update {wt=javabin&version=2} 
> {delete=[2177 (-1408484678176866304)]} 0 0
> [junit4:junit4]   2> 37411 T3647 C153 P62559 REQ /update 
> {update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
> [junit4:junit4]   2> 37411 T1871 C152 P55793 /update {wt=javabin&version=2} 
> {add=[11856 (1408484678175817728)]} 0 2
> [junit4:junit4]   2> 37413 T1857 C151 P26525 /update {wt=javabin&version=2} 
> {add=[11857 (1408484678180012032)]} 0 0
> [junit4:junit4]   2> 37414 T3647 C153 P62559 REQ /update 
> {update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
> [junit4:junit4]   2> 37414 T1872 C152 P55793 /update {wt=javabin&version=2} 
> {delete=[2177 (-1408484678178963456)]} 0 2
> [junit4:junit4]   2> 37417 T1857 C151 P26525 /update {wt=javabin&version=2} 
> {add=[2271 (1408484678184206336)]} 0 0
> [junit4:junit4]   2> 37419 T3647 C153 P62559 REQ /update 
> {update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
> [junit4:junit4]   2> 37420 T1871 C152 P55793 /update {wt=javabin&version=2} 
> {add=[11857 (1408484678182109184)]} 0 5
> [junit4:junit4]   2> 37421 T3622 C153 P62559 REQ /update 
> {update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
> [junit4:junit4]   2> 37422 T1872 C152 P55793 /update {wt=javabin&version=2} 
> {add=[2271 (1408484678186303488)]} 0 3
> [junit4:junit4]   2> 37422 T1857 C151 P26525 /update {wt=javabin&version=2} 
> {add=[11858 (1408484678189449216)]} 0 0
> [junit4:junit4]   2> 37424 T1856 C151 P26525 /update {wt=javabin&version=2} 
> {delete=[2178 (-1408484678191546368)]} 0 0
> [junit4:junit4]   2> 37426 T3622 C153 P62559 REQ /update 
> {update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
> [junit4:junit4]   2> 37426 T1872 C152 P55793 /update {wt=javabin&version=2} 
> {add=[11858 (1408484678191546368)]} 0 2
> [junit4:junit4]   2> 37428 T3647 C153 P62559 REQ /update 
> {update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
> [junit4:junit4]   2> 37429 T1871 C152 P55793 /update {wt=javabin&version=2} 
> {delete=[2178 (-1408484678193643520)]} 0 3
> [junit4:junit4]   2> 37429 T1856 C151 P26525 /update {wt=javabin&version=2} 
> {add=[11859 (1408484678195740672)]} 0 1
> [junit4:junit4]   2> 37431 T1857 C151 P26525 /update {wt=javabin&version=2} 
> {add=[2272 (1408484678198886400)]} 0 0
> [junit4:junit4]   2> 37431 T1844 oascc.ZkStateReader.updateCloudState 
> Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> 37433 T3647 C153 P62559 REQ /update 
> {update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
> [junit4:junit4]   2> 37434 T1871 C152 P55793 /update {wt=javabin&version=2} 
> {add=[11859 (1408484678198886400)]} 0 3
> [junit4:junit4]   2> added docs:1859 with 26 fails deletes:900
> [junit4:junit4]   2> 37437 T3647 C153 P62559 REQ /update 
> {update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
> [junit4:junit4]   2> 37438 T1872 C152 P55793 /update {wt=javabin&version=2} 
> {add=[2272 (1408484678202032128)]} 0 4
> [junit4:junit4]   2> added docs:2272 with 36 fails deletes:1097
> [junit4:junit4]   2> 37440 T1857 C151 P26525 oasu.DirectUpdateHandler2.commit 
> start 
> commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
> [junit4:junit4]   2> 37893 T1857 C151 P26525 oasc.SolrDeletionPolicy.onCommit 
> SolrDeletionPolicy.onCommit: commits:num=2
> [junit4:junit4]   2>          
> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-trunk/checkout/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-1343235625223/control/data/index
>  lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@72e46ea8; 
> maxCacheMB=48.0 
> maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
> [junit4:junit4]   2>          
> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-trunk/checkout/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-1343235625223/control/data/index
>  lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@72e46ea8; 
> maxCacheMB=48.0 
> maxMergeSizeMB=4.0),segFN=segments_2,generation=2,filenames=[_d0.frq, 
> _6g.tip, _c9.fnm, _6g.tim, _d0.fnm, _cy.fdt, _c9.fdx, _c9.frq, _cy.prx, 
> _6g.fnm, _cz.tip, _6g.frq, _cz.tim, _cy.tim, _6g.prx, _cy.fnm, _d1.tim, 
> _c9.prx, _d1_nrm.cfe, _d1.tip, _d0.fdt, _cy.frq, _cz_nrm.cfs, _cy_nrm.cfs, 
> _d0.si, _d0.fdx, _6g.si, _cz.fnm, _d1_nrm.cfs, _d3.frq, _d3.fnm, _c9.fdt, 
> _d3.prx, _c9_nrm.cfs, _cy.fdx, _d0.prx, _d2.tip, _cy_nrm.cfe, _c9_nrm.cfe, 
> _d1.prx, _cz.frq, _d3_nrm.cfe, _d3.si, _d0_nrm.cfs, _cz.fdx, _d2.tim, _d2.si, 
> _d3_nrm.cfs, _c9_1.del, _cz.si, _d2.fnm, _d3.tip, _d0.tip, _d0_nrm.cfe, 
> _d3.tim, _d1.si, _cy.tip, _d0.tim, _d2.frq, _6g.fdt, _6g.fdx, _6g_nrm.cfe, 
> _c9.tim, _c9.si, _d1.fdx, _cz_nrm.cfe, _d1.frq, _d2.prx, _d1.fdt, 
> _d2_nrm.cfs, _6g_1.del, _d2.fdt, _6g_nrm.cfs, _d3.fdx, _cz.fdt, _d3.fdt, 
> _cz.prx, segments_2, _cz_1.del, _cy.si, _d2_nrm.cfe, _d1.fnm, _d2.fdx, 
> _c9.tip]
> [junit4:junit4]   2> 37893 T1857 C151 P26525 
> oasc.SolrDeletionPolicy.updateCommits newest commit = 2
> [junit4:junit4]   2> 37905 T1857 C151 P26525 oass.SolrIndexSearcher.<init> 
> Opening Searcher@414b8fd3 main
> [junit4:junit4]   2> 37906 T1857 C151 P26525 oasu.DirectUpdateHandler2.commit 
> end_commit_flush
> [junit4:junit4]   2> 37906 T1867 oasc.SolrCore.registerSearcher [collection1] 
> Registered new searcher Searcher@414b8fd3 
> main{StandardDirectoryReader(segments_2:1003 _6g(5.0):C1015/22 
> _c9(5.0):C1006/73 _cz(5.0):C183/16 _cy(5.0):C10 _d0(5.0):C10 _d1(5.0):C10 
> _d2(5.0):C10 _d3(5.0):C1)}
> [junit4:junit4]   2> 37907 T1857 C151 P26525 /update 
> {waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} 
> {commit=} 0 467
> [junit4:junit4]   2> 37910 T1872 C152 P55793 oasu.DirectUpdateHandler2.commit 
> start 
> commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
> [junit4:junit4]   2> 38180 T1872 C152 P55793 oasc.SolrDeletionPolicy.onCommit 
> SolrDeletionPolicy.onCommit: commits:num=3
> [junit4:junit4]   2>          
> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-trunk/checkout/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-1343235625223/jetty1/index
>  lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5c22bd75; 
> maxCacheMB=48.0 
> maxMergeSizeMB=4.0),segFN=segments_3,generation=3,filenames=[_84.si, 
> _85_nrm.cfs, _86.fdx, _85.prx, _86.fdt, _82_nrm.cfe, _82.prx, _86_nrm.cfe, 
> _86.si, _6h_nrm.cfe, _6h.si, _85.fdx, _82_1.del, _84.fdx, _84.fdt, 
> _81_nrm.cfs, _81.prx, _82.frq, _84_nrm.cfe, _85.fdt, _83_nrm.cfs, _86.prx, 
> _82.fnm, _6h_nrm.cfs, _81.frq, _85.tim, _84.frq, _84.tip, _82.fdt, _85.tip, 
> _84.tim, _81.fnm, _82.fdx, _81.fdt, _6h_1.del, _84.fnm, _81.tim, _6h.fnm, 
> _81.tip, _81.fdx, _85.frq, _83.si, _86.fnm, _85.fnm, _86.frq, _81_nrm.cfe, 
> _6h.frq, _6h.fdt, _6h.tim, _84_nrm.cfs, _6h.fdx, _83_nrm.cfe, _83.tip, 
> _6h.tip, _83.tim, _84.prx, _85.si, _86_nrm.cfs, _82_nrm.cfs, _83.fnm, 
> _86.tim, _82.tip, _82.si, _82.tim, _81.si, _6h.prx, _83.frq, _83.fdt, 
> segments_3, _85_nrm.cfe, _86.tip, _83.prx, _83.fdx]
> [junit4:junit4]   2>          
> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-trunk/checkout/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-1343235625223/jetty1/index
>  lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5c22bd75; 
> maxCacheMB=48.0 
> maxMergeSizeMB=4.0),segFN=segments_4,generation=4,filenames=[_89.frq, 
> _89.fnm, _cb.fnm, _cb.prx, _cc_nrm.cfe, _ca.fdt, _ca.tim, _ce_nrm.cfs, 
> _ca.fdx, _cb_nrm.cfe, _ca_1.del, _89_nrm.cfs, _89.prx, _ca.tip, _cb.si, 
> _ca.frq, _ce.frq, _cc_nrm.cfs, _cb.fdx, _cd.si, _ce_nrm.cfe, _cc.tim, 
> _ce.fdt, _ca.fnm, _cb.fdt, _cb_nrm.cfs, _cc.tip, _c8.prx, _ce.fdx, _ce.fnm, 
> _89_1.del, _c8_nrm.cfs, _c8.tip, _ce.prx, _cc.prx, _c8_nrm.cfe, _cc.fdt, 
> _cc.si, _cd_nrm.cfe, _89.si, _c8.tim, _ca_nrm.cfs, _cd.frq, _c8.fdt, _cc.fdx, 
> _cd.prx, _cd.fnm, _ce.si, _cd.fdx, _cd.fdt, _89.fdt, _cc.fnm, _89_nrm.cfe, 
> _89.fdx, _cb.tim, _ce.tim, _c8.fnm, _cc.frq, _cb.tip, _ce.tip, _cb.frq, 
> _c8.frq, _c8.fdx, _cd_nrm.cfs, _ca.si, _ca.prx, _c8.si, _89.tim, _ca_nrm.cfe, 
> _cd.tip, _cd.tim, _89.tip, segments_4]
> [junit4:junit4]   2>          
> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-trunk/checkout/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-1343235625223/jetty1/index
>  lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5c22bd75; 
> maxCacheMB=48.0 
> maxMergeSizeMB=4.0),segFN=segments_5,generation=5,filenames=[_89.frq, 
> _d7.fdx, _d7.prx, _d7.fdt, _d7_nrm.cfs, _89.si, _d7_nrm.cfe, _d7.fnm, 
> _89.fnm, _d6.fdt, _d5.fnm, _d6.fdx, _d5.tip, _d6.prx, _d5.frq, _d5.tim, 
> _89.prx, _89_nrm.cfs, _d6.si, _d7.tip, _d7.tim, _89.fdt, _89.fdx, 
> _89_nrm.cfe, _d5_nrm.cfe, _d5.fdt, _d6_nrm.cfs, _d5.fdx, _d6.fnm, _89_1.del, 
> _d5.si, _d7.frq, _d5_1.del, _d5.prx, _d6.tim, _89.tim, _d5_nrm.cfs, _d6.frq, 
> _d7.si, _d6.tip, segments_5, _89.tip, _d6_nrm.cfe]
> [junit4:junit4]   2> 38223 T1872 C152 P55793 
> oasc.SolrDeletionPolicy.updateCommits newest commit = 5
> [junit4:junit4]   2> 38240 T1872 C152 P55793 oass.SolrIndexSearcher.<init> 
> Opening Searcher@263d1f2a main
> [junit4:junit4]   2> 38241 T1879 oasc.SolrCore.registerSearcher [collection1] 
> Registered new searcher Searcher@263d1f2a 
> main{StandardDirectoryReader(segments_5:1016 _89(5.0):C1302/24 
> _d5(5.0):C848/3 _d7(5.0):C6 _d6(5.0):C5)}
> [junit4:junit4]   2> 38241 T1872 C152 P55793 oasu.DirectUpdateHandler2.commit 
> end_commit_flush
> [junit4:junit4]   2> 38245 T3647 C153 P62559 oasu.DirectUpdateHandler2.commit 
> start 
> commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
> [junit4:junit4]   2> 38327 T3647 C153 P62559 oasc.SolrDeletionPolicy.onCommit 
> SolrDeletionPolicy.onCommit: commits:num=2
> [junit4:junit4]   2>          
> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-trunk/checkout/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-1343235625223/jetty2/index.20120726000101462
>  lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@29a430a0; 
> maxCacheMB=48.0 
> maxMergeSizeMB=4.0),segFN=segments_5,generation=5,filenames=[_89.frq, 
> _cu.fnm, _cu_nrm.cfe, _cw_nrm.cfs, _cw.frq, _ct_nrm.cfs, _cu.frq, _89.fnm, 
> _cw_nrm.cfe, _cw.fnm, _ct.fnm, _cr.frq, _cu.tim, _cr.fdt, _ct_nrm.cfe, 
> _ct.frq, _cu.tip, _cs.fnm, _cr.prx, _cu_nrm.cfs, _ct.prx, _89.prx, 
> _89_nrm.cfs, _cs.frq, _cr.fdx, _cv.fdx, _cw.prx, _cv.fdt, _cv.si, _ct.tim, 
> _89_1.del, _ct.tip, _cw.tip, _cw.tim, _cv_nrm.cfs, _cr_nrm.cfs, _89.si, 
> _cw.fdt, _cr_1.del, _cw.fdx, _cv_nrm.cfe, _cv.tip, _cv.tim, _cs_nrm.cfs, 
> _cw.si, _cs.tip, _cr_nrm.cfe, _cs.fdt, _cr.fnm, _cs.tim, _cs_nrm.cfe, 
> _cv.frq, _cs.fdx, _cv.fnm, _89.fdt, _ct.fdt, _cu.fdt, _89.fdx, _89_nrm.cfe, 
> _cu.fdx, _ct.fdx, _ct.si, _cv.prx, _cs.si, _cr.tim, _cs.prx, _cr.tip, _cr.si, 
> _89.tim, _cu.si, _cu.prx, segments_5, _89.tip]
> [junit4:junit4]   2>          
> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-trunk/checkout/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-1343235625223/jetty2/index.20120726000101462
>  lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@29a430a0; 
> maxCacheMB=48.0 
> maxMergeSizeMB=4.0),segFN=segments_6,generation=6,filenames=[_89.frq, 
> _d7.fdx, _d8.frq, _d8.tim, _d7.prx, _d7.fdt, _89.si, _d7_nrm.cfs, _89.fnm, 
> _d7.fnm, _d7_nrm.cfe, _d8.fnm, _d8.si, _89_nrm.cfs, _89.prx, _d7.tip, 
> _d8.prx, _89.fdt, _d7.tim, _89_nrm.cfe, _89.fdx, _d7_1.del, _89_1.del, 
> _d8_nrm.cfs, _d7.frq, _89.tim, _d8.fdt, _d7.si, _d8.tip, _d8_nrm.cfe, 
> segments_6, _d8.fdx, _89.tip]
> [junit4:junit4]   2> 38327 T3647 C153 P62559 
> oasc.SolrDeletionPolicy.updateCommits newest commit = 6
> [junit4:junit4]   2> 38335 T3647 C153 P62559 oass.SolrIndexSearcher.<init> 
> Opening Searcher@4a191c9 main
> [junit4:junit4]   2> 38336 T3647 C153 P62559 oasu.DirectUpdateHandler2.commit 
> end_commit_flush
> [junit4:junit4]   2> 38336 T3714 oasc.SolrCore.registerSearcher [collection1] 
> Registered new searcher Searcher@4a191c9 
> main{StandardDirectoryReader(segments_6:1017:nrt _89(5.0):C1302/24 
> _d7(5.0):C850/1 _d8(5.0):C4)}
> [junit4:junit4]   2> 38337 T3647 C153 P62559 /update 
> {waitSearcher=true&commit=true&wt=javabin&commit_end_point=true&softCommit=false&version=2}
>  {commit=} 0 92
> [junit4:junit4]   2> 38337 T1872 C152 P55793 /update 
> {waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} 
> {commit=} 0 427
> [junit4:junit4]   2> 38339 T1870 C152 P55793 REQ /select 
> {distrib=false&wt=javabin&q=*:*&version=2} hits=2134 status=0 QTime=1 
> [junit4:junit4]   2> 38341 T3680 C153 P62559 REQ /select 
> {distrib=false&wt=javabin&q=*:*&version=2} hits=2131 status=0 QTime=1 
> [junit4:junit4]   2> 38343 T1870 C152 P55793 REQ /select 
> {distrib=false&wt=javabin&q=*:*&version=2} hits=2134 status=0 QTime=0 
> [junit4:junit4]   2> 38345 T3680 C153 P62559 REQ /select 
> {distrib=false&wt=javabin&q=*:*&version=2} hits=2131 status=0 QTime=0 
> [junit4:junit4]   2> 38346 T1849 oazs.PrepRequestProcessor.pRequest Processed 
> session termination for sessionid: 0x138bf15110c0006
> [junit4:junit4]   2> 38347 T1898 oaz.ClientCnxn$EventThread.run EventThread 
> shut down
> [junit4:junit4]   2> 38347 T1846 oazs.NIOServerCnxn.closeSock Closed socket 
> connection for client /140.211.11.196:31128 which had sessionid 
> 0x138bf15110c0006
> [junit4:junit4]   2> 38347 T1844 oaz.ZooKeeper.close Session: 
> 0x138bf15110c0006 closed
> [junit4:junit4]   2> 38350 T1849 oazs.PrepRequestProcessor.pRequest Processed 
> session termination for sessionid: 0x138bf15110c0005
> [junit4:junit4]   2> 38351 T1894 oaz.ClientCnxn$EventThread.run EventThread 
> shut down
> [junit4:junit4]   2> 38351 T1846 oazs.NIOServerCnxn.closeSock Closed socket 
> connection for client /140.211.11.196:58254 which had sessionid 
> 0x138bf15110c0005
> [junit4:junit4]   2> 38351 T1844 oaz.ZooKeeper.close Session: 
> 0x138bf15110c0005 closed
> [junit4:junit4]   2> 38352 T1849 oazs.PrepRequestProcessor.run 
> PrepRequestProcessor exited loop!
> [junit4:junit4]   2> 38352 T1848 oazs.SyncRequestProcessor.run 
> SyncRequestProcessor exited!
> [junit4:junit4]   2> 38353 T1844 oazs.FinalRequestProcessor.shutdown shutdown 
> of request processor complete
> [junit4:junit4]   2> 38415 T1844 oazs.NIOServerCnxn.closeSock Closed socket 
> connection for client /140.211.11.196:36982 which had sessionid 
> 0x138bf15110c0003
> [junit4:junit4]   2> 38415 T1844 oazs.NIOServerCnxn.closeSock Closed socket 
> connection for client /140.211.11.196:13596 which had sessionid 
> 0x138bf15110c0002
> [junit4:junit4]   2> 38416 T1877 oaz.ClientCnxn$SendThread.run Unable to read 
> additional data from server sessionid 0x138bf15110c0003, likely server has 
> closed socket, closing socket connection and attempting reconnect
> [junit4:junit4]   2> 38416 T1863 oaz.ClientCnxn$SendThread.run Unable to read 
> additional data from server sessionid 0x138bf15110c0002, likely server has 
> closed socket, closing socket connection and attempting reconnect
> [junit4:junit4]   2> 38416 T1844 oazs.NIOServerCnxn.closeSock Closed socket 
> connection for client /140.211.11.196:54705 which had sessionid 
> 0x138bf15110c0007
> [junit4:junit4]   2> 38416 T3628 oaz.ClientCnxn$SendThread.run Unable to read 
> additional data from server sessionid 0x138bf15110c0007, likely server has 
> closed socket, closing socket connection and attempting reconnect
> [junit4:junit4]   2> 38416 T1846 oazs.NIOServerCnxn$Factory.run NIOServerCnxn 
> factory exited run method
> [junit4:junit4]   2> 38417 T1844 oazs.FinalRequestProcessor.shutdown shutdown 
> of request processor complete
> [junit4:junit4]   2> 38417 T1844 oasc.ChaosMonkey.monkeyLog monkey: stop 
> shard! 26525
> [junit4:junit4]   2> 38417 T1844 oasc.CoreContainer.shutdown Shutting down 
> CoreContainer instance=440684931
> [junit4:junit4]   2> 38417 T1844 oasc.SolrCore.close [collection1]  CLOSING 
> SolrCore org.apache.solr.core.SolrCore@74aaadcc
> [junit4:junit4]   2> 38418 T1844 oasc.SolrCore.closeSearcher [collection1] 
> Closing main searcher on request.
> [junit4:junit4]   2> 38419 T1844 oasu.DirectUpdateHandler2.close closing 
> DirectUpdateHandler2{commits=1,autocommits=0,soft 
> autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=4131,cumulative_deletesById=1997,cumulative_deletesByQuery=0,cumulative_errors=0}
> [junit4:junit4]   2> 38457 T1844 C151 P26525 oasc.SolrDeletionPolicy.onCommit 
> SolrDeletionPolicy.onCommit: commits:num=2
> [junit4:junit4]   2>          
> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-trunk/checkout/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-1343235625223/control/data/index
>  lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@72e46ea8; 
> maxCacheMB=48.0 
> maxMergeSizeMB=4.0),segFN=segments_2,generation=2,filenames=[_d0.frq, 
> _6g.tip, _c9.fnm, _6g.tim, _d0.fnm, _cy.fdt, _c9.fdx, _c9.frq, _cy.prx, 
> _6g.fnm, _cz.tip, _6g.frq, _cz.tim, _cy.tim, _6g.prx, _cy.fnm, _d1.tim, 
> _c9.prx, _d1_nrm.cfe, _d1.tip, _d0.fdt, _cy.frq, _cz_nrm.cfs, _cy_nrm.cfs, 
> _d0.si, _d0.fdx, _6g.si, _cz.fnm, _d1_nrm.cfs, _d3.frq, _d3.fnm, _c9.fdt, 
> _d3.prx, _c9_nrm.cfs, _cy.fdx, _d0.prx, _d2.tip, _cy_nrm.cfe, _c9_nrm.cfe, 
> _d1.prx, _cz.frq, _d3_nrm.cfe, _d3.si, _d0_nrm.cfs, _cz.fdx, _d2.tim, _d2.si, 
> _d3_nrm.cfs, _c9_1.del, _cz.si, _d2.fnm, _d3.tip, _d0.tip, _d0_nrm.cfe, 
> _d3.tim, _d1.si, _cy.tip, _d0.tim, _d2.frq, _6g.fdt, _6g.fdx, _6g_nrm.cfe, 
> _c9.tim, _c9.si, _d1.fdx, _cz_nrm.cfe, _d1.frq, _d2.prx, _d1.fdt, 
> _d2_nrm.cfs, _6g_1.del, _d2.fdt, _6g_nrm.cfs, _d3.fdx, _cz.fdt, _d3.fdt, 
> _cz.prx, segments_2, _cz_1.del, _cy.si, _d2_nrm.cfe, _d1.fnm, _d2.fdx, 
> _c9.tip]
> [junit4:junit4]   2>          
> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-trunk/checkout/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-1343235625223/control/data/index
>  lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@72e46ea8; 
> maxCacheMB=48.0 
> maxMergeSizeMB=4.0),segFN=segments_3,generation=3,filenames=[_d4.prx, 
> _d4.tim, _d4.fdx, _d4.si, _d4.tip, _d4.fnm, _d4_nrm.cfe, _d4_nrm.cfs, 
> segments_3, _d4.fdt, _d4.frq]
> [junit4:junit4]   2> 38457 T1844 C151 P26525 
> oasc.SolrDeletionPolicy.updateCommits newest commit = 3
> [junit4:junit4]   2> 38517 T3629 oascc.ConnectionManager.process Watcher 
> org.apache.solr.common.cloud.ConnectionManager@6ad73088 
> name:ZooKeeperConnection Watcher:127.0.0.1:56293/solr got event WatchedEvent 
> state:Disconnected type:None path:null path:null type:None
> [junit4:junit4]   2> 38517 T1878 oascc.ConnectionManager.process Watcher 
> org.apache.solr.common.cloud.ConnectionManager@4aa51c9d 
> name:ZooKeeperConnection Watcher:127.0.0.1:56293/solr got event WatchedEvent 
> state:Disconnected type:None path:null path:null type:None
> [junit4:junit4]   2> 38517 T1864 oascc.ConnectionManager.process Watcher 
> org.apache.solr.common.cloud.ConnectionManager@7b437ced 
> name:ZooKeeperConnection Watcher:127.0.0.1:56293/solr got event WatchedEvent 
> state:Disconnected type:None path:null path:null type:None
> [junit4:junit4]   2> 38517 T1844 oaz.ZooKeeper.close Session: 
> 0x138bf15110c0002 closed
> [junit4:junit4]   2> 38517 T1864 oaz.ClientCnxn$EventThread.run EventThread 
> shut down
> [junit4:junit4]   2> 38517 T1866 oasc.OverseerCollectionProcessor.run WARNING 
> Overseer cannot talk to ZK
> [junit4:junit4]   2> 38518 T1844 oejsh.ContextHandler.doStop stopped 
> o.e.j.s.ServletContextHandler{/solr,null}
> [junit4:junit4]   2> 38574 T1844 oasc.ChaosMonkey.monkeyLog monkey: stop 
> shard! 55793
> [junit4:junit4]   2> 38574 T1844 oasc.CoreContainer.shutdown Shutting down 
> CoreContainer instance=1858535252
> [junit4:junit4]   2> 38574 T1844 oasc.SolrCore.close [collection1]  CLOSING 
> SolrCore org.apache.solr.core.SolrCore@63829d01
> [junit4:junit4]   2> 38575 T1844 oasc.SolrCore.closeSearcher [collection1] 
> Closing main searcher on request.
> [junit4:junit4]   2> 38576 T1844 oasu.DirectUpdateHandler2.close closing 
> DirectUpdateHandler2{commits=4,autocommits=0,soft 
> autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=4131,cumulative_deletesById=1997,cumulative_deletesByQuery=0,cumulative_errors=0}
> [junit4:junit4]   2> 38824 T1865 oasc.Overseer$CloudStateUpdater.amILeader 
> WARNING  org.apache.zookeeper.KeeperException$SessionExpiredException: 
> KeeperErrorCode = Session expired for /overseer_elect/leader
> [junit4:junit4]   2>  at 
> org.apache.zookeeper.KeeperException.create(KeeperException.java:118)
> [junit4:junit4]   2>  at 
> org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
> [junit4:junit4]   2>  at 
> org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:927)
> [junit4:junit4]   2>  at 
> org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:289)
> [junit4:junit4]   2>  at 
> org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:286)
> [junit4:junit4]   2>  at 
> org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:67)
> [junit4:junit4]   2>  at 
> org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:286)
> [junit4:junit4]   2>  at 
> org.apache.solr.cloud.Overseer$CloudStateUpdater.amILeader(Overseer.java:187)
> [junit4:junit4]   2>  at 
> org.apache.solr.cloud.Overseer$CloudStateUpdater.run(Overseer.java:113)
> [junit4:junit4]   2>  at java.lang.Thread.run(Thread.java:679)
> [junit4:junit4]   2> 
> [junit4:junit4]   2> 38824 T1865 oasc.Overseer$CloudStateUpdater.amILeader 
> According to ZK I (id=88030289934942210-127.0.0.1:26525_solr-n_0000000000) am 
> no longer a leader.
> [junit4:junit4]   2> 39959 T1877 oaz.ClientCnxn$SendThread.startConnect 
> Opening socket connection to server localhost/127.0.0.1:56293
> [junit4:junit4]   2> 40236 T3628 oaz.ClientCnxn$SendThread.startConnect 
> Opening socket connection to server localhost/127.0.0.1:56293
> [junit4:junit4]   2> 40782 T1847 oazs.SessionTrackerImpl.run 
> SessionTrackerImpl exited loop!
> [junit4:junit4]   2> 50518 T3628 oaz.ClientCnxn$SendThread.run Client session 
> timed out, have not heard from server in 12001ms for sessionid 
> 0x138bf15110c0007, closing socket connection and attempting reconnect
> [junit4:junit4]   2> 50619 T1878 oaz.ClientCnxn$EventThread.run EventThread 
> shut down
> [junit4:junit4]   2> 50619 T1844 oaz.ZooKeeper.close Session: 
> 0x138bf15110c0003 closed
> [junit4:junit4]   2> 50620 T1844 oejsh.ContextHandler.doStop stopped 
> o.e.j.s.ServletContextHandler{/solr,null}
> [junit4:junit4]   2> 50672 T1844 oasc.ChaosMonkey.monkeyLog monkey: stop 
> shard! 62559
> [junit4:junit4]   2> 50672 T1844 oasc.CoreContainer.shutdown Shutting down 
> CoreContainer instance=149090166
> [junit4:junit4]   2> 50672 T1844 oasc.RecoveryStrategy.close WARNING Stopping 
> recovery for core collection1 zkNodeName=127.0.0.1:62559_solr_collection1
> [junit4:junit4]   2> 50672 T1844 oasc.SolrCore.close [collection1]  CLOSING 
> SolrCore org.apache.solr.core.SolrCore@17c7a8f3
> [junit4:junit4]   2> 50673 T1844 oasc.SolrCore.closeSearcher [collection1] 
> Closing main searcher on request.
> [junit4:junit4]   2> 50674 T1844 oasu.DirectUpdateHandler2.close closing 
> DirectUpdateHandler2{commits=5,autocommits=0,soft 
> autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=2286,cumulative_deletesById=1061,cumulative_deletesByQuery=0,cumulative_errors=0}
> [junit4:junit4]   2> 50675 T1844 oasc.RecoveryStrategy.close WARNING Stopping 
> recovery for core collection1 zkNodeName=127.0.0.1:62559_solr_collection1
> [junit4:junit4]   2> 52337 T3628 oaz.ClientCnxn$SendThread.startConnect 
> Opening socket connection to server localhost/127.0.0.1:56293
> [junit4:junit4]   2> 62721 T3629 oaz.ClientCnxn$EventThread.run EventThread 
> shut down
> [junit4:junit4]   2> 62721 T1844 oaz.ZooKeeper.close Session: 
> 0x138bf15110c0007 closed
> [junit4:junit4]   2> 62722 T1844 oejsh.ContextHandler.doStop stopped 
> o.e.j.s.ServletContextHandler{/solr,null}
> [junit4:junit4]   2> 62803 T1844 oas.SolrTestCaseJ4.tearDown ###Ending 
> testDistribSearch
> [junit4:junit4]   2> NOTE: reproduce with: ant test  
> -Dtestcase=RecoveryZkTest -Dtests.method=testDistribSearch 
> -Dtests.seed=464A264BD38912FE -Dtests.multiplier=3 -Dtests.slow=true 
> -Dtests.locale=ca -Dtests.timezone=Asia/Bangkok -Dtests.file.encoding=UTF-8
> [junit4:junit4]   2>
> [junit4:junit4]    > (@AfterClass output)
> [junit4:junit4]   2> 62838 T1844 oas.SolrTestCaseJ4.deleteCore ###deleteCore
> [junit4:junit4]   2> NOTE: test params are: codec=Asserting, 
> sim=RandomSimilarityProvider(queryNorm=true,coord=false): {}, locale=ca, 
> timezone=Asia/Bangkok
> [junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Sun Microsystems Inc. 
> 1.6.0_32 (64-bit)/cpus=16,threads=64,free=187258648,total=285278208
> [junit4:junit4]   2> NOTE: All tests run in this JVM: [DateMathParserTest, 
> TermsComponentTest, NotRequiredUniqueKeyTest, NoCacheHeaderTest, 
> TestXIncludeConfig, AnalysisAfterCoreReloadTest, NumericFieldsTest, 
> UpdateRequestProcessorFactoryTest, DOMUtilTest, TestSuggestSpellingConverter, 
> TestValueSourceCache, TestReload, JSONWriterTest, TestMultiCoreConfBootstrap, 
> DirectSolrConnectionTest, TestQuerySenderNoQuery, TestCollationField, 
> SolrCoreTest, DistributedTermsComponentTest, TestCoreContainer, 
> BinaryUpdateRequestHandlerTest, FileBasedSpellCheckerTest, 
> SortByFunctionTest, SuggesterFSTTest, RequestHandlersTest, 
> PreAnalyzedFieldTest, SolrCmdDistributorTest, TestDistributedGrouping, 
> TestCharFilters, TestSurroundQueryParser, SolrCoreCheckLockOnStartupTest, 
> XmlUpdateRequestHandlerTest, ZkSolrClientTest, BadComponentTest, 
> TestLMDirichletSimilarityFactory, PeerSyncTest, LoggingHandlerTest, 
> MinimalSchemaTest, TestRecovery, TestNumberUtils, CopyFieldTest, 
> UUIDFieldTest, TestJmxIntegration, BasicDistributedZkTest, 
> FullSolrCloudDistribCmdsTest, TestSearchPerf, TestPseudoReturnFields, 
> TestSort, AlternateDirectoryTest, LeaderElectionIntegrationTest, 
> IndexSchemaRuntimeFieldTest, TestLFUCache, RecoveryZkTest]
> [junit4:junit4]   2> 
> [junit4:junit4] Completed on J0 in 62.85s, 1 test, 1 failure <<< FAILURES!
> [...truncated 413 lines...]
> 
> [...truncated 36526 lines...]
> 
> [...truncated 36526 lines...]
> 
> [...truncated 36526 lines...]
> 
> [...truncated 36526 lines...]
> 
> [...truncated 36511 lines...]
> BUILD FAILED
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-trunk/checkout/build.xml:29:
>  The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-trunk/checkout/solr/build.xml:144:
>  The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-trunk/checkout/lucene/module-build.xml:62:
>  The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-trunk/checkout/lucene/common-build.xml:1094:
>  The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-trunk/checkout/lucene/common-build.xml:789:
>  There were test failures: 222 suites, 896 tests, 1 failure, 5 ignored
> 
> Total time: 27 minutes 40 seconds
> Build step 'Execute shell' marked build as failure
> Archiving artifacts
> Recording test results
> Publishing Clover coverage report...
> No Clover report will be published due to a Build Failure
> Email was triggered for: Failure
> Sending email for trigger: Failure
> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [email protected]
> For additional commands, e-mail: [email protected]

- Mark Miller
lucidimagination.com












---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to