See <http://sierranevada.servebeer.com:8080/job/slow-io-beasting/4952/>

------------------------------------------
[...truncated 14384 lines...]
[junit4:junit4]   2> 869864 T268 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 
51282
[junit4:junit4]   2> 869864 T268 oasc.CoreContainer.shutdown Shutting down 
CoreContainer instance=27111714
[junit4:junit4]   2> 869864 T268 oasc.SolrCore.close [collection1]  CLOSING 
SolrCore org.apache.solr.core.SolrCore@1654a50
[junit4:junit4]   2> 869871 T268 oasu.DirectUpdateHandler2.close closing 
DirectUpdateHandler2{commits=1,autocommits=0,soft 
autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4]   2> 869872 T268 oasc.SolrCore.decrefSolrCoreState Closing 
SolrCoreState
[junit4:junit4]   2> 869872 T268 oasu.DefaultSolrCoreState.closeIndexWriter 
SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4]   2> 869872 T268 oasu.DefaultSolrCoreState.closeIndexWriter 
closing IndexWriter with IndexWriterCloser
[junit4:junit4]   2> 869874 T268 oasc.SolrCore.closeSearcher [collection1] 
Closing main searcher on request.
[junit4:junit4]   2> 870015 T329 oaz.ClientCnxn$SendThread.run WARNING Session 
0x13aa447af930005 for server null, unexpected error, closing socket connection 
and attempting reconnect java.net.ConnectException: Connection refused: no 
further information
[junit4:junit4]   2>    at sun.nio.ch.SocketChannelImpl.checkConnect(Native 
Method)
[junit4:junit4]   2>    at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
[junit4:junit4]   2>    at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1143)
[junit4:junit4]   2> 
[junit4:junit4]   2> 870015 T342 oaz.ClientCnxn$SendThread.run WARNING Session 
0x13aa447af930006 for server null, unexpected error, closing socket connection 
and attempting reconnect java.net.ConnectException: Connection refused: no 
further information
[junit4:junit4]   2>    at sun.nio.ch.SocketChannelImpl.checkConnect(Native 
Method)
[junit4:junit4]   2>    at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
[junit4:junit4]   2>    at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1143)
[junit4:junit4]   2> 
[junit4:junit4]   2> 871329 T317 oaz.ClientCnxn$SendThread.startConnect Opening 
socket connection to server 127.0.0.1/127.0.0.1:50184
[junit4:junit4]   2> 871423 T329 oaz.ClientCnxn$SendThread.startConnect Opening 
socket connection to server 127.0.0.1/127.0.0.1:50184
[junit4:junit4]   2> 871548 T342 oaz.ClientCnxn$SendThread.startConnect Opening 
socket connection to server 127.0.0.1/127.0.0.1:50184
[junit4:junit4]   2> 872421 T329 oaz.ClientCnxn$SendThread.run WARNING Session 
0x13aa447af930005 for server null, unexpected error, closing socket connection 
and attempting reconnect java.net.ConnectException: Connection refused: no 
further information
[junit4:junit4]   2>    at sun.nio.ch.SocketChannelImpl.checkConnect(Native 
Method)
[junit4:junit4]   2>    at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
[junit4:junit4]   2>    at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1143)
[junit4:junit4]   2> 
[junit4:junit4]   2> 872437 T318 oaz.ClientCnxn$EventThread.run EventThread 
shut down
[junit4:junit4]   2> 872437 T268 oaz.ZooKeeper.close Session: 0x13aa447af930004 
closed
[junit4:junit4]   2> 872452 T268 oejsh.ContextHandler.doStop stopped 
o.e.j.s.ServletContextHandler{/solr,null}
[junit4:junit4]   2> 872503 T268 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 
51325
[junit4:junit4]   2> 872503 T268 oasc.CoreContainer.shutdown Shutting down 
CoreContainer instance=133970
[junit4:junit4]   2> 872503 T268 oasc.SolrCore.close [collection1]  CLOSING 
SolrCore org.apache.solr.core.SolrCore@bf7795
[junit4:junit4]   2> 872509 T268 oasu.DirectUpdateHandler2.close closing 
DirectUpdateHandler2{commits=0,autocommits=0,soft 
autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4]   2> 872509 T268 oasc.SolrCore.decrefSolrCoreState Closing 
SolrCoreState
[junit4:junit4]   2> 872510 T268 oasu.DefaultSolrCoreState.closeIndexWriter 
SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4]   2> 872510 T268 oasu.DefaultSolrCoreState.closeIndexWriter 
closing IndexWriter with IndexWriterCloser
[junit4:junit4]   2> 872510 T268 oasc.SolrCore.closeSearcher [collection1] 
Closing main searcher on request.
[junit4:junit4]   2> 872514 T342 oaz.ClientCnxn$SendThread.run WARNING Session 
0x13aa447af930006 for server null, unexpected error, closing socket connection 
and attempting reconnect java.net.ConnectException: Connection refused: no 
further information
[junit4:junit4]   2>    at sun.nio.ch.SocketChannelImpl.checkConnect(Native 
Method)
[junit4:junit4]   2>    at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
[junit4:junit4]   2>    at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1143)
[junit4:junit4]   2> 
[junit4:junit4]   2> 873783 T342 oaz.ClientCnxn$SendThread.startConnect Opening 
socket connection to server 127.0.0.1/127.0.0.1:50184
[junit4:junit4]   2> 874251 T329 oaz.ClientCnxn$SendThread.startConnect Opening 
socket connection to server 127.0.0.1/127.0.0.1:50184
[junit4:junit4]   2> 874766 T342 oaz.ClientCnxn$SendThread.run WARNING Session 
0x13aa447af930006 for server null, unexpected error, closing socket connection 
and attempting reconnect java.net.ConnectException: Connection refused: no 
further information
[junit4:junit4]   2>    at sun.nio.ch.SocketChannelImpl.checkConnect(Native 
Method)
[junit4:junit4]   2>    at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
[junit4:junit4]   2>    at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1143)
[junit4:junit4]   2> 
[junit4:junit4]   2> 875360 T330 oaz.ClientCnxn$EventThread.run EventThread 
shut down
[junit4:junit4]   2> 875360 T268 oaz.ZooKeeper.close Session: 0x13aa447af930005 
closed
[junit4:junit4]   2> 875376 T268 oejsh.ContextHandler.doStop stopped 
o.e.j.s.ServletContextHandler{/solr,null}
[junit4:junit4]   2> 875427 T268 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 
51731
[junit4:junit4]   2> 875427 T268 oasc.CoreContainer.shutdown Shutting down 
CoreContainer instance=8748903
[junit4:junit4]   2> 875427 T268 oasc.SolrCore.close [collection1]  CLOSING 
SolrCore org.apache.solr.core.SolrCore@ac93db
[junit4:junit4]   2> 875458 T268 oasu.DirectUpdateHandler2.close closing 
DirectUpdateHandler2{commits=1,autocommits=0,soft 
autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4]   2> 875458 T268 oasc.SolrCore.decrefSolrCoreState Closing 
SolrCoreState
[junit4:junit4]   2> 875458 T268 oasu.DefaultSolrCoreState.closeIndexWriter 
SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4]   2> 875458 T268 oasu.DefaultSolrCoreState.closeIndexWriter 
closing IndexWriter with IndexWriterCloser
[junit4:junit4]   2> 875458 T268 oasc.SolrCore.closeSearcher [collection1] 
Closing main searcher on request.
[junit4:junit4]   2> 876004 T342 oaz.ClientCnxn$SendThread.startConnect Opening 
socket connection to server 127.0.0.1/127.0.0.1:50184
[junit4:junit4]   2> 877111 T343 oaz.ClientCnxn$EventThread.run EventThread 
shut down
[junit4:junit4]   2> 877111 T268 oaz.ZooKeeper.close Session: 0x13aa447af930006 
closed
[junit4:junit4]   2> 877127 T268 oejsh.ContextHandler.doStop stopped 
o.e.j.s.ServletContextHandler{/solr,null}
[junit4:junit4]   2> 877412 T268 oas.SolrTestCaseJ4.tearDown ###Ending 
testDistribSearch
[junit4:junit4]   2> NOTE: reproduce with: ant test  
-Dtestcase=FullSolrCloudDistribCmdsTest -Dtests.method=testDistribSearch 
-Dtests.seed=3E9DE1AB4AC5B630 -Dtests.slow=true -Dtests.locale=de 
-Dtests.timezone=Antarctica/Syowa -Dtests.file.encoding=ISO-8859-1
[junit4:junit4] FAILURE  877s J4 | 
FullSolrCloudDistribCmdsTest.testDistribSearch <<<
[junit4:junit4]    > Throwable #1: java.lang.AssertionError: There are still 
nodes recoverying
[junit4:junit4]    >    at 
__randomizedtesting.SeedInfo.seed([3E9DE1AB4AC5B630:BF7B6FB33D9AD60C]:0)
[junit4:junit4]    >    at org.junit.Assert.fail(Assert.java:93)
[junit4:junit4]    >    at 
org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:154)
[junit4:junit4]    >    at 
org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:115)
[junit4:junit4]    >    at 
org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:110)
[junit4:junit4]    >    at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:553)
[junit4:junit4]    >    at 
org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.doTest(FullSolrCloudDistribCmdsTest.java:65)
[junit4:junit4]    >    at 
org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:695)
[junit4:junit4]    >    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native 
Method)
[junit4:junit4]    >    at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
[junit4:junit4]    >    at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[junit4:junit4]    >    at java.lang.reflect.Method.invoke(Method.java:597)
[junit4:junit4]    >    at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
[junit4:junit4]    >    at 
com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
[junit4:junit4]    >    at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
[junit4:junit4]    >    at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
[junit4:junit4]    >    at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
[junit4:junit4]    >    at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4]    >    at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
[junit4:junit4]    >    at 
org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
[junit4:junit4]    >    at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4:junit4]    >    at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4]    >    at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4:junit4]    >    at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4]    >    at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4]    >    at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]    >    at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4]    >    at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
[junit4:junit4]    >    at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
[junit4:junit4]    >    at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
[junit4:junit4]    >    at 
com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
[junit4:junit4]    >    at 
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
[junit4:junit4]    >    at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
[junit4:junit4]    >    at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]    >    at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4]    >    at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4:junit4]    >    at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4:junit4]    >    at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4]    >    at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4]    >    at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4]    >    at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]    >    at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
[junit4:junit4]    >    at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4]    >    at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4]    >    at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4:junit4]    >    at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]    >    at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4]    >    at java.lang.Thread.run(Thread.java:662)
[junit4:junit4]   2> 877428 T268 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4]   2> 1000122 T268 oas.SolrTestCaseJ4.endTrackingSearchers 
SEVERE ERROR: SolrIndexSearcher opens=7 closes=6
[junit4:junit4]   2> 1000122 T267 ccr.ThreadLeakControl.checkThreadLeaks 
WARNING Will linger awaiting termination of 1 leaked thread(s).
[junit4:junit4]   2> 1020277 T267 ccr.ThreadLeakControl.checkThreadLeaks SEVERE 
1 thread leaked from SUITE scope at 
org.apache.solr.cloud.FullSolrCloudDistribCmdsTest: 
[junit4:junit4]   2>       1) Thread[id=305, 
name=searcherExecutor-139-thread-1, state=WAITING, 
group=TGRP-FullSolrCloudDistribCmdsTest]
[junit4:junit4]   2>            at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2>            at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
[junit4:junit4]   2>            at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
[junit4:junit4]   2>            at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4]   2>            at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
[junit4:junit4]   2>            at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
[junit4:junit4]   2>            at java.lang.Thread.run(Thread.java:662)
[junit4:junit4]   2> 1020277 T267 ccr.ThreadLeakControl.tryToInterruptAll 
Starting to interrupt leaked threads:
[junit4:junit4]   2>       1) Thread[id=305, 
name=searcherExecutor-139-thread-1, state=WAITING, 
group=TGRP-FullSolrCloudDistribCmdsTest]
[junit4:junit4]   2> 1023366 T267 ccr.ThreadLeakControl.tryToInterruptAll 
SEVERE There are still zombie threads that couldn't be terminated:
[junit4:junit4]   2>       1) Thread[id=305, 
name=searcherExecutor-139-thread-1, state=WAITING, 
group=TGRP-FullSolrCloudDistribCmdsTest]
[junit4:junit4]   2>            at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2>            at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
[junit4:junit4]   2>            at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
[junit4:junit4]   2>            at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4]   2>            at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
[junit4:junit4]   2>            at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
[junit4:junit4]   2>            at java.lang.Thread.run(Thread.java:662)
[junit4:junit4]   2> NOTE: test params are: codec=Lucene40, 
sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, locale=de, 
timezone=Antarctica/Syowa
[junit4:junit4]   2> NOTE: Windows Vista 6.0 x86/Sun Microsystems Inc. 1.6.0_23 
(32-bit)/cpus=4,threads=2,free=1933040,total=40804352
[junit4:junit4]   2> NOTE: All tests run in this JVM: [StatsComponentTest, 
TestStressReorder, TestReload, URLClassifyProcessorTest, ScriptEngineTest, 
SolrCmdDistributorTest, FullSolrCloudDistribCmdsTest]
[junit4:junit4]   2> NOTE: reproduce with: ant test  
-Dtestcase=FullSolrCloudDistribCmdsTest -Dtests.seed=3E9DE1AB4AC5B630 
-Dtests.slow=true -Dtests.locale=de -Dtests.timezone=Antarctica/Syowa 
-Dtests.file.encoding=ISO-8859-1
[junit4:junit4] ERROR   0.00s J4 | FullSolrCloudDistribCmdsTest (suite) <<<
[junit4:junit4]    > Throwable #1: java.lang.AssertionError: ERROR: 
SolrIndexSearcher opens=7 closes=6
[junit4:junit4]    >    at 
__randomizedtesting.SeedInfo.seed([3E9DE1AB4AC5B630]:0)
[junit4:junit4]    >    at org.junit.Assert.fail(Assert.java:93)
[junit4:junit4]    >    at 
org.apache.solr.SolrTestCaseJ4.endTrackingSearchers(SolrTestCaseJ4.java:242)
[junit4:junit4]    >    at 
org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:97)
[junit4:junit4]    >    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native 
Method)
[junit4:junit4]    >    at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
[junit4:junit4]    >    at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[junit4:junit4]    >    at java.lang.reflect.Method.invoke(Method.java:597)
[junit4:junit4]    >    at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
[junit4:junit4]    >    at 
com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
[junit4:junit4]    >    at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:700)
[junit4:junit4]    >    at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]    >    at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4]    >    at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4:junit4]    >    at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4:junit4]    >    at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4]    >    at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4]    >    at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4]    >    at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]    >    at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
[junit4:junit4]    >    at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4]    >    at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4]    >    at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4:junit4]    >    at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]    >    at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4]    >    at java.lang.Thread.run(Thread.java:662)
[junit4:junit4]    > Throwable #2: 
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE 
scope at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest: 
[junit4:junit4]    >    1) Thread[id=305, name=searcherExecutor-139-thread-1, 
state=WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]
[junit4:junit4]    >         at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]    >         at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
[junit4:junit4]    >         at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
[junit4:junit4]    >         at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4]    >         at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
[junit4:junit4]    >         at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:662)
[junit4:junit4]    >    at 
__randomizedtesting.SeedInfo.seed([3E9DE1AB4AC5B630]:0)
[junit4:junit4]    > Throwable #3: 
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie 
threads that couldn't be terminated:
[junit4:junit4]    >    1) Thread[id=305, name=searcherExecutor-139-thread-1, 
state=WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]
[junit4:junit4]    >         at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]    >         at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
[junit4:junit4]    >         at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
[junit4:junit4]    >         at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4]    >         at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
[junit4:junit4]    >         at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:662)
[junit4:junit4]    >    at 
__randomizedtesting.SeedInfo.seed([3E9DE1AB4AC5B630]:0)
[junit4:junit4] Completed on J4 in 1023.43s, 1 test, 2 failures, 2 errors <<< 
FAILURES!
[junit4:junit4] 
[junit4:junit4] Suite: org.apache.solr.handler.XsltUpdateRequestHandlerTest
[junit4:junit4] Completed on J4 in 0.02s, 2 tests, 2 skipped
[junit4:junit4] 
[junit4:junit4] Suite: org.apache.solr.internal.csv.ExtendedBufferedReaderTest
[junit4:junit4] Completed on J4 in 0.02s, 8 tests
[junit4:junit4] 
[junit4:junit4] Suite: org.apache.solr.schema.TestBinaryField
[junit4:junit4] Completed on J4 in 0.02s, 1 test, 1 skipped
[junit4:junit4] 
[junit4:junit4] Suite: org.apache.solr.update.TestIndexingPerformance
[junit4:junit4] Completed on J4 in 0.00s, 1 test, 1 skipped
[junit4:junit4] 
[junit4:junit4] Suite: org.apache.solr.handler.PingRequestHandlerTest
[junit4:junit4] Completed on J4 in 0.00s, 5 tests, 5 skipped
[junit4:junit4] 
[junit4:junit4] Suite: org.apache.solr.SampleTest
[junit4:junit4] Completed on J4 in 0.00s, 2 tests, 2 skipped
[junit4:junit4] 
[junit4:junit4] Suite: org.apache.solr.response.TestCSVResponseWriter
[junit4:junit4] Completed on J4 in 0.00s, 1 test, 1 skipped
[junit4:junit4] 
[junit4:junit4] Suite: org.apache.solr.handler.admin.SystemInfoHandlerTest
[junit4:junit4] Completed on J4 in 0.00s, 1 test, 1 skipped
[junit4:junit4] 
[junit4:junit4] Suite: org.apache.solr.internal.csv.writer.CSVFieldTest
[junit4:junit4] Completed on J4 in 0.00s, 2 tests
[junit4:junit4] 
[junit4:junit4] Suite: org.apache.solr.schema.MultiTermTest
[junit4:junit4] Completed on J4 in 0.00s, 3 tests, 3 skipped
[junit4:junit4] 
[junit4:junit4] 
[junit4:junit4] Tests with failures:
[junit4:junit4]   - 
org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.testDistribSearch
[junit4:junit4]   - org.apache.solr.cloud.FullSolrCloudDistribCmdsTest (suite)
[junit4:junit4] 
[junit4:junit4] 
[junit4:junit4] JVM J0:     2.89 ..   269.82 =   266.93s
[junit4:junit4] JVM J1:     2.64 ..   272.23 =   269.59s
[junit4:junit4] JVM J2:     2.87 ..   278.56 =   275.69s
[junit4:junit4] JVM J3:     2.64 ..   270.81 =   268.18s
[junit4:junit4] JVM J4:     2.64 ..  1168.01 =  1165.38s
[junit4:junit4] JVM J5:     2.64 ..   269.82 =   267.18s
[junit4:junit4] JVM J6:     2.62 ..   340.69 =   338.07s
[junit4:junit4] JVM J7:     2.62 ..   285.62 =   283.00s
[junit4:junit4] Execution time total: 19 minutes 28 seconds
[junit4:junit4] Tests summary: 233 suites, 965 tests, 3 suite-level errors, 1 
failure, 28 ignored (7 assumptions)

BUILD FAILED
<http://sierranevada.servebeer.com:8080/job/slow-io-beasting/ws/build.xml>:39: 
The following error occurred while executing this line:
<http://sierranevada.servebeer.com:8080/job/slow-io-beasting/ws/solr/build.xml>:178:
 The following error occurred while executing this line:
<http://sierranevada.servebeer.com:8080/job/slow-io-beasting/ws/lucene/module-build.xml>:61:
 The following error occurred while executing this line:
<http://sierranevada.servebeer.com:8080/job/slow-io-beasting/ws/lucene/common-build.xml>:1159:
 The following error occurred while executing this line:
<http://sierranevada.servebeer.com:8080/job/slow-io-beasting/ws/lucene/common-build.xml>:827:
 There were test failures: 233 suites, 965 tests, 3 suite-level errors, 1 
failure, 28 ignored (7 assumptions)

Total time: 27 minutes 31 seconds
Build step 'Invoke Ant' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Reply via email to