See <http://sierranevada.servebeer.com:8080/job/slow-io-beasting/5023/>
------------------------------------------
[...truncated 13977 lines...]
[junit4:junit4] 2> 686941 T216 oaz.ClientCnxn$SendThread.run Unable to read
additional data from server sessionid 0x13aa9d8483b0002, likely server has
closed socket, closing socket connection and attempting reconnect
[junit4:junit4] 2> 686941 T197 oazs.NIOServerCnxn.closeSock Closed socket
connection for client /127.0.0.1:49533 which had sessionid 0x13aa9d8483b0004
[junit4:junit4] 2> 686941 T197 oazs.NIOServerCnxn.closeSock Closed socket
connection for client /127.0.0.1:49604 which had sessionid 0x13aa9d8483b0006
[junit4:junit4] 2> 686941 T268 oaz.ClientCnxn$SendThread.run Unable to read
additional data from server sessionid 0x13aa9d8483b0006, likely server has
closed socket, closing socket connection and attempting reconnect
[junit4:junit4] 2> 686941 T197 oazs.NIOServerCnxn.closeSock Closed socket
connection for client /127.0.0.1:49582 which had sessionid 0x13aa9d8483b0005
[junit4:junit4] 2> 686941 T199 oazs.NIOServerCnxn$Factory.run NIOServerCnxn
factory exited run method
[junit4:junit4] 2> 686941 T197 oazs.FinalRequestProcessor.shutdown shutdown
of request processor complete
[junit4:junit4] 2> 686941 T197 oasc.ChaosMonkey.monkeyLog monkey: stop shard!
49404
[junit4:junit4] 2> 686941 T197 oasc.CoreContainer.shutdown Shutting down
CoreContainer instance=21402892
[junit4:junit4] 2> 686941 T197 oasc.SolrCore.close [collection1] CLOSING
SolrCore org.apache.solr.core.SolrCore@1adc6a5
[junit4:junit4] 2> 686941 T197 oasu.DirectUpdateHandler2.close closing
DirectUpdateHandler2{commits=0,autocommits=0,soft
autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 686941 T197 oasc.SolrCore.decrefSolrCoreState Closing
SolrCoreState
[junit4:junit4] 2> 686941 T197 oasu.DefaultSolrCoreState.closeIndexWriter
SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 686941 T197 oasu.DefaultSolrCoreState.closeIndexWriter
closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 686941 T197 oasc.SolrCore.closeSearcher [collection1]
Closing main searcher on request.
[junit4:junit4] 2> 686941 T218 oasc.Overseer$ClusterStateUpdater.amILeader
According to ZK I (id=88569889077002242-127.0.0.1:49404_solr-n_0000000000) am
no longer a leader.
[junit4:junit4] 2> 687050 T197 oaz.ZooKeeper.close Session: 0x13aa9d8483b0002
closed
[junit4:junit4] 2> 687050 T269 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@10713e3 name:ZooKeeperConnection
Watcher:127.0.0.1:49386/solr got event WatchedEvent state:Disconnected
type:None path:null path:null type:None
[junit4:junit4] 2> 687050 T217 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@f048dc name:ZooKeeperConnection
Watcher:127.0.0.1:49386/solr got event WatchedEvent state:Disconnected
type:None path:null path:null type:None
[junit4:junit4] 2> 687050 T231 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@1bf3d77 name:ZooKeeperConnection
Watcher:127.0.0.1:49386/solr got event WatchedEvent state:Disconnected
type:None path:null path:null type:None
[junit4:junit4] 2> 687050 T255 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@159bc95 name:ZooKeeperConnection
Watcher:127.0.0.1:49386/solr got event WatchedEvent state:Disconnected
type:None path:null path:null type:None
[junit4:junit4] 2> 687050 T243 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@a37a18 name:ZooKeeperConnection
Watcher:127.0.0.1:49386/solr got event WatchedEvent state:Disconnected
type:None path:null path:null type:None
[junit4:junit4] 2> 687050 T231 oascc.ConnectionManager.process zkClient has
disconnected
[junit4:junit4] 2> 687050 T269 oascc.ConnectionManager.process zkClient has
disconnected
[junit4:junit4] 2> 687050 T243 oascc.ConnectionManager.process zkClient has
disconnected
[junit4:junit4] 2> 687050 T217 oascc.ConnectionManager.process
Client->ZooKeeper status change trigger but we are already closed
[junit4:junit4] 2> 687050 T255 oascc.ConnectionManager.process zkClient has
disconnected
[junit4:junit4] 2> 687050 T217 oaz.ClientCnxn$EventThread.run EventThread
shut down
[junit4:junit4] 2> 687051 T197 oejsh.ContextHandler.doStop stopped
o.e.j.s.ServletContextHandler{/solr,null}
[junit4:junit4] 2> 687102 T197 oasc.ChaosMonkey.monkeyLog monkey: stop shard!
49470
[junit4:junit4] 2> 687102 T197 oasc.CoreContainer.shutdown Shutting down
CoreContainer instance=31291218
[junit4:junit4] 2> 687102 T197 oasc.SolrCore.close [collection1] CLOSING
SolrCore org.apache.solr.core.SolrCore@52a72a
[junit4:junit4] 2> 687105 T197 oasu.DirectUpdateHandler2.close closing
DirectUpdateHandler2{commits=1,autocommits=0,soft
autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 687106 T197 oasc.SolrCore.decrefSolrCoreState Closing
SolrCoreState
[junit4:junit4] 2> 687106 T197 oasu.DefaultSolrCoreState.closeIndexWriter
SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 687106 T197 oasu.DefaultSolrCoreState.closeIndexWriter
closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 687108 T197 oasc.SolrCore.closeSearcher [collection1]
Closing main searcher on request.
[junit4:junit4] 2> 688310 T242 oaz.ClientCnxn$SendThread.startConnect Opening
socket connection to server 127.0.0.1/127.0.0.1:49386
[junit4:junit4] 2> 688388 T254 oaz.ClientCnxn$SendThread.startConnect Opening
socket connection to server 127.0.0.1/127.0.0.1:49386
[junit4:junit4] 2> 688513 T230 oaz.ClientCnxn$SendThread.startConnect Opening
socket connection to server 127.0.0.1/127.0.0.1:49386
[junit4:junit4] 2> 688622 T268 oaz.ClientCnxn$SendThread.startConnect Opening
socket connection to server 127.0.0.1/127.0.0.1:49386
[junit4:junit4] 2> 689324 T242 oaz.ClientCnxn$SendThread.run WARNING Session
0x13aa9d8483b0004 for server null, unexpected error, closing socket connection
and attempting reconnect java.net.ConnectException: Connection refused: no
further information
[junit4:junit4] 2> at sun.nio.ch.SocketChannelImpl.checkConnect(Native
Method)
[junit4:junit4] 2> at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
[junit4:junit4] 2> at
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1143)
[junit4:junit4] 2>
[junit4:junit4] 2> 689386 T254 oaz.ClientCnxn$SendThread.run WARNING Session
0x13aa9d8483b0005 for server null, unexpected error, closing socket connection
and attempting reconnect java.net.ConnectException: Connection refused: no
further information
[junit4:junit4] 2> at sun.nio.ch.SocketChannelImpl.checkConnect(Native
Method)
[junit4:junit4] 2> at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
[junit4:junit4] 2> at
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1143)
[junit4:junit4] 2>
[junit4:junit4] 2> 689585 T231 oaz.ClientCnxn$EventThread.run EventThread
shut down
[junit4:junit4] 2> 689585 T197 oaz.ZooKeeper.close Session: 0x13aa9d8483b0003
closed
[junit4:junit4] 2> 689595 T268 oaz.ClientCnxn$SendThread.run WARNING Session
0x13aa9d8483b0006 for server null, unexpected error, closing socket connection
and attempting reconnect java.net.ConnectException: Connection refused: no
further information
[junit4:junit4] 2> at sun.nio.ch.SocketChannelImpl.checkConnect(Native
Method)
[junit4:junit4] 2> at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
[junit4:junit4] 2> at
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1143)
[junit4:junit4] 2>
[junit4:junit4] 2> 689595 T197 oejsh.ContextHandler.doStop stopped
o.e.j.s.ServletContextHandler{/solr,null}
[junit4:junit4] 2> 689646 T197 oasc.ChaosMonkey.monkeyLog monkey: stop shard!
49525
[junit4:junit4] 2> 689646 T197 oasc.CoreContainer.shutdown Shutting down
CoreContainer instance=29897499
[junit4:junit4] 2> 689646 T197 oasc.SolrCore.close [collection1] CLOSING
SolrCore org.apache.solr.core.SolrCore@1937bf4
[junit4:junit4] 2> 689650 T197 oasu.DirectUpdateHandler2.close closing
DirectUpdateHandler2{commits=0,autocommits=0,soft
autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 689650 T197 oasc.SolrCore.decrefSolrCoreState Closing
SolrCoreState
[junit4:junit4] 2> 689650 T197 oasu.DefaultSolrCoreState.closeIndexWriter
SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 689650 T197 oasu.DefaultSolrCoreState.closeIndexWriter
closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 689651 T197 oasc.SolrCore.closeSearcher [collection1]
Closing main searcher on request.
[junit4:junit4] 2> 690832 T242 oaz.ClientCnxn$SendThread.startConnect Opening
socket connection to server 127.0.0.1/127.0.0.1:49386
[junit4:junit4] 2> 691020 T254 oaz.ClientCnxn$SendThread.startConnect Opening
socket connection to server 127.0.0.1/127.0.0.1:49386
[junit4:junit4] 2> 691644 T268 oaz.ClientCnxn$SendThread.startConnect Opening
socket connection to server 127.0.0.1/127.0.0.1:49386
[junit4:junit4] 2> 691940 T243 oaz.ClientCnxn$EventThread.run EventThread
shut down
[junit4:junit4] 2> 691940 T197 oaz.ZooKeeper.close Session: 0x13aa9d8483b0004
closed
[junit4:junit4] 2> 691956 T197 oejsh.ContextHandler.doStop stopped
o.e.j.s.ServletContextHandler{/solr,null}
[junit4:junit4] 2> 692007 T197 oasc.ChaosMonkey.monkeyLog monkey: stop shard!
49577
[junit4:junit4] 2> 692007 T197 oasc.CoreContainer.shutdown Shutting down
CoreContainer instance=20808963
[junit4:junit4] 2> 692007 T197 oasc.SolrCore.close [collection1] CLOSING
SolrCore org.apache.solr.core.SolrCore@3049cf
[junit4:junit4] 2> 692022 T254 oaz.ClientCnxn$SendThread.run WARNING Session
0x13aa9d8483b0005 for server null, unexpected error, closing socket connection
and attempting reconnect java.net.ConnectException: Connection refused: no
further information
[junit4:junit4] 2> at sun.nio.ch.SocketChannelImpl.checkConnect(Native
Method)
[junit4:junit4] 2> at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
[junit4:junit4] 2> at
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1143)
[junit4:junit4] 2>
[junit4:junit4] 2> 692038 T197 oasu.DirectUpdateHandler2.close closing
DirectUpdateHandler2{commits=1,autocommits=0,soft
autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 692038 T197 oasc.SolrCore.decrefSolrCoreState Closing
SolrCoreState
[junit4:junit4] 2> 692038 T197 oasu.DefaultSolrCoreState.closeIndexWriter
SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 692038 T197 oasu.DefaultSolrCoreState.closeIndexWriter
closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 692038 T197 oasc.SolrCore.closeSearcher [collection1]
Closing main searcher on request.
[junit4:junit4] 2> 692147 T255 oaz.ClientCnxn$EventThread.run EventThread
shut down
[junit4:junit4] 2> 692147 T197 oaz.ZooKeeper.close Session: 0x13aa9d8483b0005
closed
[junit4:junit4] 2> 692163 T197 oejsh.ContextHandler.doStop stopped
o.e.j.s.ServletContextHandler{/solr,null}
[junit4:junit4] 2> 692214 T197 oasc.ChaosMonkey.monkeyLog monkey: stop shard!
49599
[junit4:junit4] 2> 692214 T197 oasc.CoreContainer.shutdown Shutting down
CoreContainer instance=29549291
[junit4:junit4] 2> 692760 T269 oaz.ClientCnxn$EventThread.run EventThread
shut down
[junit4:junit4] 2> 692760 T197 oaz.ZooKeeper.close Session: 0x13aa9d8483b0006
closed
[junit4:junit4] 2> 692775 T197 oejsh.ContextHandler.doStop stopped
o.e.j.s.ServletContextHandler{/solr,null}
[junit4:junit4] 2> 693029 T197 oas.SolrTestCaseJ4.tearDown ###Ending
testDistribSearch
[junit4:junit4] 2> NOTE: reproduce with: ant test
-Dtestcase=FullSolrCloudDistribCmdsTest -Dtests.method=testDistribSearch
-Dtests.seed=7CB79CDD2E99B88 -Dtests.slow=true -Dtests.locale=mt
-Dtests.timezone=Africa/Ndjamena -Dtests.file.encoding=UTF-8
[junit4:junit4] FAILURE 693s J6 |
FullSolrCloudDistribCmdsTest.testDistribSearch <<<
[junit4:junit4] > Throwable #1: java.lang.AssertionError: There are still
nodes recoverying - waited for 600 seconds
[junit4:junit4] > at
__randomizedtesting.SeedInfo.seed([7CB79CDD2E99B88:862DF7D5A5B6FBB4]:0)
[junit4:junit4] > at org.junit.Assert.fail(Assert.java:93)
[junit4:junit4] > at
org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:156)
[junit4:junit4] > at
org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:115)
[junit4:junit4] > at
org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:110)
[junit4:junit4] > at
org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:553)
[junit4:junit4] > at
org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.doTest(FullSolrCloudDistribCmdsTest.java:65)
[junit4:junit4] > at
org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:695)
[junit4:junit4] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
[junit4:junit4] > at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
[junit4:junit4] > at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[junit4:junit4] > at java.lang.reflect.Method.invoke(Method.java:597)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4] > at
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
[junit4:junit4] > at
org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
[junit4:junit4] > at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4] > at
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4:junit4] > at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4] > at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4] > at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4:junit4] > at
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
[junit4:junit4] > at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4] > at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4] > at
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:662)
[junit4:junit4] 2> 693045 T197 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4] 2> 815739 T197 oas.SolrTestCaseJ4.endTrackingSearchers SEVERE
ERROR: SolrIndexSearcher opens=7 closes=6
[junit4:junit4] 2> 815739 T196 ccr.ThreadLeakControl.checkThreadLeaks WARNING
Will linger awaiting termination of 1 leaked thread(s).
[junit4:junit4] 2> 835894 T196 ccr.ThreadLeakControl.checkThreadLeaks SEVERE
1 thread leaked from SUITE scope at
org.apache.solr.cloud.FullSolrCloudDistribCmdsTest:
[junit4:junit4] 2> 1) Thread[id=270,
name=searcherExecutor-141-thread-1, state=WAITING,
group=TGRP-FullSolrCloudDistribCmdsTest]
[junit4:junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4:junit4] 2> at
java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
[junit4:junit4] 2> at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
[junit4:junit4] 2> at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4] 2> at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
[junit4:junit4] 2> at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:662)
[junit4:junit4] 2> 835894 T196 ccr.ThreadLeakControl.tryToInterruptAll
Starting to interrupt leaked threads:
[junit4:junit4] 2> 1) Thread[id=270,
name=searcherExecutor-141-thread-1, state=WAITING,
group=TGRP-FullSolrCloudDistribCmdsTest]
[junit4:junit4] 2> 838983 T196 ccr.ThreadLeakControl.tryToInterruptAll SEVERE
There are still zombie threads that couldn't be terminated:
[junit4:junit4] 2> 1) Thread[id=270,
name=searcherExecutor-141-thread-1, state=WAITING,
group=TGRP-FullSolrCloudDistribCmdsTest]
[junit4:junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4:junit4] 2> at
java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
[junit4:junit4] 2> at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
[junit4:junit4] 2> at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4] 2> at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
[junit4:junit4] 2> at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:662)
[junit4:junit4] 2> NOTE: test params are: codec=Asserting,
sim=DefaultSimilarity, locale=mt, timezone=Africa/Ndjamena
[junit4:junit4] 2> NOTE: Windows Vista 6.0 x86/Sun Microsystems Inc. 1.6.0_23
(32-bit)/cpus=4,threads=2,free=36586920,total=76013568
[junit4:junit4] 2> NOTE: All tests run in this JVM: [TermsComponentTest,
SearchHandlerTest, TestStressLucene, TestRealTimeGet, TimeZoneUtilsTest,
TestBM25SimilarityFactory, CurrencyFieldTest, RAMDirectoryFactoryTest,
JSONWriterTest, UniqFieldsUpdateProcessorFactoryTest, TestSolr4Spatial,
SolrCoreTest, TestMultiCoreConfBootstrap, TestNumberUtils, TestRemoteStreaming,
FieldMutatingUpdateProcessorTest, FullSolrCloudDistribCmdsTest]
[junit4:junit4] 2> NOTE: reproduce with: ant test
-Dtestcase=FullSolrCloudDistribCmdsTest -Dtests.seed=7CB79CDD2E99B88
-Dtests.slow=true -Dtests.locale=mt -Dtests.timezone=Africa/Ndjamena
-Dtests.file.encoding=UTF-8
[junit4:junit4] ERROR 0.00s J6 | FullSolrCloudDistribCmdsTest (suite) <<<
[junit4:junit4] > Throwable #1: java.lang.AssertionError: ERROR:
SolrIndexSearcher opens=7 closes=6
[junit4:junit4] > at
__randomizedtesting.SeedInfo.seed([7CB79CDD2E99B88]:0)
[junit4:junit4] > at org.junit.Assert.fail(Assert.java:93)
[junit4:junit4] > at
org.apache.solr.SolrTestCaseJ4.endTrackingSearchers(SolrTestCaseJ4.java:242)
[junit4:junit4] > at
org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:97)
[junit4:junit4] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
[junit4:junit4] > at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
[junit4:junit4] > at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[junit4:junit4] > at java.lang.reflect.Method.invoke(Method.java:597)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:700)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4] > at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4:junit4] > at
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
[junit4:junit4] > at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4] > at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4] > at
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:662)
[junit4:junit4] > Throwable #2:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE
scope at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest:
[junit4:junit4] > 1) Thread[id=270, name=searcherExecutor-141-thread-1,
state=WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]
[junit4:junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4:junit4] > at
java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
[junit4:junit4] > at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
[junit4:junit4] > at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4] > at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
[junit4:junit4] > at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:662)
[junit4:junit4] > at
__randomizedtesting.SeedInfo.seed([7CB79CDD2E99B88]:0)
[junit4:junit4] > Throwable #3:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie
threads that couldn't be terminated:
[junit4:junit4] > 1) Thread[id=270, name=searcherExecutor-141-thread-1,
state=WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]
[junit4:junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4:junit4] > at
java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
[junit4:junit4] > at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
[junit4:junit4] > at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4] > at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
[junit4:junit4] > at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:662)
[junit4:junit4] > at
__randomizedtesting.SeedInfo.seed([7CB79CDD2E99B88]:0)
[junit4:junit4] Completed on J6 in 839.03s, 1 test, 2 failures, 2 errors <<<
FAILURES!
[junit4:junit4]
[junit4:junit4]
[junit4:junit4] Tests with failures:
[junit4:junit4] -
org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.testDistribSearch
[junit4:junit4] - org.apache.solr.cloud.FullSolrCloudDistribCmdsTest (suite)
[junit4:junit4]
[junit4:junit4]
[junit4:junit4] JVM J0: 2.61 .. 274.96 = 272.35s
[junit4:junit4] JVM J1: 2.87 .. 268.05 = 265.18s
[junit4:junit4] JVM J2: 2.62 .. 273.02 = 270.40s
[junit4:junit4] JVM J3: 2.61 .. 276.25 = 273.65s
[junit4:junit4] JVM J4: 2.08 .. 267.53 = 265.46s
[junit4:junit4] JVM J5: 2.62 .. 317.66 = 315.04s
[junit4:junit4] JVM J6: 2.62 .. 970.30 = 967.68s
[junit4:junit4] JVM J7: 4.20 .. 320.83 = 316.63s
[junit4:junit4] Execution time total: 16 minutes 10 seconds
[junit4:junit4] Tests summary: 233 suites, 965 tests, 3 suite-level errors, 1
failure, 12 ignored (7 assumptions)
BUILD FAILED
<http://sierranevada.servebeer.com:8080/job/slow-io-beasting/ws/build.xml>:39:
The following error occurred while executing this line:
<http://sierranevada.servebeer.com:8080/job/slow-io-beasting/ws/solr/build.xml>:178:
The following error occurred while executing this line:
<http://sierranevada.servebeer.com:8080/job/slow-io-beasting/ws/lucene/module-build.xml>:61:
The following error occurred while executing this line:
<http://sierranevada.servebeer.com:8080/job/slow-io-beasting/ws/lucene/common-build.xml>:1159:
The following error occurred while executing this line:
<http://sierranevada.servebeer.com:8080/job/slow-io-beasting/ws/lucene/common-build.xml>:827:
There were test failures: 233 suites, 965 tests, 3 suite-level errors, 1
failure, 12 ignored (7 assumptions)
Total time: 24 minutes 52 seconds
Build step 'Invoke Ant' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
