See <http://sierranevada.servebeer.com:8080/job/slow-io-beasting/4982/>
------------------------------------------
[...truncated 14009 lines...]
[junit4:junit4] 2> 694989 T17 oazs.NIOServerCnxn.closeSock Closed socket
connection for client /127.0.0.1:57720 which had sessionid 0x13aa7129a150002
[junit4:junit4] 2> 694989 T37 oaz.ClientCnxn$SendThread.run Unable to read
additional data from server sessionid 0x13aa7129a150002, likely server has
closed socket, closing socket connection and attempting reconnect
[junit4:junit4] 2> 694989 T17 oazs.NIOServerCnxn.closeSock Closed socket
connection for client /127.0.0.1:57753 which had sessionid 0x13aa7129a150004
[junit4:junit4] 2> 694989 T19 oazs.NIOServerCnxn$Factory.run NIOServerCnxn
factory exited run method
[junit4:junit4] 2> 694989 T67 oaz.ClientCnxn$SendThread.run Unable to read
additional data from server sessionid 0x13aa7129a150004, likely server has
closed socket, closing socket connection and attempting reconnect
[junit4:junit4] 2> 694989 T17 oazs.FinalRequestProcessor.shutdown shutdown of
request processor complete
[junit4:junit4] 2> 694989 T17 oasc.ChaosMonkey.monkeyLog monkey: stop shard!
57715
[junit4:junit4] 2> 694989 T17 oasc.CoreContainer.shutdown Shutting down
CoreContainer instance=8578291
[junit4:junit4] 2> 694989 T17 oasc.SolrCore.close [collection1] CLOSING
SolrCore org.apache.solr.core.SolrCore@193c366
[junit4:junit4] 2> 695005 T17 oasu.DirectUpdateHandler2.close closing
DirectUpdateHandler2{commits=0,autocommits=0,soft
autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 695005 T17 oasc.SolrCore.decrefSolrCoreState Closing
SolrCoreState
[junit4:junit4] 2> 695005 T17 oasu.DefaultSolrCoreState.closeIndexWriter
SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 695005 T17 oasu.DefaultSolrCoreState.closeIndexWriter
closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 695005 T17 oasc.SolrCore.closeSearcher [collection1]
Closing main searcher on request.
[junit4:junit4] 2> 695005 T39 oasc.Overseer$ClusterStateUpdater.amILeader
According to ZK I (id=88566841023463426-127.0.0.1:57715_solr-n_0000000000) am
no longer a leader.
[junit4:junit4] 2> 695099 T68 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@14b03ea name:ZooKeeperConnection
Watcher:127.0.0.1:57708/solr got event WatchedEvent state:Disconnected
type:None path:null path:null type:None
[junit4:junit4] 2> 695099 T54 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@15da7d name:ZooKeeperConnection
Watcher:127.0.0.1:57708/solr got event WatchedEvent state:Disconnected
type:None path:null path:null type:None
[junit4:junit4] 2> 695099 T96 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@24c22b name:ZooKeeperConnection
Watcher:127.0.0.1:57708/solr got event WatchedEvent state:Disconnected
type:None path:null path:null type:None
[junit4:junit4] 2> 695099 T82 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@1ea6a1c name:ZooKeeperConnection
Watcher:127.0.0.1:57708/solr got event WatchedEvent state:Disconnected
type:None path:null path:null type:None
[junit4:junit4] 2> 695099 T96 oascc.ConnectionManager.process zkClient has
disconnected
[junit4:junit4] 2> 695099 T54 oascc.ConnectionManager.process zkClient has
disconnected
[junit4:junit4] 2> 695099 T82 oascc.ConnectionManager.process zkClient has
disconnected
[junit4:junit4] 2> 695099 T68 oascc.ConnectionManager.process zkClient has
disconnected
[junit4:junit4] 2> 695099 T38 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@cd6aa0 name:ZooKeeperConnection
Watcher:127.0.0.1:57708/solr got event WatchedEvent state:Disconnected
type:None path:null path:null type:None
[junit4:junit4] 2> 695099 T17 oaz.ZooKeeper.close Session: 0x13aa7129a150002
closed
[junit4:junit4] 2> 695099 T38 oascc.ConnectionManager.process zkClient has
disconnected
[junit4:junit4] 2> 695099 T38 oaz.ClientCnxn$EventThread.run EventThread shut
down
[junit4:junit4] 2> 695100 T17 oejsh.ContextHandler.doStop stopped
o.e.j.s.ServletContextHandler{/solr,null}
[junit4:junit4] 2> 695154 T17 oasc.ChaosMonkey.monkeyLog monkey: stop shard!
57724
[junit4:junit4] 2> 695154 T17 oasc.CoreContainer.shutdown Shutting down
CoreContainer instance=7281109
[junit4:junit4] 2> 695154 T17 oasc.SolrCore.close [collection1] CLOSING
SolrCore org.apache.solr.core.SolrCore@1408325
[junit4:junit4] 2> 695161 T17 oasu.DirectUpdateHandler2.close closing
DirectUpdateHandler2{commits=0,autocommits=0,soft
autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 695162 T17 oasc.SolrCore.decrefSolrCoreState Closing
SolrCoreState
[junit4:junit4] 2> 695162 T17 oasu.DefaultSolrCoreState.closeIndexWriter
SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 695162 T17 oasu.DefaultSolrCoreState.closeIndexWriter
closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 695162 T17 oasc.SolrCore.closeSearcher [collection1]
Closing main searcher on request.
[junit4:junit4] 2> 696125 T95 oaz.ClientCnxn$SendThread.startConnect Opening
socket connection to server 127.0.0.1/127.0.0.1:57708
[junit4:junit4] 2> 696140 T67 oaz.ClientCnxn$SendThread.startConnect Opening
socket connection to server 127.0.0.1/127.0.0.1:57708
[junit4:junit4] 2> 696187 T53 oaz.ClientCnxn$SendThread.startConnect Opening
socket connection to server 127.0.0.1/127.0.0.1:57708
[junit4:junit4] 2> 696655 T81 oaz.ClientCnxn$SendThread.startConnect Opening
socket connection to server 127.0.0.1/127.0.0.1:57708
[junit4:junit4] 2> 697123 T95 oaz.ClientCnxn$SendThread.run WARNING Session
0x13aa7129a150006 for server null, unexpected error, closing socket connection
and attempting reconnect java.net.ConnectException: Connection refused: no
further information
[junit4:junit4] 2> at sun.nio.ch.SocketChannelImpl.checkConnect(Native
Method)
[junit4:junit4] 2> at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
[junit4:junit4] 2> at
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1143)
[junit4:junit4] 2>
[junit4:junit4] 2> 697139 T67 oaz.ClientCnxn$SendThread.run WARNING Session
0x13aa7129a150004 for server null, unexpected error, closing socket connection
and attempting reconnect java.net.ConnectException: Connection refused: no
further information
[junit4:junit4] 2> at sun.nio.ch.SocketChannelImpl.checkConnect(Native
Method)
[junit4:junit4] 2> at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
[junit4:junit4] 2> at
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1143)
[junit4:junit4] 2>
[junit4:junit4] 2> 697297 T54 oaz.ClientCnxn$EventThread.run EventThread shut
down
[junit4:junit4] 2> 697297 T17 oaz.ZooKeeper.close Session: 0x13aa7129a150003
closed
[junit4:junit4] 2> 697312 T17 oejsh.ContextHandler.doStop stopped
o.e.j.s.ServletContextHandler{/solr,null}
[junit4:junit4] 2> 697363 T17 oasc.ChaosMonkey.monkeyLog monkey: stop shard!
57748
[junit4:junit4] 2> 697363 T17 oasc.CoreContainer.shutdown Shutting down
CoreContainer instance=6847200
[junit4:junit4] 2> 697363 T17 oasc.SolrCore.close [collection1] CLOSING
SolrCore org.apache.solr.core.SolrCore@405f6
[junit4:junit4] 2> 697395 T17 oasu.DirectUpdateHandler2.close closing
DirectUpdateHandler2{commits=1,autocommits=0,soft
autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 697395 T17 oasc.SolrCore.decrefSolrCoreState Closing
SolrCoreState
[junit4:junit4] 2> 697395 T17 oasu.DefaultSolrCoreState.closeIndexWriter
SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 697395 T17 oasu.DefaultSolrCoreState.closeIndexWriter
closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 697395 T17 oasc.SolrCore.closeSearcher [collection1]
Closing main searcher on request.
[junit4:junit4] 2> 697660 T81 oaz.ClientCnxn$SendThread.run WARNING Session
0x13aa7129a150005 for server null, unexpected error, closing socket connection
and attempting reconnect java.net.ConnectException: Connection refused: no
further information
[junit4:junit4] 2> at sun.nio.ch.SocketChannelImpl.checkConnect(Native
Method)
[junit4:junit4] 2> at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
[junit4:junit4] 2> at
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1143)
[junit4:junit4] 2>
[junit4:junit4] 2> 698280 T67 oaz.ClientCnxn$SendThread.startConnect Opening
socket connection to server 127.0.0.1/127.0.0.1:57708
[junit4:junit4] 2> 698826 T95 oaz.ClientCnxn$SendThread.startConnect Opening
socket connection to server 127.0.0.1/127.0.0.1:57708
[junit4:junit4] 2> 699248 T81 oaz.ClientCnxn$SendThread.startConnect Opening
socket connection to server 127.0.0.1/127.0.0.1:57708
[junit4:junit4] 2> 699372 T68 oaz.ClientCnxn$EventThread.run EventThread shut
down
[junit4:junit4] 2> 699372 T17 oaz.ZooKeeper.close Session: 0x13aa7129a150004
closed
[junit4:junit4] 2> 699388 T17 oejsh.ContextHandler.doStop stopped
o.e.j.s.ServletContextHandler{/solr,null}
[junit4:junit4] 2> 699439 T17 oasc.ChaosMonkey.monkeyLog monkey: stop shard!
57771
[junit4:junit4] 2> 699439 T17 oasc.CoreContainer.shutdown Shutting down
CoreContainer instance=12298774
[junit4:junit4] 2> 699829 T95 oaz.ClientCnxn$SendThread.run WARNING Session
0x13aa7129a150006 for server null, unexpected error, closing socket connection
and attempting reconnect java.net.ConnectException: Connection refused: no
further information
[junit4:junit4] 2> at sun.nio.ch.SocketChannelImpl.checkConnect(Native
Method)
[junit4:junit4] 2> at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
[junit4:junit4] 2> at
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1143)
[junit4:junit4] 2>
[junit4:junit4] 2> 700320 T82 oaz.ClientCnxn$EventThread.run EventThread shut
down
[junit4:junit4] 2> 700320 T17 oaz.ZooKeeper.close Session: 0x13aa7129a150005
closed
[junit4:junit4] 2> 700330 T17 oejsh.ContextHandler.doStop stopped
o.e.j.s.ServletContextHandler{/solr,null}
[junit4:junit4] 2> 700381 T17 oasc.ChaosMonkey.monkeyLog monkey: stop shard!
58511
[junit4:junit4] 2> 700381 T17 oasc.CoreContainer.shutdown Shutting down
CoreContainer instance=29057197
[junit4:junit4] 2> 700381 T17 oasc.SolrCore.close [collection1] CLOSING
SolrCore org.apache.solr.core.SolrCore@789d63
[junit4:junit4] 2> 700393 T17 oasu.DirectUpdateHandler2.close closing
DirectUpdateHandler2{commits=1,autocommits=0,soft
autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 700393 T17 oasc.SolrCore.decrefSolrCoreState Closing
SolrCoreState
[junit4:junit4] 2> 700394 T17 oasu.DefaultSolrCoreState.closeIndexWriter
SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 700394 T17 oasu.DefaultSolrCoreState.closeIndexWriter
closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 700396 T17 oasc.SolrCore.closeSearcher [collection1]
Closing main searcher on request.
[junit4:junit4] 2> 701595 T95 oaz.ClientCnxn$SendThread.startConnect Opening
socket connection to server 127.0.0.1/127.0.0.1:57708
[junit4:junit4] 2> 702702 T96 oaz.ClientCnxn$EventThread.run EventThread shut
down
[junit4:junit4] 2> 702702 T17 oaz.ZooKeeper.close Session: 0x13aa7129a150006
closed
[junit4:junit4] 2> 702718 T17 oejsh.ContextHandler.doStop stopped
o.e.j.s.ServletContextHandler{/solr,null}
[junit4:junit4] 2> 702972 T17 oas.SolrTestCaseJ4.tearDown ###Ending
testDistribSearch
[junit4:junit4] 2> NOTE: reproduce with: ant test
-Dtestcase=BasicDistributedZkTest -Dtests.method=testDistribSearch
-Dtests.seed=AED2C0218DB5A9CA -Dtests.slow=true -Dtests.locale=sr
-Dtests.timezone=America/Regina -Dtests.file.encoding=UTF-8
[junit4:junit4] FAILURE 703s J0 | BasicDistributedZkTest.testDistribSearch <<<
[junit4:junit4] > Throwable #1: java.lang.AssertionError: There are still
nodes recoverying
[junit4:junit4] > at
__randomizedtesting.SeedInfo.seed([AED2C0218DB5A9CA:2F344E39FAEAC9F6]:0)
[junit4:junit4] > at org.junit.Assert.fail(Assert.java:93)
[junit4:junit4] > at
org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:154)
[junit4:junit4] > at
org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:115)
[junit4:junit4] > at
org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:110)
[junit4:junit4] > at
org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:553)
[junit4:junit4] > at
org.apache.solr.cloud.BasicDistributedZkTest.doTest(BasicDistributedZkTest.java:162)
[junit4:junit4] > at
org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:695)
[junit4:junit4] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
[junit4:junit4] > at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
[junit4:junit4] > at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[junit4:junit4] > at java.lang.reflect.Method.invoke(Method.java:597)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4] > at
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
[junit4:junit4] > at
org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
[junit4:junit4] > at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4] > at
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4:junit4] > at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4] > at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4] > at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4:junit4] > at
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
[junit4:junit4] > at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4] > at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4] > at
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:662)
[junit4:junit4] 2> 703019 T17 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4] 2> 825713 T17 oas.SolrTestCaseJ4.endTrackingSearchers SEVERE
ERROR: SolrIndexSearcher opens=7 closes=6
[junit4:junit4] 2> 825713 T16 ccr.ThreadLeakControl.checkThreadLeaks WARNING
Will linger awaiting termination of 1 leaked thread(s).
[junit4:junit4] 2> 845868 T16 ccr.ThreadLeakControl.checkThreadLeaks SEVERE 1
thread leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest:
[junit4:junit4] 2> 1) Thread[id=83, name=searcherExecutor-44-thread-1,
state=WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4:junit4] 2> at
java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
[junit4:junit4] 2> at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
[junit4:junit4] 2> at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4] 2> at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
[junit4:junit4] 2> at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:662)
[junit4:junit4] 2> 845868 T16 ccr.ThreadLeakControl.tryToInterruptAll
Starting to interrupt leaked threads:
[junit4:junit4] 2> 1) Thread[id=83, name=searcherExecutor-44-thread-1,
state=WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4] 2> 848957 T16 ccr.ThreadLeakControl.tryToInterruptAll SEVERE
There are still zombie threads that couldn't be terminated:
[junit4:junit4] 2> 1) Thread[id=83, name=searcherExecutor-44-thread-1,
state=WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4:junit4] 2> at
java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
[junit4:junit4] 2> at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
[junit4:junit4] 2> at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4] 2> at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
[junit4:junit4] 2> at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:662)
[junit4:junit4] 2> NOTE: test params are: codec=Asserting,
sim=DefaultSimilarity, locale=sr, timezone=America/Regina
[junit4:junit4] 2> NOTE: Windows Vista 6.0 x86/Sun Microsystems Inc. 1.6.0_23
(32-bit)/cpus=4,threads=2,free=5104312,total=16515072
[junit4:junit4] 2> NOTE: All tests run in this JVM: [TestFastWriter,
CoreContainerCoreInitFailuresTest, BasicDistributedZkTest]
[junit4:junit4] 2> NOTE: reproduce with: ant test
-Dtestcase=BasicDistributedZkTest -Dtests.seed=AED2C0218DB5A9CA
-Dtests.slow=true -Dtests.locale=sr -Dtests.timezone=America/Regina
-Dtests.file.encoding=UTF-8
[junit4:junit4] ERROR 0.00s J0 | BasicDistributedZkTest (suite) <<<
[junit4:junit4] > Throwable #1: java.lang.AssertionError: ERROR:
SolrIndexSearcher opens=7 closes=6
[junit4:junit4] > at
__randomizedtesting.SeedInfo.seed([AED2C0218DB5A9CA]:0)
[junit4:junit4] > at org.junit.Assert.fail(Assert.java:93)
[junit4:junit4] > at
org.apache.solr.SolrTestCaseJ4.endTrackingSearchers(SolrTestCaseJ4.java:242)
[junit4:junit4] > at
org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:97)
[junit4:junit4] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
[junit4:junit4] > at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
[junit4:junit4] > at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[junit4:junit4] > at java.lang.reflect.Method.invoke(Method.java:597)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:700)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4] > at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4:junit4] > at
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
[junit4:junit4] > at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4] > at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4] > at
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:662)
[junit4:junit4] > Throwable #2:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE
scope at org.apache.solr.cloud.BasicDistributedZkTest:
[junit4:junit4] > 1) Thread[id=83, name=searcherExecutor-44-thread-1,
state=WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4:junit4] > at
java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
[junit4:junit4] > at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
[junit4:junit4] > at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4] > at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
[junit4:junit4] > at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:662)
[junit4:junit4] > at
__randomizedtesting.SeedInfo.seed([AED2C0218DB5A9CA]:0)
[junit4:junit4] > Throwable #3:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie
threads that couldn't be terminated:
[junit4:junit4] > 1) Thread[id=83, name=searcherExecutor-44-thread-1,
state=WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4:junit4] > at
java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
[junit4:junit4] > at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
[junit4:junit4] > at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4] > at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
[junit4:junit4] > at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:662)
[junit4:junit4] > at
__randomizedtesting.SeedInfo.seed([AED2C0218DB5A9CA]:0)
[junit4:junit4] Completed on J0 in 849.00s, 1 test, 2 failures, 2 errors <<<
FAILURES!
[junit4:junit4]
[junit4:junit4]
[junit4:junit4] Tests with failures:
[junit4:junit4] -
org.apache.solr.cloud.BasicDistributedZkTest.testDistribSearch
[junit4:junit4] - org.apache.solr.cloud.BasicDistributedZkTest (suite)
[junit4:junit4]
[junit4:junit4]
[junit4:junit4] JVM J0: 2.31 .. 868.61 = 866.30s
[junit4:junit4] JVM J1: 2.56 .. 235.31 = 232.75s
[junit4:junit4] JVM J2: 2.56 .. 299.30 = 296.74s
[junit4:junit4] JVM J3: 2.56 .. 241.05 = 238.49s
[junit4:junit4] JVM J4: 3.10 .. 249.83 = 246.73s
[junit4:junit4] JVM J5: 2.29 .. 246.90 = 244.60s
[junit4:junit4] JVM J6: 3.35 .. 239.75 = 236.40s
[junit4:junit4] JVM J7: 2.56 .. 227.09 = 224.53s
[junit4:junit4] Execution time total: 14 minutes 28 seconds
[junit4:junit4] Tests summary: 233 suites, 965 tests, 3 suite-level errors, 1
failure, 12 ignored (7 assumptions)
BUILD FAILED
<http://sierranevada.servebeer.com:8080/job/slow-io-beasting/ws/build.xml>:39:
The following error occurred while executing this line:
<http://sierranevada.servebeer.com:8080/job/slow-io-beasting/ws/solr/build.xml>:178:
The following error occurred while executing this line:
<http://sierranevada.servebeer.com:8080/job/slow-io-beasting/ws/lucene/module-build.xml>:61:
The following error occurred while executing this line:
<http://sierranevada.servebeer.com:8080/job/slow-io-beasting/ws/lucene/common-build.xml>:1159:
The following error occurred while executing this line:
<http://sierranevada.servebeer.com:8080/job/slow-io-beasting/ws/lucene/common-build.xml>:827:
There were test failures: 233 suites, 965 tests, 3 suite-level errors, 1
failure, 12 ignored (7 assumptions)
Total time: 22 minutes 59 seconds
Build step 'Invoke Ant' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
