Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/9736/
Java: 64bit/jdk1.7.0_60-ea-b10 -XX:-UseCompressedOops -XX:+UseSerialGC

4 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.solr.cloud.TestRequestStatusCollectionAPI

Error Message:
ERROR: SolrZkClient opens=4 closes=3

Stack Trace:
java.lang.AssertionError: ERROR: SolrZkClient opens=4 closes=3
        at __randomizedtesting.SeedInfo.seed([BCC754BFE9236E64]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at 
org.apache.solr.SolrTestCaseJ4.endTrackingZkClients(SolrTestCaseJ4.java:420)
        at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:177)
        at sun.reflect.GeneratedMethodAccessor28.invoke(Unknown Source)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1617)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:789)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:359)
        at java.lang.Thread.run(Thread.java:744)


FAILED:  
junit.framework.TestSuite.org.apache.solr.cloud.TestRequestStatusCollectionAPI

Error Message:
2 threads leaked from SUITE scope at 
org.apache.solr.cloud.TestRequestStatusCollectionAPI:     1) Thread[id=7343, 
name=TEST-TestRequestStatusCollectionAPI.testDistribSearch-seed#[BCC754BFE9236E64]-SendThread(localhost.localdomain:34283),
 state=TIMED_WAITING, group=TGRP-TestRequestStatusCollectionAPI]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:86) 
        at 
org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:937)    
     at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993)    
2) Thread[id=7344, 
name=TEST-TestRequestStatusCollectionAPI.testDistribSearch-seed#[BCC754BFE9236E64]-EventThread,
 state=WAITING, group=TGRP-TestRequestStatusCollectionAPI]         at 
sun.misc.Unsafe.park(Native Method)         at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
         at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)     
    at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:491)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE 
scope at org.apache.solr.cloud.TestRequestStatusCollectionAPI: 
   1) Thread[id=7343, 
name=TEST-TestRequestStatusCollectionAPI.testDistribSearch-seed#[BCC754BFE9236E64]-SendThread(localhost.localdomain:34283),
 state=TIMED_WAITING, group=TGRP-TestRequestStatusCollectionAPI]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:86)
        at 
org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:937)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993)
   2) Thread[id=7344, 
name=TEST-TestRequestStatusCollectionAPI.testDistribSearch-seed#[BCC754BFE9236E64]-EventThread,
 state=WAITING, group=TGRP-TestRequestStatusCollectionAPI]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:491)
        at __randomizedtesting.SeedInfo.seed([BCC754BFE9236E64]:0)


FAILED:  
junit.framework.TestSuite.org.apache.solr.cloud.TestRequestStatusCollectionAPI

Error Message:
There are still zombie threads that couldn't be terminated:    1) 
Thread[id=7343, 
name=TEST-TestRequestStatusCollectionAPI.testDistribSearch-seed#[BCC754BFE9236E64]-SendThread(localhost.localdomain:34283),
 state=TIMED_WAITING, group=TGRP-TestRequestStatusCollectionAPI]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:86) 
        at 
org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:937)    
     at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie 
threads that couldn't be terminated:
   1) Thread[id=7343, 
name=TEST-TestRequestStatusCollectionAPI.testDistribSearch-seed#[BCC754BFE9236E64]-SendThread(localhost.localdomain:34283),
 state=TIMED_WAITING, group=TGRP-TestRequestStatusCollectionAPI]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:86)
        at 
org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:937)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993)
        at __randomizedtesting.SeedInfo.seed([BCC754BFE9236E64]:0)


REGRESSION:  
org.apache.solr.cloud.TestRequestStatusCollectionAPI.testDistribSearch

Error Message:


Stack Trace:
org.apache.solr.common.cloud.ZooKeeperException: 
        at 
__randomizedtesting.SeedInfo.seed([BCC754BFE9236E64:3D21DAA79E7C0E58]:0)
        at 
org.apache.solr.client.solrj.impl.CloudSolrServer.connect(CloudSolrServer.java:249)
        at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.initCloud(AbstractFullDistribZkTestBase.java:256)
        at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.createServers(AbstractFullDistribZkTestBase.java:312)
        at 
org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:868)
        at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1617)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:826)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:862)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:876)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
        at 
org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:359)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:783)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:443)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:835)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:737)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:771)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:782)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:359)
        at java.lang.Thread.run(Thread.java:744)
Caused by: org.apache.zookeeper.KeeperException$NoNodeException: 
KeeperErrorCode = NoNode for /live_nodes
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
        at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
        at 
org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:259)
        at 
org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:256)
        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
        at 
org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:256)
        at 
org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:299)
        at 
org.apache.solr.client.solrj.impl.CloudSolrServer.connect(CloudSolrServer.java:242)
        ... 41 more




Build Log:
[...truncated 11248 lines...]
   [junit4] Suite: org.apache.solr.cloud.TestRequestStatusCollectionAPI
   [junit4]   2> 1641076 T7322 oas.SolrTestCaseJ4.startTrackingSearchers WARN 
startTrackingSearchers: numOpens=6 numCloses=6
   [junit4]   2> 1641076 T7322 
oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system 
property: /gqmb/
   [junit4]   2> 1641080 T7322 oas.SolrTestCaseJ4.setUp ###Starting 
testDistribSearch
   [junit4]   2> Creating dataDir: 
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-TestRequestStatusCollectionAPI-1395211717555
   [junit4]   2> 1641080 T7322 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   2> 1641081 T7323 oasc.ZkTestServer$ZKServerMain.runFromConfig 
Starting server
   [junit4]   2> 1641181 T7322 oasc.ZkTestServer.run start zk server on 
port:34283
   [junit4]   2> 1641182 T7322 oascc.ConnectionManager.waitForConnected Waiting 
for client to connect to ZooKeeper
   [junit4]   2> 1641186 T7329 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@6033d8ce 
name:ZooKeeperConnection Watcher:127.0.0.1:34283 got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1641186 T7322 oascc.ConnectionManager.waitForConnected Client 
is connected to ZooKeeper
   [junit4]   2> 1641186 T7322 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 1641190 T7322 oascc.ConnectionManager.waitForConnected Waiting 
for client to connect to ZooKeeper
   [junit4]   2> 1641191 T7331 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@205fefcf 
name:ZooKeeperConnection Watcher:127.0.0.1:34283/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1641191 T7322 oascc.ConnectionManager.waitForConnected Client 
is connected to ZooKeeper
   [junit4]   2> 1641191 T7322 oascc.SolrZkClient.makePath makePath: 
/collections/collection1
   [junit4]   2> 1641193 T7322 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/shards
   [junit4]   2> 1641195 T7322 oascc.SolrZkClient.makePath makePath: 
/collections/control_collection
   [junit4]   2> 1641196 T7322 oascc.SolrZkClient.makePath makePath: 
/collections/control_collection/shards
   [junit4]   2> 1641198 T7322 oasc.AbstractZkTestCase.putConfig put 
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml
 to /configs/conf1/solrconfig.xml
   [junit4]   2> 1641199 T7322 oascc.SolrZkClient.makePath makePath: 
/configs/conf1/solrconfig.xml
   [junit4]   2> 1641201 T7322 oasc.AbstractZkTestCase.putConfig put 
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/core/src/test-files/solr/collection1/conf/schema15.xml
 to /configs/conf1/schema.xml
   [junit4]   2> 1641202 T7322 oascc.SolrZkClient.makePath makePath: 
/configs/conf1/schema.xml
   [junit4]   2> 1641204 T7322 oasc.AbstractZkTestCase.putConfig put 
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml
 to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1641205 T7322 oascc.SolrZkClient.makePath makePath: 
/configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1641207 T7322 oasc.AbstractZkTestCase.putConfig put 
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt
 to /configs/conf1/stopwords.txt
   [junit4]   2> 1641207 T7322 oascc.SolrZkClient.makePath makePath: 
/configs/conf1/stopwords.txt
   [junit4]   2> 1641209 T7322 oasc.AbstractZkTestCase.putConfig put 
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt
 to /configs/conf1/protwords.txt
   [junit4]   2> 1641210 T7322 oascc.SolrZkClient.makePath makePath: 
/configs/conf1/protwords.txt
   [junit4]   2> 1641211 T7322 oasc.AbstractZkTestCase.putConfig put 
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml
 to /configs/conf1/currency.xml
   [junit4]   2> 1641212 T7322 oascc.SolrZkClient.makePath makePath: 
/configs/conf1/currency.xml
   [junit4]   2> 1641215 T7322 oasc.AbstractZkTestCase.putConfig put 
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml
 to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1641215 T7322 oascc.SolrZkClient.makePath makePath: 
/configs/conf1/enumsConfig.xml
   [junit4]   2> 1641218 T7322 oasc.AbstractZkTestCase.putConfig put 
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json
 to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1641219 T7322 oascc.SolrZkClient.makePath makePath: 
/configs/conf1/open-exchange-rates.json
   [junit4]   2> 1641221 T7322 oasc.AbstractZkTestCase.putConfig put 
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt
 to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1641221 T7322 oascc.SolrZkClient.makePath makePath: 
/configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1641223 T7322 oasc.AbstractZkTestCase.putConfig put 
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt
 to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1641223 T7322 oascc.SolrZkClient.makePath makePath: 
/configs/conf1/old_synonyms.txt
   [junit4]   2> 1641226 T7322 oasc.AbstractZkTestCase.putConfig put 
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt
 to /configs/conf1/synonyms.txt
   [junit4]   2> 1641226 T7322 oascc.SolrZkClient.makePath makePath: 
/configs/conf1/synonyms.txt
   [junit4]   2> 1641470 T7322 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 1641473 T7322 oejs.AbstractConnector.doStart Started 
[email protected]:47403
   [junit4]   2> 1641473 T7322 oass.SolrDispatchFilter.init 
SolrDispatchFilter.init()
   [junit4]   2> 1641474 T7322 oasc.SolrResourceLoader.locateSolrHome JNDI not 
configured for solr (NoInitialContextEx)
   [junit4]   2> 1641474 T7322 oasc.SolrResourceLoader.locateSolrHome using 
system property solr.solr.home: 
../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.TestRequestStatusCollectionAPI-controljetty-1395211717703
   [junit4]   2> 1641474 T7322 oasc.SolrResourceLoader.<init> new 
SolrResourceLoader for directory: 
'../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.TestRequestStatusCollectionAPI-controljetty-1395211717703/'
   [junit4]   2> 1641484 T7322 oass.SolrDispatchFilter.loadConfigSolr Trying to 
read solr.xml from 127.0.0.1:34283/solr
   [junit4]   2> 1641485 T7322 oascc.ConnectionManager.waitForConnected Waiting 
for client to connect to ZooKeeper
   [junit4]   2> 1641486 T7342 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@51747c94 
name:ZooKeeperConnection Watcher:127.0.0.1:34283/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1641486 T7322 oascc.ConnectionManager.waitForConnected Client 
is connected to ZooKeeper
   [junit4]   2> 1641488 T7322 oass.SolrDispatchFilter.init ERROR Could not 
start Solr. Check solr/home property and the logs
   [junit4]   2> 1641488 T7322 oasc.SolrException.log ERROR 
null:org.apache.solr.common.SolrException: Could not load solr.xml from 
zookeeper
   [junit4]   2>                at 
org.apache.solr.servlet.SolrDispatchFilter.loadConfigSolr(SolrDispatchFilter.java:181)
   [junit4]   2>                at 
org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:197)
   [junit4]   2>                at 
org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:146)
   [junit4]   2>                at 
org.eclipse.jetty.servlet.FilterHolder.doStart(FilterHolder.java:119)
   [junit4]   2>                at 
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:64)
   [junit4]   2>                at 
org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:719)
   [junit4]   2>                at 
org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1309)
   [junit4]   2>                at 
org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1345)
   [junit4]   2>                at 
org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1085)
   [junit4]   2>                at 
org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:931)
   [junit4]   2>                at 
org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:888)
   [junit4]   2>                at 
org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:340)
   [junit4]   2>                at 
org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:325)
   [junit4]   2>                at 
org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:174)
   [junit4]   2>                at 
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:65)
   [junit4]   2>                at 
org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:430)
   [junit4]   2>                at 
org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:403)
   [junit4]   2>                at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.createJetty(AbstractFullDistribZkTestBase.java:490)
   [junit4]   2>                at 
org.apache.solr.BaseDistributedSearchTestCase.createJetty(BaseDistributedSearchTestCase.java:356)
   [junit4]   2>                at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.createServers(AbstractFullDistribZkTestBase.java:291)
   [junit4]   2>                at 
org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:868)
   [junit4]   2>                at 
sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source)
   [junit4]   2>                at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2>                at 
java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1617)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:826)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:862)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:876)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
   [junit4]   2>                at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2>                at 
org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
   [junit4]   2>                at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
   [junit4]   2>                at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2>                at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
   [junit4]   2>                at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:359)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:783)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:443)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:835)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:737)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:771)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:782)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
   [junit4]   2>                at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>                at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>                at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
   [junit4]   2>                at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>                at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
   [junit4]   2>                at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:359)
   [junit4]   2>                at java.lang.Thread.run(Thread.java:744)
   [junit4]   2>        Caused by: org.apache.solr.common.SolrException: Could 
not load solr.xml from zookeeper: node not found
   [junit4]   2>                at 
org.apache.solr.servlet.SolrDispatchFilter.loadConfigSolr(SolrDispatchFilter.java:177)
   [junit4]   2>                ... 58 more
   [junit4]   2>        
   [junit4]   2> 1641489 T7322 oass.SolrDispatchFilter.init 
SolrDispatchFilter.init() done
   [junit4]   2> 1641490 T7322 oascc.ConnectionManager.waitForConnected Waiting 
for client to connect to ZooKeeper
   [junit4]   2> 1641491 T7344 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@38ee7452 
name:ZooKeeperConnection Watcher:127.0.0.1:34283/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1641491 T7322 oascc.ConnectionManager.waitForConnected Client 
is connected to ZooKeeper
   [junit4]   2> 1641492 T7322 oascc.SolrZkClient.makePath makePath: 
/clusterstate.json
   [junit4]   2> 1641493 T7322 oascc.SolrZkClient.makePath makePath: 
/aliases.json
   [junit4]   2> 1641495 T7322 
oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state 
from ZooKeeper... 
   [junit4]   2> 1641496 T7322 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 
47403
   [junit4]   2> 1641517 T7322 oejsh.ContextHandler.doStop stopped 
o.e.j.s.ServletContextHandler{/gqmb,null}
   [junit4]   2> 1641570 T7322 oas.SolrTestCaseJ4.tearDown ###Ending 
testDistribSearch
   [junit4]   2> 1641572 T7322 oasc.ZkTestServer.send4LetterWord connecting to 
127.0.0.1:34283 34283
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> NOTE: reproduce with: ant test  
-Dtestcase=TestRequestStatusCollectionAPI -Dtests.method=testDistribSearch 
-Dtests.seed=BCC754BFE9236E64 -Dtests.multiplier=3 -Dtests.slow=true 
-Dtests.locale=de -Dtests.timezone=Africa/El_Aaiun -Dtests.file.encoding=UTF-8
   [junit4] ERROR   5.64s J0 | TestRequestStatusCollectionAPI.testDistribSearch 
<<<
   [junit4]    > Throwable #1: org.apache.solr.common.cloud.ZooKeeperException: 
   [junit4]    >        at 
__randomizedtesting.SeedInfo.seed([BCC754BFE9236E64:3D21DAA79E7C0E58]:0)
   [junit4]    >        at 
org.apache.solr.client.solrj.impl.CloudSolrServer.connect(CloudSolrServer.java:249)
   [junit4]    >        at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.initCloud(AbstractFullDistribZkTestBase.java:256)
   [junit4]    >        at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.createServers(AbstractFullDistribZkTestBase.java:312)
   [junit4]    >        at 
org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:868)
   [junit4]    >        at java.lang.Thread.run(Thread.java:744)
   [junit4]    > Caused by: 
org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode 
for /live_nodes
   [junit4]    >        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
   [junit4]    >        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]    >        at 
org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
   [junit4]    >        at 
org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:259)
   [junit4]    >        at 
org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:256)
   [junit4]    >        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]    >        at 
org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:256)
   [junit4]    >        at 
org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:299)
   [junit4]    >        at 
org.apache.solr.client.solrj.impl.CloudSolrServer.connect(CloudSolrServer.java:242)
   [junit4]    >        ... 41 more
   [junit4]   2> 1646712 T7322 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> 1646712 T7322 oas.SolrTestCaseJ4.endTrackingZkClients ERROR 
ERROR: SolrZkClient opens=4 closes=3
   [junit4]   2> 5640 T7321 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will 
linger awaiting termination of 2 leaked thread(s).
   [junit4]   2> 1646794 T7344 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@38ee7452 
name:ZooKeeperConnection Watcher:127.0.0.1:34283/solr got event WatchedEvent 
state:Disconnected type:None path:null path:null type:None
   [junit4]   2> 1646795 T7344 oascc.ConnectionManager.process zkClient has 
disconnected
   [junit4]   2> 1648261 T7343 oaz.ClientCnxn$SendThread.run WARN Session 
0x144d9194bb50003 for server null, unexpected error, closing socket connection 
and attempting reconnect java.net.ConnectException: Connection refused
   [junit4]   2>        at sun.nio.ch.SocketChannelImpl.checkConnect(Native 
Method)
   [junit4]   2>        at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1649933 T7343 oaz.ClientCnxn$SendThread.run WARN Session 
0x144d9194bb50003 for server null, unexpected error, closing socket connection 
and attempting reconnect java.net.ConnectException: Connection refused
   [junit4]   2>        at sun.nio.ch.SocketChannelImpl.checkConnect(Native 
Method)
   [junit4]   2>        at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1651545 T7343 oaz.ClientCnxn$SendThread.run WARN Session 
0x144d9194bb50003 for server null, unexpected error, closing socket connection 
and attempting reconnect java.net.ConnectException: Connection refused
   [junit4]   2>        at sun.nio.ch.SocketChannelImpl.checkConnect(Native 
Method)
   [junit4]   2>        at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1653330 T7343 oaz.ClientCnxn$SendThread.run WARN Session 
0x144d9194bb50003 for server null, unexpected error, closing socket connection 
and attempting reconnect java.net.ConnectException: Connection refused
   [junit4]   2>        at sun.nio.ch.SocketChannelImpl.checkConnect(Native 
Method)
   [junit4]   2>        at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1654620 T7343 oaz.ClientCnxn$SendThread.run WARN Session 
0x144d9194bb50003 for server null, unexpected error, closing socket connection 
and attempting reconnect java.net.ConnectException: Connection refused
   [junit4]   2>        at sun.nio.ch.SocketChannelImpl.checkConnect(Native 
Method)
   [junit4]   2>        at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1655835 T7343 oaz.ClientCnxn$SendThread.run WARN Session 
0x144d9194bb50003 for server null, unexpected error, closing socket connection 
and attempting reconnect java.net.ConnectException: Connection refused
   [junit4]   2>        at sun.nio.ch.SocketChannelImpl.checkConnect(Native 
Method)
   [junit4]   2>        at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1657319 T7343 oaz.ClientCnxn$SendThread.run WARN Session 
0x144d9194bb50003 for server null, unexpected error, closing socket connection 
and attempting reconnect java.net.ConnectException: Connection refused
   [junit4]   2>        at sun.nio.ch.SocketChannelImpl.checkConnect(Native 
Method)
   [junit4]   2>        at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1659153 T7343 oaz.ClientCnxn$SendThread.run WARN Session 
0x144d9194bb50003 for server null, unexpected error, closing socket connection 
and attempting reconnect java.net.ConnectException: Connection refused
   [junit4]   2>        at sun.nio.ch.SocketChannelImpl.checkConnect(Native 
Method)
   [junit4]   2>        at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1661089 T7343 oaz.ClientCnxn$SendThread.run WARN Session 
0x144d9194bb50003 for server null, unexpected error, closing socket connection 
and attempting reconnect java.net.ConnectException: Connection refused
   [junit4]   2>        at sun.nio.ch.SocketChannelImpl.checkConnect(Native 
Method)
   [junit4]   2>        at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1663183 T7343 oaz.ClientCnxn$SendThread.run WARN Session 
0x144d9194bb50003 for server null, unexpected error, closing socket connection 
and attempting reconnect java.net.ConnectException: Connection refused
   [junit4]   2>        at sun.nio.ch.SocketChannelImpl.checkConnect(Native 
Method)
   [junit4]   2>        at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1664617 T7343 oaz.ClientCnxn$SendThread.run WARN Session 
0x144d9194bb50003 for server null, unexpected error, closing socket connection 
and attempting reconnect java.net.ConnectException: Connection refused
   [junit4]   2>        at sun.nio.ch.SocketChannelImpl.checkConnect(Native 
Method)
   [junit4]   2>        at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1666127 T7343 oaz.ClientCnxn$SendThread.run WARN Session 
0x144d9194bb50003 for server null, unexpected error, closing socket connection 
and attempting reconnect java.net.ConnectException: Connection refused
   [junit4]   2>        at sun.nio.ch.SocketChannelImpl.checkConnect(Native 
Method)
   [junit4]   2>        at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 25771 T7321 ccr.ThreadLeakControl.checkThreadLeaks SEVERE 2 
threads leaked from SUITE scope at 
org.apache.solr.cloud.TestRequestStatusCollectionAPI: 
   [junit4]   2>           1) Thread[id=7343, 
name=TEST-TestRequestStatusCollectionAPI.testDistribSearch-seed#[BCC754BFE9236E64]-SendThread(localhost.localdomain:34283),
 state=TIMED_WAITING, group=TGRP-TestRequestStatusCollectionAPI]
   [junit4]   2>                at java.lang.Thread.sleep(Native Method)
   [junit4]   2>                at 
org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:86)
   [junit4]   2>                at 
org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:937)
   [junit4]   2>                at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993)
   [junit4]   2>           2) Thread[id=7344, 
name=TEST-TestRequestStatusCollectionAPI.testDistribSearch-seed#[BCC754BFE9236E64]-EventThread,
 state=WAITING, group=TGRP-TestRequestStatusCollectionAPI]
   [junit4]   2>                at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>                at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]   2>                at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]   2>                at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2>                at 
org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:491)
   [junit4]   2> 25772 T7321 ccr.ThreadLeakControl.tryToInterruptAll Starting 
to interrupt leaked threads:
   [junit4]   2>           1) Thread[id=7343, 
name=TEST-TestRequestStatusCollectionAPI.testDistribSearch-seed#[BCC754BFE9236E64]-SendThread(localhost.localdomain:34283),
 state=TIMED_WAITING, group=TGRP-TestRequestStatusCollectionAPI]
   [junit4]   2>           2) Thread[id=7344, 
name=TEST-TestRequestStatusCollectionAPI.testDistribSearch-seed#[BCC754BFE9236E64]-EventThread,
 state=WAITING, group=TGRP-TestRequestStatusCollectionAPI]
   [junit4]   2> 1666848 T7343 oazc.StaticHostProvider.next WARN Unexpected 
exception java.lang.InterruptedException: sleep interrupted
   [junit4]   2>        at java.lang.Thread.sleep(Native Method)
   [junit4]   2>        at 
org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:86)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:937)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993)
   [junit4]   2> 
   [junit4]   2> 1666848 T7344 oaz.ClientCnxn$EventThread.run ERROR Event 
thread exiting due to interruption java.lang.InterruptedException
   [junit4]   2>        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017)
   [junit4]   2>        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2052)
   [junit4]   2>        at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:491)
   [junit4]   2> 
   [junit4]   2> 1666850 T7343 oaz.ClientCnxn$SendThread.run WARN Session 
0x144d9194bb50003 for server null, unexpected error, closing socket connection 
and attempting reconnect java.net.ConnectException: Connection refused
   [junit4]   2>        at sun.nio.ch.SocketChannelImpl.checkConnect(Native 
Method)
   [junit4]   2>        at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1667349 T7343 oazc.StaticHostProvider.next WARN Unexpected 
exception java.lang.InterruptedException: sleep interrupted
   [junit4]   2>        at java.lang.Thread.sleep(Native Method)
   [junit4]   2>        at 
org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:86)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:937)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993)
   [junit4]   2> 
   [junit4]   2> 1667350 T7343 oaz.ClientCnxn$SendThread.run WARN Session 
0x144d9194bb50003 for server null, unexpected error, closing socket connection 
and attempting reconnect java.net.ConnectException: Connection refused
   [junit4]   2>        at sun.nio.ch.SocketChannelImpl.checkConnect(Native 
Method)
   [junit4]   2>        at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1667849 T7343 oazc.StaticHostProvider.next WARN Unexpected 
exception java.lang.InterruptedException: sleep interrupted
   [junit4]   2>        at java.lang.Thread.sleep(Native Method)
   [junit4]   2>        at 
org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:86)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:937)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993)
   [junit4]   2> 
   [junit4]   2> 1667850 T7343 oaz.ClientCnxn$SendThread.run WARN Session 
0x144d9194bb50003 for server null, unexpected error, closing socket connection 
and attempting reconnect java.net.ConnectException: Connection refused
   [junit4]   2>        at sun.nio.ch.SocketChannelImpl.checkConnect(Native 
Method)
   [junit4]   2>        at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1668349 T7343 oaz.ClientCnxn$SendThread.run WARN Unexpected 
exception java.lang.InterruptedException: sleep interrupted
   [junit4]   2>        at java.lang.Thread.sleep(Native Method)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:984)
   [junit4]   2> 
   [junit4]   2> 1668850 T7343 oazc.StaticHostProvider.next WARN Unexpected 
exception java.lang.InterruptedException: sleep interrupted
   [junit4]   2>        at java.lang.Thread.sleep(Native Method)
   [junit4]   2>        at 
org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:86)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:937)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993)
   [junit4]   2> 
   [junit4]   2> 1668851 T7343 oaz.ClientCnxn$SendThread.run WARN Session 
0x144d9194bb50003 for server null, unexpected error, closing socket connection 
and attempting reconnect java.net.ConnectException: Connection refused
   [junit4]   2>        at sun.nio.ch.SocketChannelImpl.checkConnect(Native 
Method)
   [junit4]   2>        at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1669349 T7343 oaz.ClientCnxn$SendThread.run WARN Unexpected 
exception java.lang.InterruptedException: sleep interrupted
   [junit4]   2>        at java.lang.Thread.sleep(Native Method)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:984)
   [junit4]   2> 
   [junit4]   2> 28774 T7321 ccr.ThreadLeakControl.tryToInterruptAll SEVERE 
There are still zombie threads that couldn't be terminated:
   [junit4]   2>           1) Thread[id=7343, 
name=TEST-TestRequestStatusCollectionAPI.testDistribSearch-seed#[BCC754BFE9236E64]-SendThread(localhost.localdomain:34283),
 state=TIMED_WAITING, group=TGRP-TestRequestStatusCollectionAPI]
   [junit4]   2>                at java.lang.Thread.sleep(Native Method)
   [junit4]   2>                at 
org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:86)
   [junit4]   2>                at 
org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:937)
   [junit4]   2>                at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993)
   [junit4]   2> NOTE: test params are: codec=Appending, sim=DefaultSimilarity, 
locale=de, timezone=Africa/El_Aaiun
   [junit4]   2> NOTE: Linux 3.8.0-37-generic amd64/Oracle Corporation 
1.7.0_60-ea (64-bit)/cpus=8,threads=2,free=241361336,total=406970368
   [junit4]   2> NOTE: All tests run in this JVM: [TestReloadAndDeleteDocs, 
SystemInfoHandlerTest, TestNonDefinedSimilarityFactory, AliasIntegrationTest, 
PreAnalyzedUpdateProcessorTest, BasicDistributedZkTest, PrimitiveFieldTypeTest, 
HdfsBasicDistributedZk2Test, RecoveryZkTest, DistributedTermsComponentTest, 
DocValuesMissingTest, SampleTest, DocValuesTest, 
HdfsWriteToMultipleCollectionsTest, TestUniqueKeyFieldResource, 
CoreAdminHandlerTest, TestWriterPerf, TestLMDirichletSimilarityFactory, 
TestSolrDeletionPolicy1, DOMUtilTest, ChangedSchemaMergeTest, 
RegexBoostProcessorTest, ShardRoutingTest, CustomCollectionTest, 
QueryEqualityTest, TestTrie, ZkControllerTest, TestCollationKeyRangeQueries, 
TestHighlightDedupGrouping, EchoParamsTest, DirectUpdateHandlerOptimizeTest, 
TriLevelCompositeIdRoutingTest, TestSchemaVersionResource, TestNonNRTOpen, 
UpdateParamsTest, TestReversedWildcardFilterFactory, 
TestSchemaSimilarityResource, TestInitQParser, 
SchemaVersionSpecificBehaviorTest, HdfsRecoveryZkTest, SolrTestCaseJ4Test, 
TestStressUserVersions, ShardSplitTest, TestConfig, TestNoOpRegenerator, 
ResponseLogComponentTest, DeleteShardTest, MBeansHandlerTest, 
TestRandomMergePolicy, IndexSchemaTest, TestBM25SimilarityFactory, 
SuggesterTest, TestSolrIndexConfig, TestLRUCache, 
DistributedSpellCheckComponentTest, TermVectorComponentTest, 
ShowFileRequestHandlerTest, SliceStateTest, 
DistributedQueryElevationComponentTest, TestJoin, DocumentBuilderTest, 
CursorMarkTest, QueryParsingTest, TestDocumentBuilder, BasicFunctionalityTest, 
ConnectionManagerTest, TestRealTimeGet, AutoCommitTest, 
TestShortCircuitedRequests, TestCopyFieldCollectionResource, 
TestLuceneMatchVersion, QueryResultKeyTest, JsonLoaderTest, LoggingHandlerTest, 
SolrIndexSplitterTest, ZkCLITest, SynonymTokenizerTest, 
FastVectorHighlighterTest, BinaryUpdateRequestHandlerTest, TestManagedSchema, 
OverseerTest, TestShardHandlerFactory, TestPseudoReturnFields, 
TestSolrXMLSerializer, TestSolrQueryParser, CursorPagingTest, BadCopyFieldTest, 
ResourceLoaderTest, LukeRequestHandlerTest, ChaosMonkeySafeLeaderTest, 
BasicDistributedZk2Test, UnloadDistributedZkTest, 
CollectionsAPIDistributedZkTest, SyncSliceTest, TestReplicationHandler, 
LeaderElectionTest, TestZkChroot, ShardRoutingCustomTest, SliceStateUpdateTest, 
TestFaceting, TestRecovery, TestStressReorder, TestReload, HardAutoCommitTest, 
TestCoreContainer, SolrCoreTest, TestGroupingSearch, TestFunctionQuery, 
TestLazyCores, OverseerCollectionProcessorTest, SoftAutoCommitTest, 
SpatialFilterTest, WordBreakSolrSpellCheckerTest, TestAtomicUpdateErrorCases, 
StatelessScriptUpdateProcessorFactoryTest, DocValuesMultiTest, 
XsltUpdateRequestHandlerTest, IndexBasedSpellCheckerTest, 
DirectSolrSpellCheckerTest, TestOmitPositions, TermsComponentTest, 
MoreLikeThisHandlerTest, TestCollationField, ReturnFieldsTest, 
TestAnalyzedSuggestions, TestPHPSerializedResponseWriter, 
CSVRequestHandlerTest, SearchHandlerTest, TestLFUCache, SOLR749Test, 
CopyFieldTest, TestMergePolicyConfig, TestBinaryField, MinimalSchemaTest, 
OutputWriterTest, DirectSolrConnectionTest, TestPhraseSuggestions, 
TestCharFilters, TestPerFieldSimilarity, TestFastWriter, TestJmxMonitoredMap, 
ScriptEngineTest, PluginInfoTest, DateMathParserTest, PrimUtilsTest, 
ClusterStateTest, TestUtils, CircularListTest, TestCursorMarkWithoutUniqueKey, 
AssignTest, CollectionsAPIAsyncDistributedZkTest, DeleteReplicaTest, 
MigrateRouteKeyTest, RemoteQueryErrorTest, TestRequestStatusCollectionAPI]
   [junit4]   2> NOTE: reproduce with: ant test  
-Dtestcase=TestRequestStatusCollectionAPI -Dtests.seed=BCC754BFE9236E64 
-Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=de 
-Dtests.timezone=Africa/El_Aaiun -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.00s J0 | TestRequestStatusCollectionAPI (suite) <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: ERROR: SolrZkClient 
opens=4 closes=3
   [junit4]    >        at 
__randomizedtesting.SeedInfo.seed([BCC754BFE9236E64]:0)
   [junit4]    >        at 
org.apache.solr.SolrTestCaseJ4.endTrackingZkClients(SolrTestCaseJ4.java:420)
   [junit4]    >        at 
org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:177)
   [junit4]    >        at java.lang.Thread.run(Thread.java:744)Throwable #2: 
com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE 
scope at org.apache.solr.cloud.TestRequestStatusCollectionAPI: 
   [junit4]    >    1) Thread[id=7343, 
name=TEST-TestRequestStatusCollectionAPI.testDistribSearch-seed#[BCC754BFE9236E64]-SendThread(localhost.localdomain:34283),
 state=TIMED_WAITING, group=TGRP-TestRequestStatusCollectionAPI]
   [junit4]    >         at java.lang.Thread.sleep(Native Method)
   [junit4]    >         at 
org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:86)
   [junit4]    >         at 
org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:937)
   [junit4]    >         at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993)
   [junit4]    >    2) Thread[id=7344, 
name=TEST-TestRequestStatusCollectionAPI.testDistribSearch-seed#[BCC754BFE9236E64]-EventThread,
 state=WAITING, group=TGRP-TestRequestStatusCollectionAPI]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at 
org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:491)
   [junit4]    >        at 
__randomizedtesting.SeedInfo.seed([BCC754BFE9236E64]:0)Throwable #3: 
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie 
threads that couldn't be terminated:
   [junit4]    >    1) Thread[id=7343, 
name=TEST-TestRequestStatusCollectionAPI.testDistribSearch-seed#[BCC754BFE9236E64]-SendThread(localhost.localdomain:34283),
 state=TIMED_WAITING, group=TGRP-TestRequestStatusCollectionAPI]
   [junit4]    >         at java.lang.Thread.sleep(Native Method)
   [junit4]    >         at 
org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:86)
   [junit4]    >         at 
org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:937)
   [junit4]    >         at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993)
   [junit4]    >        at 
__randomizedtesting.SeedInfo.seed([BCC754BFE9236E64]:0)
   [junit4] Completed on J0 in 28.82s, 1 test, 1 failure, 3 errors <<< FAILURES!

[...truncated 119 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:467: The following 
error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:447: The following 
error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:45: The following 
error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/extra-targets.xml:37: The 
following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build.xml:189: The 
following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/common-build.xml:490: The 
following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:1276: 
The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:908: 
There were test failures: 377 suites, 1597 tests, 3 suite-level errors, 1 
error, 158 ignored (16 assumptions)

Total time: 55 minutes 28 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.7.0_60-ea-b10 -XX:-UseCompressedOops 
-XX:+UseSerialGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to