[JENKINS] Lucene-Solr-master-Windows (64bit/jdk1.8.0_144) - Build # 6843 - Still Unstable!

Wed, 23 Aug 2017 23:35:46 -0700 

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Windows/6843/
Java: 64bit/jdk1.8.0_144 -XX:+UseCompressedOops -XX:+UseParallelGC

5 tests failed.
FAILED:  org.apache.solr.metrics.reporters.SolrSlf4jReporterTest.testReporter

Error Message:
No 'solr.node' logs in: {numFound=0,start=0,docs=[]}

Stack Trace:
java.lang.AssertionError: No 'solr.node' logs in: {numFound=0,start=0,docs=[]}
        at 
__randomizedtesting.SeedInfo.seed([1685ACBEFE763618:49618189957AA55D]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at 
org.apache.solr.metrics.reporters.SolrSlf4jReporterTest.testReporter(SolrSlf4jReporterTest.java:85)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  
org.apache.solr.client.solrj.io.stream.StreamExpressionTest.testParallelExecutorStream

Error Message:
Error from server at http://127.0.0.1:53645/solr/workQueue_shard2_replica_n3: 
Expected mime type application/octet-stream but got text/html. <html> <head> 
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/> 
<title>Error 404 </title> </head> <body> <h2>HTTP ERROR: 404</h2> <p>Problem 
accessing /solr/workQueue_shard2_replica_n3/update. Reason: <pre>    Can not 
find: /solr/workQueue_shard2_replica_n3/update</pre></p> <hr /><a 
href="http://eclipse.org/jetty";>Powered by Jetty:// 9.3.20.v20170531</a><hr/> 
</body> </html> 

Stack Trace:
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: Error from 
server at http://127.0.0.1:53645/solr/workQueue_shard2_replica_n3: Expected 
mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 404 </title>
</head>
<body>
<h2>HTTP ERROR: 404</h2>
<p>Problem accessing /solr/workQueue_shard2_replica_n3/update. Reason:
<pre>    Can not find: /solr/workQueue_shard2_replica_n3/update</pre></p>
<hr /><a href="http://eclipse.org/jetty";>Powered by Jetty:// 
9.3.20.v20170531</a><hr/>
</body>
</html>

        at 
__randomizedtesting.SeedInfo.seed([CA26E6DC85B2322F:773193C5BC9E0F72]:0)
        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:539)
        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:993)
        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:862)
        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:793)
        at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
        at 
org.apache.solr.client.solrj.request.UpdateRequest.commit(UpdateRequest.java:233)
        at 
org.apache.solr.client.solrj.io.stream.StreamExpressionTest.testParallelExecutorStream(StreamExpressionTest.java:6884)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)
Caused by: 
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at http://127.0.0.1:53645/solr/workQueue_shard2_replica_n3: 
Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 404 </title>
</head>
<body>
<h2>HTTP ERROR: 404</h2>
<p>Problem accessing /solr/workQueue_shard2_replica_n3/update. Reason:
<pre>    Can not find: /solr/workQueue_shard2_replica_n3/update</pre></p>
<hr /><a href="http://eclipse.org/jetty";>Powered by Jetty:// 
9.3.20.v20170531</a><hr/>
</body>
</html>

        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:591)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:253)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242)
        at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.lambda$directUpdate$0(CloudSolrClient.java:516)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        ... 1 more


FAILED:  org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.test

Error Message:
Timeout occured while waiting response from server at: 
http://127.0.0.1:50569/_g/wz/collection1

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting 
response from server at: http://127.0.0.1:50569/_g/wz/collection1
        at 
__randomizedtesting.SeedInfo.seed([1685ACBEFE763618:9ED19364508A5BE0]:0)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:638)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:253)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242)
        at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1121)
        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:862)
        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:793)
        at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
        at org.apache.solr.client.solrj.SolrClient.commit(SolrClient.java:484)
        at org.apache.solr.client.solrj.SolrClient.commit(SolrClient.java:463)
        at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.commit(AbstractFullDistribZkTestBase.java:1583)
        at 
org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.test(ChaosMonkeyNothingIsSafeTest.java:213)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketTimeoutException: Read timed out
        at java.net.SocketInputStream.socketRead0(Native Method)
        at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
        at java.net.SocketInputStream.read(SocketInputStream.java:171)
        at java.net.SocketInputStream.read(SocketInputStream.java:141)
        at 
org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139)
        at 
org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155)
        at 
org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284)
        at 
org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
        at 
org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
        at 
org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
        at 
org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165)
        at 
org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:167)
        at 
org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
        at 
org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
        at 
org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:271)
        at 
org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
        at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
        at 
org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
        at 
org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
        at 
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
        at 
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:526)
        ... 53 more


FAILED:  
junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest

Error Message:
9 threads leaked from SUITE scope at 
org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest:     1) Thread[id=22226, 
name=zkCallback-3515-thread-4, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeyNothingIsSafeTest]         at sun.misc.Unsafe.park(Native 
Method)         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         
at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
         at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
         at 
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         
at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)   
      at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
        at java.lang.Thread.run(Thread.java:748)    2) Thread[id=22078, 
name=TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[1685ACBEFE763618]-SendThread(127.0.0.1:50527),
 state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1051)    3) 
Thread[id=22077, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeyNothingIsSafeTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:748)    4) Thread[id=22223, 
name=zkCallback-3515-thread-2, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeyNothingIsSafeTest]         at sun.misc.Unsafe.park(Native 
Method)         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         
at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
         at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
         at 
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         
at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)   
      at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
        at java.lang.Thread.run(Thread.java:748)    5) Thread[id=22280, 
name=zkCallback-3515-thread-6, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeyNothingIsSafeTest]         at sun.misc.Unsafe.park(Native 
Method)         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         
at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
         at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
         at 
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         
at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)   
      at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
        at java.lang.Thread.run(Thread.java:748)    6) Thread[id=22080, 
name=zkCallback-3515-thread-1, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeyNothingIsSafeTest]         at sun.misc.Unsafe.park(Native 
Method)         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         
at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
         at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
         at 
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         
at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)   
      at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
        at java.lang.Thread.run(Thread.java:748)    7) Thread[id=22079, 
name=TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[1685ACBEFE763618]-EventThread,
 state=WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]         at 
sun.misc.Unsafe.park(Native Method)         at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
         at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)     
    at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501)    
8) Thread[id=22227, name=zkCallback-3515-thread-5, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeyNothingIsSafeTest]         at sun.misc.Unsafe.park(Native 
Method)         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         
at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
         at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
         at 
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         
at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)   
      at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
        at java.lang.Thread.run(Thread.java:748)    9) Thread[id=22225, 
name=zkCallback-3515-thread-3, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeyNothingIsSafeTest]         at sun.misc.Unsafe.park(Native 
Method)         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         
at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
         at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
         at 
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         
at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)   
      at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
        at java.lang.Thread.run(Thread.java:748)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 9 threads leaked from SUITE 
scope at org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest: 
   1) Thread[id=22226, name=zkCallback-3515-thread-4, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeyNothingIsSafeTest]
        at sun.misc.Unsafe.park(Native Method)
        at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
   2) Thread[id=22078, 
name=TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[1685ACBEFE763618]-SendThread(127.0.0.1:50527),
 state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1051)
   3) Thread[id=22077, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeyNothingIsSafeTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:748)
   4) Thread[id=22223, name=zkCallback-3515-thread-2, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeyNothingIsSafeTest]
        at sun.misc.Unsafe.park(Native Method)
        at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
   5) Thread[id=22280, name=zkCallback-3515-thread-6, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeyNothingIsSafeTest]
        at sun.misc.Unsafe.park(Native Method)
        at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
   6) Thread[id=22080, name=zkCallback-3515-thread-1, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeyNothingIsSafeTest]
        at sun.misc.Unsafe.park(Native Method)
        at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
   7) Thread[id=22079, 
name=TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[1685ACBEFE763618]-EventThread,
 state=WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501)
   8) Thread[id=22227, name=zkCallback-3515-thread-5, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeyNothingIsSafeTest]
        at sun.misc.Unsafe.park(Native Method)
        at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
   9) Thread[id=22225, name=zkCallback-3515-thread-3, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeyNothingIsSafeTest]
        at sun.misc.Unsafe.park(Native Method)
        at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
        at __randomizedtesting.SeedInfo.seed([1685ACBEFE763618]:0)


FAILED:  
junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) 
Thread[id=22226, name=zkCallback-3515-thread-4, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeyNothingIsSafeTest]         at sun.misc.Unsafe.park(Native 
Method)         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         
at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
         at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
         at 
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         
at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)   
      at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
        at java.lang.Thread.run(Thread.java:748)    2) Thread[id=22078, 
name=TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[1685ACBEFE763618]-SendThread(127.0.0.1:50527),
 state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1051)    3) 
Thread[id=22223, name=zkCallback-3515-thread-2, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeyNothingIsSafeTest]         at sun.misc.Unsafe.park(Native 
Method)         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         
at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
         at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
         at 
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         
at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)   
      at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
        at java.lang.Thread.run(Thread.java:748)    4) Thread[id=22280, 
name=zkCallback-3515-thread-6, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeyNothingIsSafeTest]         at sun.misc.Unsafe.park(Native 
Method)         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         
at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
         at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
         at 
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         
at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)   
      at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
        at java.lang.Thread.run(Thread.java:748)    5) Thread[id=22080, 
name=zkCallback-3515-thread-1, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeyNothingIsSafeTest]         at sun.misc.Unsafe.park(Native 
Method)         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         
at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
         at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
         at 
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         
at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)   
      at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
        at java.lang.Thread.run(Thread.java:748)    6) Thread[id=22227, 
name=zkCallback-3515-thread-5, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeyNothingIsSafeTest]         at sun.misc.Unsafe.park(Native 
Method)         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         
at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
         at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
         at 
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         
at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)   
      at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
        at java.lang.Thread.run(Thread.java:748)    7) Thread[id=22225, 
name=zkCallback-3515-thread-3, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeyNothingIsSafeTest]         at sun.misc.Unsafe.park(Native 
Method)         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         
at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
         at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
         at 
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         
at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)   
      at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
        at java.lang.Thread.run(Thread.java:748)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie 
threads that couldn't be terminated:
   1) Thread[id=22226, name=zkCallback-3515-thread-4, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeyNothingIsSafeTest]
        at sun.misc.Unsafe.park(Native Method)
        at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
   2) Thread[id=22078, 
name=TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[1685ACBEFE763618]-SendThread(127.0.0.1:50527),
 state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1051)
   3) Thread[id=22223, name=zkCallback-3515-thread-2, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeyNothingIsSafeTest]
        at sun.misc.Unsafe.park(Native Method)
        at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
   4) Thread[id=22280, name=zkCallback-3515-thread-6, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeyNothingIsSafeTest]
        at sun.misc.Unsafe.park(Native Method)
        at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
   5) Thread[id=22080, name=zkCallback-3515-thread-1, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeyNothingIsSafeTest]
        at sun.misc.Unsafe.park(Native Method)
        at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
   6) Thread[id=22227, name=zkCallback-3515-thread-5, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeyNothingIsSafeTest]
        at sun.misc.Unsafe.park(Native Method)
        at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
   7) Thread[id=22225, name=zkCallback-3515-thread-3, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeyNothingIsSafeTest]
        at sun.misc.Unsafe.park(Native Method)
        at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
        at __randomizedtesting.SeedInfo.seed([1685ACBEFE763618]:0)




Build Log:
[...truncated 11073 lines...]
   [junit4] Suite: org.apache.solr.metrics.reporters.SolrSlf4jReporterTest
   [junit4]   2> Creating dataDir: 
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.metrics.reporters.SolrSlf4jReporterTest_1685ACBEFE763618-001\init-core-data-001
   [junit4]   2> 593972 WARN  
(SUITE-SolrSlf4jReporterTest-seed#[1685ACBEFE763618]-worker) [    ] 
o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
   [junit4]   2> NOTE: reproduce with: ant test  
-Dtestcase=SolrSlf4jReporterTest -Dtests.method=testReporter 
-Dtests.seed=1685ACBEFE763618 -Dtests.slow=true -Dtests.locale=he-IL 
-Dtests.timezone=CET -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] FAILURE 6.41s J1 | SolrSlf4jReporterTest.testReporter <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: No 'solr.node' logs 
in: {numFound=0,start=0,docs=[]}
   [junit4]    >        at 
__randomizedtesting.SeedInfo.seed([1685ACBEFE763618:49618189957AA55D]:0)
   [junit4]    >        at 
org.apache.solr.metrics.reporters.SolrSlf4jReporterTest.testReporter(SolrSlf4jReporterTest.java:85)
   [junit4]    >        at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> NOTE: leaving temporary files on disk at: 
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.metrics.reporters.SolrSlf4jReporterTest_1685ACBEFE763618-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {}, 
docValues:{}, maxPointsInLeafNode=254, maxMBSortInHeap=6.0207371704754, 
sim=RandomSimilarity(queryNorm=false): {}, locale=he-IL, timezone=CET
   [junit4]   2> NOTE: Windows 10 10.0 amd64/Oracle Corporation 1.8.0_144 
(64-bit)/cpus=3,threads=1,free=106395432,total=361758720
   [junit4]   2> NOTE: All tests run in this JVM: [QueryEqualityTest, TestTrie, 
ConnectionManagerTest, TestCrossCoreJoin, TestNumericTerms64, 
UpdateRequestProcessorFactoryTest, PingRequestHandlerTest, DocumentBuilderTest, 
ChaosMonkeySafeLeaderTest, BaseCdcrDistributedZkTest, SmileWriterTest, 
SolrJmxReporterCloudTest, TestLMJelinekMercerSimilarityFactory, 
TestPayloadCheckQParserPlugin, TestSchemaSimilarityResource, 
AddBlockUpdateTest, HdfsRestartWhileUpdatingTest, TestLegacyFieldCache, 
TestComponentsName, TestRealTimeGet, AssignTest, 
TermVectorComponentDistributedTest, DistributedQueueTest, 
LeaderFailureAfterFreshStartTest, DirectoryFactoryTest, 
SchemaVersionSpecificBehaviorTest, TestReplicaProperties, 
TestSolrCoreSnapshots, OpenExchangeRatesOrgProviderTest, TestStressLucene, 
ConfigureRecoveryStrategyTest, TestStressCloudBlindAtomicUpdates, 
TestUseDocValuesAsStored2, TestDownShardTolerantSearch, 
ClassificationUpdateProcessorTest, CloudMLTQParserTest, TestConfigSets, 
TestRequestStatusCollectionAPI, TestMacros, SolrCmdDistributorTest, 
PeerSyncWithIndexFingerprintCachingTest, V2ApiIntegrationTest, 
HdfsDirectoryFactoryTest, SystemInfoHandlerTest, SearchHandlerTest, 
MetricsHandlerTest, TestFastWriter, TestPivotHelperCode, LargeFieldTest, 
TestConfig, AtomicUpdateProcessorFactoryTest, TestSubQueryTransformer, 
TestManagedResourceStorage, SuggesterFSTTest, TestQueryTypes, 
SolrCLIZkUtilsTest, TestHashQParserPlugin, TestFieldCacheSort, ZkNodePropsTest, 
TestMissingGroups, TestCoreDiscovery, TestFastOutputStream, 
TermVectorComponentTest, BJQParserTest, TestPKIAuthenticationPlugin, 
DistributedFacetPivotLargeTest, ReplaceNodeTest, AutoCommitTest, 
JavabinLoaderTest, SearchHandlerTest, TestRecoveryHdfs, ConvertedLegacyTest, 
TestPullReplicaErrorHandling, TestStressLiveNodes, PolyFieldTest, 
DeleteLastCustomShardedReplicaTest, SuggestComponentTest, TestCustomSort, 
SSLMigrationTest, TestFieldTypeCollectionResource, TestManagedSchemaAPI, 
TestDocSet, ParsingFieldUpdateProcessorsTest, CollectionsAPIDistributedZkTest, 
TestLegacyTerms, GraphQueryTest, TestOrdValues, TestRemoteStreaming, 
SpellCheckComponentTest, ReplicaListTransformerTest, 
PKIAuthenticationIntegrationTest, TestSolrXml, 
DistribDocExpirationUpdateProcessorTest, CdcrReplicationDistributedZkTest, 
PreAnalyzedFieldTest, RulesTest, TestFreeTextSuggestions, SolrSlf4jReporterTest]
   [junit4] Completed [110/732 (1!)] on J1 in 6.46s, 1 test, 1 failure <<< 
FAILURES!

[...truncated 1159 lines...]
   [junit4] Suite: org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest
   [junit4]   2> Creating dataDir: 
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.ChaosMonkeyNothingIsSafeTest_1685ACBEFE763618-001\init-core-data-001
   [junit4]   2> 1708130 INFO  
(SUITE-ChaosMonkeyNothingIsSafeTest-seed#[1685ACBEFE763618]-worker) [    ] 
o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) 
w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 1708133 INFO  
(SUITE-ChaosMonkeyNothingIsSafeTest-seed#[1685ACBEFE763618]-worker) [    ] 
o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: 
@org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
   [junit4]   2> 1708133 INFO  
(SUITE-ChaosMonkeyNothingIsSafeTest-seed#[1685ACBEFE763618]-worker) [    ] 
o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: 
test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 1708143 ERROR (Thread-4330) [    ] o.a.z.s.ZooKeeperServer 
ZKShutdownHandler is not registered, so ZooKeeper server won't take any action 
on ERROR or SHUTDOWN server state changes
   [junit4]   2> 1708294 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] 
o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from 
client sessionid 0x15e12cf4ddd0001, likely client has closed socket
   [junit4]   2>        at 
org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
   [junit4]   2>        at 
org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 1708710 ERROR 
(TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[1685ACBEFE763618]) [    ] 
o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 1708733 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] 
o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from 
client sessionid 0x15e12cf4ddd0003, likely client has closed socket
   [junit4]   2>        at 
org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
   [junit4]   2>        at 
org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 1710183 WARN  (qtp1428370184-22045) [n:127.0.0.1:50534__g%2Fwz 
c:control_collection s:shard1  x:control_collection_shard1_replica_n1] 
o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = 
requestHandler,name = /dump,class = DumpRequestHandler,attributes = 
{initParams=a, name=/dump, class=DumpRequestHandler},args = 
{defaults={a=A,b=B}}}
   [junit4]   2> 1711420 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] 
o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from 
client sessionid 0x15e12cf4ddd0005, likely client has closed socket
   [junit4]   2>        at 
org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
   [junit4]   2>        at 
org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 1711445 WARN  
(OverseerThreadFactory-9160-thread-2-processing-n:127.0.0.1:50534__g%2Fwz) 
[n:127.0.0.1:50534__g%2Fwz    ] o.a.s.c.CreateCollectionCmd It is unusual to 
create a collection (collection1) without cores.
   [junit4]   2> 1712072 ERROR 
(TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[1685ACBEFE763618]) [    ] 
o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 1713429 WARN  (qtp1144678291-22086) [n:127.0.0.1:50555__g%2Fwz 
c:collection1 s:shard1  x:collection1_shard1_replica_t21] 
o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = 
requestHandler,name = /dump,class = DumpRequestHandler,attributes = 
{initParams=a, name=/dump, class=DumpRequestHandler},args = 
{defaults={a=A,b=B}}}
   [junit4]   2> 1714050 ERROR 
(TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[1685ACBEFE763618]) [    ] 
o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 1715368 WARN  (qtp1609904884-22113) [n:127.0.0.1:50569__g%2Fwz 
c:collection1 s:shard1  x:collection1_shard1_replica_t23] 
o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = 
requestHandler,name = /dump,class = DumpRequestHandler,attributes = 
{initParams=a, name=/dump, class=DumpRequestHandler},args = 
{defaults={a=A,b=B}}}
   [junit4]   2> 1715820 ERROR 
(TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[1685ACBEFE763618]) [    ] 
o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 1717088 WARN  (qtp1174853924-22145) [n:127.0.0.1:50583__g%2Fwz 
c:collection1 s:shard1  x:collection1_shard1_replica_t25] 
o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = 
requestHandler,name = /dump,class = DumpRequestHandler,attributes = 
{initParams=a, name=/dump, class=DumpRequestHandler},args = 
{defaults={a=A,b=B}}}
   [junit4]   2> 1717164 INFO  
(TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[1685ACBEFE763618]) [    ] 
o.a.s.SolrTestCaseJ4 ###Starting test
   [junit4]   2> 1728870 ERROR (StoppableIndexingThread) [    ] 
o.a.s.c.s.i.CloudSolrClient Request to collection collection1 failed due to (0) 
org.apache.http.NoHttpResponseException: 127.0.0.1:50555 failed to respond, 
retry? 0
   [junit4]   2> REQUEST FAILED for id=0-3312
   [junit4]   2> java.lang.NullPointerException
   [junit4]   2>        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:912)
   [junit4]   2>        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:793)
   [junit4]   2>        at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
   [junit4]   2>        at 
org.apache.solr.client.solrj.SolrClient.deleteById(SolrClient.java:753)
   [junit4]   2>        at 
org.apache.solr.client.solrj.SolrClient.deleteById(SolrClient.java:716)
   [junit4]   2>        at 
org.apache.solr.client.solrj.SolrClient.deleteById(SolrClient.java:731)
   [junit4]   2>        at 
org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:93)
   [junit4]   2> 1728878 ERROR (StoppableIndexingThread) [    ] 
o.a.s.c.s.i.CloudSolrClient Request to collection collection1 failed due to (0) 
java.net.SocketException: Software caused connection abort: recv failed, retry? 0
   [junit4]   2> REQUEST FAILED for id=0-3426
   [junit4]   2> java.lang.NullPointerException
   [junit4]   2>        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:912)
   [junit4]   2>        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:793)
   [junit4]   2>        at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
   [junit4]   2>        at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:195)
   [junit4]   2>        at 
org.apache.solr.cloud.StoppableIndexingThread.indexDocs(StoppableIndexingThread.java:181)
   [junit4]   2>        at 
org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:115)
   [junit4]   2> 1729149 WARN  
(zkCallback-3520-thread-1-processing-n:127.0.0.1:50555__g%2Fwz) 
[n:127.0.0.1:50555__g%2Fwz    ] o.a.s.c.c.ZkStateReader ZooKeeper watch 
triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for 
/live_nodes]
   [junit4]   2> 1729156 ERROR 
(TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[1685ACBEFE763618]) [    ] 
o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper 
server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 1729163 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] 
o.a.z.s.NIOServerCnxn Exception causing close of session 0x15e12cf4ddd000c due 
to java.io.IOException: ZooKeeperServer not running
   [junit4]   2> 1729172 WARN  (Thread-4330) [    ] o.a.s.c.ZkTestServer Watch 
limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2>        7       /solr/aliases.json
   [junit4]   2>        4       /solr/security.json
   [junit4]   2>        4       /solr/configs/conf1
   [junit4]   2>        3       /solr/collections/collection1/state.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2>        7       /solr/clusterstate.json
   [junit4]   2>        7       /solr/clusterprops.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2>        7       /solr/live_nodes
   [junit4]   2>        7       /solr/collections
   [junit4]   2> 
   [junit4]   2> 1729264 WARN  
(zkCallback-3526-thread-4-processing-n:127.0.0.1:50569__g%2Fwz) 
[n:127.0.0.1:50569__g%2Fwz    ] o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@5c23a501 name: 
ZooKeeperConnection Watcher:127.0.0.1:50527/solr got event WatchedEvent 
state:Disconnected type:None path:null path: null type: None
   [junit4]   2> 1729265 WARN  
(zkCallback-3526-thread-4-processing-n:127.0.0.1:50569__g%2Fwz) 
[n:127.0.0.1:50569__g%2Fwz    ] o.a.s.c.c.ConnectionManager zkClient has 
disconnected
   [junit4]   2> 1729273 WARN  
(zkCallback-3508-thread-6-processing-n:127.0.0.1:50534__g%2Fwz) 
[n:127.0.0.1:50534__g%2Fwz    ] o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@14dd5b6a name: 
ZooKeeperConnection Watcher:127.0.0.1:50527/solr got event WatchedEvent 
state:Disconnected type:None path:null path: null type: None
   [junit4]   2> 1729273 WARN  
(zkCallback-3508-thread-6-processing-n:127.0.0.1:50534__g%2Fwz) 
[n:127.0.0.1:50534__g%2Fwz    ] o.a.s.c.c.ConnectionManager zkClient has 
disconnected
   [junit4]   2> 1729274 WARN  (zkCallback-3515-thread-3) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@6d6626c name: 
ZooKeeperConnection Watcher:127.0.0.1:50527/solr got event WatchedEvent 
state:Disconnected type:None path:null path: null type: None
   [junit4]   2> 1729274 WARN  (zkCallback-3515-thread-3) [    ] 
o.a.s.c.c.ConnectionManager zkClient has disconnected
   [junit4]   2> 1729274 WARN  (zkCallback-3538-thread-3) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@21681142 name: 
ZooKeeperConnection Watcher:127.0.0.1:50527/solr got event WatchedEvent 
state:Disconnected type:None path:null path: null type: None
   [junit4]   2> 1729274 WARN  (zkCallback-3538-thread-3) [    ] 
o.a.s.c.c.ConnectionManager zkClient has disconnected
   [junit4]   2> 1729275 WARN  
(zkCallback-3533-thread-10-processing-n:127.0.0.1:50583__g%2Fwz) 
[n:127.0.0.1:50583__g%2Fwz    ] o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@715bcbb3 name: 
ZooKeeperConnection Watcher:127.0.0.1:50527/solr got event WatchedEvent 
state:Disconnected type:None path:null path: null type: None
   [junit4]   2> 1729276 WARN  
(zkCallback-3533-thread-10-processing-n:127.0.0.1:50583__g%2Fwz) 
[n:127.0.0.1:50583__g%2Fwz    ] o.a.s.c.c.ConnectionManager zkClient has 
disconnected
   [junit4]   2> 1731603 WARN  
(TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[1685ACBEFE763618]-SendThread(127.0.0.1:50527))
 [n:127.0.0.1:50583__g%2Fwz    ] o.a.z.ClientCnxn Session 0x15e12cf4ddd000f for 
server null, unexpected error, closing socket connection and attempting 
reconnect
   [junit4]   2> java.net.ConnectException: Connection refused: no further 
information
   [junit4]   2>        at sun.nio.ch.SocketChannelImpl.checkConnect(Native 
Method)
   [junit4]   2>        at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1141)
   [junit4]   2> 1731877 WARN  
(TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[1685ACBEFE763618]-SendThread(127.0.0.1:50527))
 [n:127.0.0.1:50534__g%2Fwz    ] o.a.z.ClientCnxn Session 0x15e12cf4ddd0004 for 
server null, unexpected error, closing socket connection and attempting 
reconnect
   [junit4]   2> java.net.ConnectException: Connection refused: no further 
information
   [junit4]   2>        at sun.nio.ch.SocketChannelImpl.checkConnect(Native 
Method)
   [junit4]   2>        at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1141)
   [junit4]   2> 1731994 WARN  
(TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[1685ACBEFE763618]-SendThread(127.0.0.1:50527))
 [    ] o.a.z.ClientCnxn Session 0x15e12cf4ddd0010 for server null, unexpected 
error, closing socket connection and attempting reconnect
   [junit4]   2> java.net.ConnectException: Connection refused: no further 
information
   [junit4]   2>        at sun.nio.ch.SocketChannelImpl.checkConnect(Native 
Method)
   [junit4]   2>        at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1141)
   [junit4]   2> 1732030 WARN  
(TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[1685ACBEFE763618]-SendThread(127.0.0.1:50527))
 [    ] o.a.z.ClientCnxn Session 0x15e12cf4ddd0006 for server null, unexpected 
error, closing socket connection and attempting reconnect
   [junit4]   2> java.net.ConnectException: Connection refused: no further 
information
   [junit4]   2>        at sun.nio.ch.SocketChannelImpl.checkConnect(Native 
Method)
   [junit4]   2>        at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1141)
   [junit4]   2> 1732201 WARN  
(TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[1685ACBEFE763618]-SendThread(127.0.0.1:50527))
 [n:127.0.0.1:50569__g%2Fwz    ] o.a.z.ClientCnxn Session 0x15e12cf4ddd000c for 
server null, unexpected error, closing socket connection and attempting 
reconnect
   [junit4]   2> java.net.ConnectException: Connection refused: no further 
information
   [junit4]   2>        at sun.nio.ch.SocketChannelImpl.checkConnect(Native 
Method)
   [junit4]   2>        at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1141)
   [junit4]   2> 1733790 WARN  
(TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[1685ACBEFE763618]-SendThread(127.0.0.1:50527))
 [n:127.0.0.1:50583__g%2Fwz    ] o.a.z.ClientCnxn Session 0x15e12cf4ddd000f for 
server null, unexpected error, closing socket connection and attempting 
reconnect
   [junit4]   2> java.net.ConnectException: Connection refused: no further 
information
   [junit4]   2>        at sun.nio.ch.SocketChannelImpl.checkConnect(Native 
Method)
   [junit4]   2>        at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1141)
   [junit4]   2> 1734248 ERROR (qtp1609904884-22117) [n:127.0.0.1:50569__g%2Fwz 
c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_t23] 
o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: No registered 
leader was found after waiting for 4000ms , collection: collection1 slice: 
shard1
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:758)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:744)
   [junit4]   2>        at 
org.apache.solr.update.processor.DistributedUpdateProcessor.setupRequest(DistributedUpdateProcessor.java:404)
   [junit4]   2>        at 
org.apache.solr.update.processor.DistributedUpdateProcessor.processDelete(DistributedUpdateProcessor.java:1410)
   [junit4]   2>        at 
org.apache.solr.handler.loader.JavabinLoader.delete(JavabinLoader.java:149)
   [junit4]   2>        at 
org.apache.solr.handler.loader.JavabinLoader.parseAndLoadDocs(JavabinLoader.java:113)
   [junit4]   2>        at 
org.apache.solr.handler.loader.JavabinLoader.load(JavabinLoader.java:55)
   [junit4]   2>        at 
org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:97)
   [junit4]   2>        at 
org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:68)
   [junit4]   2>        at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)
   [junit4]   2>        at 
org.apache.solr.core.SolrCore.execute(SolrCore.java:2483)
   [junit4]   2>        at 
org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:720)
   [junit4]   2>        at 
org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:526)
   [junit4]   2>        at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:382)
   [junit4]   2>        at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
   [junit4]   2>        at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
   [junit4]   2>        at 
org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
   [junit4]   2>        at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
   [junit4]   2>        at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
   [junit4]   2>        at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
   [junit4]   2>        at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
   [junit4]   2>        at 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
   [junit4]   2>        at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
   [junit4]   2>        at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
   [junit4]   2>        at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
   [junit4]   2>        at 
org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:426)
   [junit4]   2>        at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
   [junit4]   2>        at 
org.eclipse.jetty.server.Server.handle(Server.java:534)
   [junit4]   2>        at 
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
   [junit4]   2>        at 
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
   [junit4]   2>        at 
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
   [junit4]   2>        at 
org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
   [junit4]   2>        at 
org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
   [junit4]   2>        at 
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
   [junit4]   2>        at 
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
   [junit4]   2>        at 
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
   [junit4]   2>        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
   [junit4]   2>        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 
   [junit4]   2> 1734248 ERROR (StoppableIndexingThread) [    ] 
o.a.s.c.s.i.CloudSolrClient Request to collection collection1 failed due to 
(503) org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: 
Error from server at 
http://127.0.0.1:50569/_g/wz/collection1_shard1_replica_t23: No registered 
leader was found after waiting for 4000ms , collection: collection1 slice: 
shard1, retry? 0
   [junit4]   2> REQUEST FAILED for id=0-3314
   [junit4]   2> 
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: Error from 
server at http://127.0.0.1:50569/_g/wz/collection1_shard1_replica_t23: No 
registered leader was found after waiting for 4000ms , collection: collection1 
slice: shard1
   [junit4]   2>        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:539)
   [junit4]   2>        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:993)
   [junit4]   2>        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:862)
   [junit4]   2>        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:793)
   [junit4]   2>        at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
   [junit4]   2>        at 
org.apache.solr.client.solrj.SolrClient.deleteById(SolrClient.java:753)
   [junit4]   2>        at 
org.apache.solr.client.solrj.SolrClient.deleteById(SolrClient.java:716)
   [junit4]   2>        at 
org.apache.solr.client.solrj.SolrClient.deleteById(SolrClient.java:731)
   [junit4]   2>        at 
org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:93)
   [junit4]   2> Caused by: 
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at http://127.0.0.1:50569/_g/wz/collection1_shard1_replica_t23: No 
registered leader was found after waiting for 4000ms , collection: collection1 
slice: shard1
   [junit4]   2>        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:627)
   [junit4]   2>        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:253)
   [junit4]   2>        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242)
   [junit4]   2>        at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
   [junit4]   2>        at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
   [junit4]   2>        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.lambda$directUpdate$0(CloudSolrClient.java:516)
   [junit4]   2>        at 
java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]   2>        at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 1734389 WARN  
(TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[1685ACBEFE763618]-SendThread(127.0.0.1:50527))
 [    ] o.a.z.ClientCnxn Session 0x15e12cf4ddd0006 for server null, unexpected 
error, closing socket connection and attempting reconnect
   [junit4]   2> java.net.ConnectException: Connection refused: no further 
information
   [junit4]   2>        at sun.nio.ch.SocketChannelImpl.checkConnect(Native 
Method)
   [junit4]   2>        at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1141)
   [junit4]   2> 1734471 WARN  
(TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[1685ACBEFE763618]-SendThread(127.0.0.1:50527))
 [n:127.0.0.1:50534__g%2Fwz    ] o.a.z.ClientCnxn Session 0x15e12cf4ddd0004 for 
server null, unexpected error, closing socket connection and attempting 
reconnect
   [junit4]   2> java.net.ConnectException: Connection refused: no further 
information
   [junit4]   2>        at sun.nio.ch.SocketChannelImpl.checkConnect(Native 
Method)
   [junit4]   2>        at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1141)
   [junit4]   2> 1734493 WARN  
(TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[1685ACBEFE763618]-SendThread(127.0.0.1:50527))
 [    ] o.a.z.ClientCnxn Session 0x15e12cf4ddd0010 for server null, unexpected 
error, closing socket connection and attempting reconnect
   [junit4]   2> java.net.ConnectException: Connection refused: no further 
information
   [junit4]   2>        at sun.nio.ch.SocketChannelImpl.checkConnect(Native 
Method)
   [junit4]   2>        at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1141)
   [junit4]   2> 1734681 WARN  
(TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[1685ACBEFE763618]-SendThread(127.0.0.1:50527))
 [n:127.0.0.1:50569__g%2Fwz    ] o.a.z.ClientCnxn Session 0x15e12cf4ddd000c for 
server null, unexpected error, closing socket connection and attempting 
reconnect
   [junit4]   2> java.net.ConnectException: Connection refused: no further 
information
   [junit4]   2>        at sun.nio.ch.SocketChannelImpl.checkConnect(Native 
Method)
   [junit4]   2>        at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1141)
   [junit4]   2> 1736727 ERROR (indexFetcher-9204-thread-1) 
[n:127.0.0.1:50583__g%2Fwz c:collection1 s:shard1 r:core_node26 
x:collection1_shard1_replica_t25] o.a.s.h.ReplicationHandler Index fetch failed 
:org.apache.solr.common.SolrException: No registered leader was found after 
waiting for 4000ms , collection: collection1 slice: shard1
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:758)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:744)
   [junit4]   2>        at 
org.apache.solr.handler.IndexFetcher.getLeaderReplica(IndexFetcher.java:667)
   [junit4]   2>        at 
org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:367)
   [junit4]   2>        at 
org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:332)
   [junit4]   2>        at 
org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:419)
   [junit4]   2>        at 
org.apache.solr.handler.ReplicationHandler.lambda$setupPolling$12(ReplicationHandler.java:1183)
   [junit4]   2>        at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
   [junit4]   2>        at 
java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
   [junit4]   2>        at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
   [junit4]   2>        at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 
   [junit4]   2> 1736889 WARN  
(TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[1685ACBEFE763618]-SendThread(127.0.0.1:50527))
 [n:127.0.0.1:50583__g%2Fwz    ] o.a.z.ClientCnxn Session 0x15e12cf4ddd000f for 
server null, unexpected error, closing socket connection and attempting 
reconnect
   [junit4]   2> java.net.ConnectException: Connection refused: no further 
information
   [junit4]   2>        at sun.nio.ch.SocketChannelImpl.checkConnect(Native 
Method)
   [junit4]   2>        at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1141)
   [junit4]   2> 1737022 ERROR (indexFetcher-9197-thread-1) 
[n:127.0.0.1:50569__g%2Fwz c:collection1 s:shard1 r:core_node24 
x:collection1_shard1_replica_t23] o.a.s.h.ReplicationHandler Index fetch failed 
:org.apache.solr.common.SolrException: No registered leader was found after 
waiting for 4000ms , collection: collection1 slice: shard1
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:758)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:744)
   [junit4]   2>        at 
org.apache.solr.handler.IndexFetcher.getLeaderReplica(IndexFetcher.java:667)
   [junit4]   2>        at 
org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:367)
   [junit4]   2>        at 
org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:332)
   [junit4]   2>        at 
org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:419)
   [junit4]   2>        at 
org.apache.solr.handler.ReplicationHandler.lambda$setupPolling$12(ReplicationHandler.java:1183)
   [junit4]   2>        at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
   [junit4]   2>        at 
java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
   [junit4]   2>        at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
   [junit4]   2>        at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 
   [junit4]   2> 1737190 ERROR (Thread-4369) [    ] o.a.z.s.ZooKeeperServer 
ZKShutdownHandler is not registered, so ZooKeeper server won't take any action 
on ERROR or SHUTDOWN server state changes
   [junit4]   2> 1738299 ERROR (qtp1609904884-22113) [n:127.0.0.1:50569__g%2Fwz 
c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_t23] 
o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: No registered 
leader was found after waiting for 4000ms , collection: collection1 slice: 
shard1
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:758)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:744)
   [junit4]   2>        at 
org.apache.solr.update.processor.DistributedUpdateProcessor.setupRequest(DistributedUpdateProcessor.java:404)
   [junit4]   2>        at 
org.apache.solr.update.processor.DistributedUpdateProcessor.setupRequest(DistributedUpdateProcessor.java:350)
   [junit4]   2>        at 
org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:708)
   [junit4]   2>        at 
org.apache.solr.handler.loader.JavabinLoader$1.update(JavabinLoader.java:98)
   [junit4]   2>        at 
org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$1.readOuterMostDocIterator(JavaBinUpdateRequestCodec.java:188)
   [junit4]   2>        at 
org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$1.readIterator(JavaBinUpdateRequestCodec.java:144)
   [junit4]   2>        at 
org.apache.solr.common.util.JavaBinCodec.readObject(JavaBinCodec.java:311)
   [junit4]   2>        at 
org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:256)
   [junit4]   2>        at 
org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$1.readNamedList(JavaBinUpdateRequestCodec.java:130)
   [junit4]   2>        at 
org.apache.solr.common.util.JavaBinCodec.readObject(JavaBinCodec.java:276)
   [junit4]   2>        at 
org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:256)
   [junit4]   2>        at 
org.apache.solr.common.util.JavaBinCodec.unmarshal(JavaBinCodec.java:178)
   [junit4]   2>        at 
org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec.unmarshal(JavaBinUpdateRequestCodec.java:195)
   [junit4]   2>        at 
org.apache.solr.handler.loader.JavabinLoader.parseAndLoadDocs(JavabinLoader.java:108)
   [junit4]   2>        at 
org.apache.solr.handler.loader.JavabinLoader.load(JavabinLoader.java:55)
   [junit4]   2>        at 
org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:97)
   [junit4]   2>        at 
org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:68)
   [junit4]   2>        at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)
   [junit4]   2>        at 
org.apache.solr.core.SolrCore.execute(SolrCore.java:2483)
   [junit4]   2>        at 
org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:720)
   [junit4]   2>        at 
org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:526)
   [junit4]   2>        at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:382)
   [junit4]   2>        at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
   [junit4]   2>        at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
   [junit4]   2>        at 
org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
   [junit4]   2>        at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
   [junit4]   2>        at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
   [junit4]   2>        at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
   [junit4]   2>        at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
   [junit4]   2>        at 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
   [junit4]   2>        at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
   [junit4]   2>        at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
   [junit4]   2>        at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
   [junit4]   2>        at 
org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:426)
   [junit4]   2>        at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
   [junit4]   2>        at 
org.eclipse.jetty.server.Server.handle(Server.java:534)
   [junit4]   2>        at 
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
   [junit4]   2>        at 
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
   [junit4]   2>        at 
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
   [junit4]   2>        at 
org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
   [junit4]   2>        at 
org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
   [junit4]   2>        at 
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
   [junit4]   2>        at 
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
   [junit4]   2>        at 
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
   [junit4]   2>        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
   [junit4]   2>        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 
   [junit4]   2> 1738300 ERROR (StoppableIndexingThread) [    ] 
o.a.s.c.s.i.CloudSolrClient Request to collection collection1 failed due to 
(503) org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: 
Error from server at 
http://127.0.0.1:50569/_g/wz/collection1_shard1_replica_t23: No registered 
leader was found after waiting for 4000ms , collection: collection1 slice: 
shard1, retry? 0
   [junit4]   2> REQUEST FAILED for id=0-3427
   [junit4]   2> 
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: Error from 
server at http://127.0.0.1:50569/_g/wz/collection1_shard1_replica_t23: No 
registered leader was found after waiting for 4000ms , collection: collection1 
slice: shard1
   [junit4]   2>        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:539)
   [junit4]   2>        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:993)
   [junit4]   2>        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:862)
   [junit4]   2>        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:793)
   [junit4]   2>        at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
   [junit4]   2>        at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:195)
   [junit4]   2>        at 
org.apache.solr.cloud.StoppableIndexingThread.indexDocs(StoppableIndexingThread.java:181)
   [junit4]   2>        at 
org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:115)
   [junit4]   2> Caused by: 
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at http://127.0.0.1:50569/_g/wz/collection1_shard1_replica_t23: No 
registered leader was found after waiting for 4000ms , collection: collection1 
slice: shard1
   [junit4]   2>        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:627)
   [junit4]   2>        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:253)
   [junit4]   2>        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242)
   [junit4]   2>        at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
   [junit4]   2>        at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
   [junit4]   2>        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.lambda$directUpdate$0(CloudSolrClient.java:516)
   [junit4]   2>        at 
java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]   2>        at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> added docs:3428 with 4 fails deletes:1696
   [junit4]   2> 1740390 WARN  
(recoveryExecutor-9187-thread-1-processing-n:127.0.0.1:50569__g%2Fwz 
x:collection1_shard1_replica_t23 s:shard1 c:collection1 r:core_node24) 
[n:127.0.0.1:50569__g%2Fwz c:collection1 s:shard1 r:core_node24 
x:collection1_shard1_replica_t23] o.a.s.u.UpdateLog Starting log replay 
tlog{file=C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.ChaosMonkeyNothingIsSafeTest_1685ACBEFE763618-001\shard-2-001\cores\collection1_shard1_replica_t23\data\tlog\tlog.0000000000000000000
 refcount=2} active=false starting pos=0 inSortedOrder=true
   [junit4]   2> 1740631 WARN  
(recoveryExecutor-9187-thread-1-processing-n:127.0.0.1:50569__g%2Fwz 
x:collection1_shard1_replica_t23 s:shard1 c:collection1 r:core_node24) 
[n:127.0.0.1:50569__g%2Fwz c:collection1 s:shard1 r:core_node24 
x:collection1_shard1_replica_t23] o.a.s.u.UpdateLog Log replay finished. 
recoveryInfo=RecoveryInfo{adds=3623 deletes=1785 deleteByQuery=1 errors=0 
positionOfStart=0}
   [junit4]   2> 1742834 WARN  (qtp1609904884-22111) [n:127.0.0.1:50569__g%2Fwz 
c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_t23] 
o.a.s.h.ReplicationHandler Exception while invoking 'details' method for 
replication on master 
   [junit4]   2> org.apache.solr.client.solrj.SolrServerException: Server 
refused connection at: 
http://127.0.0.1:50555/_g/wz/collection1_shard1_replica_t21
   [junit4]   2>        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:634)
   [junit4]   2>        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:253)
   [junit4]   2>        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242)
   [junit4]   2>        at 
org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
   [junit4]   2>        at 
org.apache.solr.handler.IndexFetcher.getDetails(IndexFetcher.java:1813)
   [junit4]   2>        at 
org.apache.solr.handler.ReplicationHandler.getReplicationDetails(ReplicationHandler.java:952)
   [junit4]   2>        at 
org.apache.solr.handler.ReplicationHandler.handleRequestBody(ReplicationHandler.java:330)
   [junit4]   2>        at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)
   [junit4]   2>        at 
org.apache.solr.core.SolrCore.execute(SolrCore.java:2483)
   [junit4]   2>        at 
org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:720)
   [junit4]   2>        at 
org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:526)
   [junit4]   2>        at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:382)
   [junit4]   2>        at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
   [junit4]   2>        at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
   [junit4]   2>        at 
org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
   [junit4]   2>        at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
   [junit4]   2>        at org.eclipse.jetty.servlet.Serv

[...truncated too long message...]

 metric reporters for registry=solr.collection.collection1.shard1.leader, 
tag=950095689
   [junit4]   2> 275457 INFO  (jetty-closer-1071-thread-3) [    ] 
o.a.s.c.Overseer Overseer 
(id=98536948330790923-127.0.0.1:53645_solr-n_0000000000) closing
   [junit4]   2> 275457 INFO  
(OverseerStateUpdate-98536948330790923-127.0.0.1:53645_solr-n_0000000000) 
[n:127.0.0.1:53645_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 
127.0.0.1:53645_solr
   [junit4]   2> 275457 INFO  
(zkCallback-1086-thread-1-processing-n:127.0.0.1:53643_solr) 
[n:127.0.0.1:53643_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (3) -> (2)
   [junit4]   2> 275457 INFO  
(zkCallback-1088-thread-1-processing-n:127.0.0.1:53647_solr) 
[n:127.0.0.1:53647_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (3) -> (2)
   [junit4]   2> 275457 INFO  
(zkCallback-1086-thread-2-processing-n:127.0.0.1:53643_solr) 
[n:127.0.0.1:53643_solr    ] o.a.s.c.OverseerElectionContext I am going to be 
the leader 127.0.0.1:53643_solr
   [junit4]   2> 275457 INFO  (coreCloseExecutor-1012-thread-1) 
[n:127.0.0.1:53647_solr c:workQueue s:shard2 r:core_node4 
x:workQueue_shard2_replica_n2] o.a.s.m.SolrMetricManager Closing metric 
reporters for registry=solr.collection.workQueue.shard2.leader, tag=335349711
   [junit4]   2> 275457 INFO  (coreCloseExecutor-1013-thread-3) 
[n:127.0.0.1:53643_solr c:mainCorpus s:shard1 r:core_node3 
x:mainCorpus_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric 
reporters for registry=solr.collection.mainCorpus.shard1.leader, tag=1570432178
   [junit4]   2> 275457 INFO  (coreCloseExecutor-1013-thread-2) 
[n:127.0.0.1:53643_solr c:workQueue s:shard1 r:core_node3 
x:workQueue_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric 
reporters for registry=solr.core.workQueue.shard1.replica_n1, tag=1488129059
   [junit4]   2> 275457 INFO  (coreCloseExecutor-1013-thread-2) 
[n:127.0.0.1:53643_solr c:workQueue s:shard1 r:core_node3 
x:workQueue_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter Closing reporter 
[org.apache.solr.metrics.reporters.SolrJmxReporter@10ab1e3a: rootName = 
solr_53643, domain = solr.core.workQueue.shard1.replica_n1, service url = null, 
agent id = null] for registry solr.core.workQueue.shard1.replica_n1 / 
com.codahale.metrics.MetricRegistry@68396791
   [junit4]   2> 275457 WARN  
(zkCallback-1088-thread-1-processing-n:127.0.0.1:53647_solr) 
[n:127.0.0.1:53647_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, 
but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 275457 INFO  
(zkCallback-1086-thread-2-processing-n:127.0.0.1:53643_solr) 
[n:127.0.0.1:53643_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (2) -> (1)
   [junit4]   2> 275457 INFO  (jetty-closer-1071-thread-2) [    ] 
o.e.j.s.h.ContextHandler Stopped 
o.e.j.s.ServletContextHandler@18074a3{/solr,null,UNAVAILABLE}
   [junit4]   2> 275473 INFO  (coreCloseExecutor-1013-thread-2) 
[n:127.0.0.1:53643_solr c:workQueue s:shard1 r:core_node3 
x:workQueue_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric 
reporters for registry=solr.collection.workQueue.shard1.leader, tag=1488129059
   [junit4]   2> 275473 INFO  (coreCloseExecutor-1013-thread-2) 
[n:127.0.0.1:53643_solr c:workQueue s:shard1 r:core_node3 
x:workQueue_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 Committing on 
IndexWriter close.
   [junit4]   2> 275473 INFO  (coreCloseExecutor-1013-thread-2) 
[n:127.0.0.1:53643_solr c:workQueue s:shard1 r:core_node3 
x:workQueue_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData 
with IW:org.apache.solr.update.SolrIndexWriter@62c10ce1 commitCommandVersion:0
   [junit4]   2> 275473 INFO  (coreCloseExecutor-1013-thread-1) 
[n:127.0.0.1:53643_solr c:collection1 s:shard2 r:core_node4 
x:collection1_shard2_replica_n2] o.a.s.m.SolrMetricManager Closing metric 
reporters for registry=solr.core.collection1.shard2.replica_n2, tag=636885371
   [junit4]   2> 275473 INFO  (coreCloseExecutor-1013-thread-1) 
[n:127.0.0.1:53643_solr c:collection1 s:shard2 r:core_node4 
x:collection1_shard2_replica_n2] o.a.s.m.r.SolrJmxReporter Closing reporter 
[org.apache.solr.metrics.reporters.SolrJmxReporter@25dd99d5: rootName = 
solr_53643, domain = solr.core.collection1.shard2.replica_n2, service url = 
null, agent id = null] for registry solr.core.collection1.shard2.replica_n2 / 
com.codahale.metrics.MetricRegistry@34779a17
   [junit4]   2> 275473 INFO  (coreCloseExecutor-1013-thread-1) 
[n:127.0.0.1:53643_solr c:collection1 s:shard2 r:core_node4 
x:collection1_shard2_replica_n2] o.a.s.m.SolrMetricManager Closing metric 
reporters for registry=solr.collection.collection1.shard2.leader, tag=636885371
   [junit4]   2> 275488 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] 
o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from 
client sessionid 0x15e12d51a820008, likely client has closed socket
   [junit4]   2>        at 
org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
   [junit4]   2>        at 
org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 275488 WARN  
(zkCallback-1086-thread-2-processing-n:127.0.0.1:53643_solr) 
[n:127.0.0.1:53643_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, 
but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 275488 INFO  (jetty-closer-1071-thread-1) [    ] 
o.e.j.s.h.ContextHandler Stopped 
o.e.j.s.ServletContextHandler@6000886a{/solr,null,UNAVAILABLE}
   [junit4]   2> 276971 WARN  
(zkCallback-1091-thread-1-processing-n:127.0.0.1:53645_solr) 
[n:127.0.0.1:53645_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, 
but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 276971 INFO  (jetty-closer-1071-thread-3) [    ] 
o.e.j.s.h.ContextHandler Stopped 
o.e.j.s.ServletContextHandler@717e8fe4{/solr,null,UNAVAILABLE}
   [junit4]   2> 276971 ERROR 
(SUITE-StreamExpressionTest-seed#[CA26E6DC85B2322F]-worker) [    ] 
o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper 
server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 276971 INFO  
(SUITE-StreamExpressionTest-seed#[CA26E6DC85B2322F]-worker) [    ] 
o.a.s.c.ZkTestServer connecting to 127.0.0.1:53639 53639
   [junit4]   2> 276971 INFO  (Thread-719) [    ] o.a.s.c.ZkTestServer 
connecting to 127.0.0.1:53639 53639
   [junit4]   2> 276971 WARN  (Thread-719) [    ] o.a.s.c.ZkTestServer Watch 
limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2>        91      /solr/aliases.json
   [junit4]   2>        91      /solr/clusterprops.json
   [junit4]   2>        9       /solr/configs/conf
   [junit4]   2>        8       /solr/configs/ml
   [junit4]   2>        4       /solr/security.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2>        91      /solr/clusterstate.json
   [junit4]   2>        40      
/solr/collections/destinationCollection/state.json
   [junit4]   2>        28      
/solr/collections/parallelDestinationCollection1/state.json
   [junit4]   2>        20      /solr/collections/destination/state.json
   [junit4]   2>        20      
/solr/collections/parallelDestinationCollection/state.json
   [junit4]   2>        20      /solr/collections/workQueue/state.json
   [junit4]   2>        20      /solr/collections/mainCorpus/state.json
   [junit4]   2>        12      /solr/collections/modelCollection/state.json
   [junit4]   2>        10      /solr/collections/collection1/state.json
   [junit4]   2>        10      /solr/collections/uknownCollection/state.json
   [junit4]   2>        10      
/solr/collections/checkpointCollection/state.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2>        91      /solr/live_nodes
   [junit4]   2>        91      /solr/collections
   [junit4]   2> 
   [junit4]   2> NOTE: leaving temporary files on disk at: 
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-solrj\test\J1\temp\solr.client.solrj.io.stream.StreamExpressionTest_CA26E6DC85B2322F-001
   [junit4]   2> Aug 24, 2017 6:03:52 AM 
com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked 
thread(s).
   [junit4]   2> NOTE: test params are: 
codec=FastCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST,
 chunkSize=21707, maxDocsPerChunk=666, blockSize=296), 
termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST, 
chunkSize=21707, blockSize=296)), sim=RandomSimilarity(queryNorm=false): {}, 
locale=de, timezone=Australia/Broken_Hill
   [junit4]   2> NOTE: Windows 10 10.0 amd64/Oracle Corporation 1.8.0_144 
(64-bit)/cpus=3,threads=1,free=134153856,total=358612992
   [junit4]   2> NOTE: All tests run in this JVM: [TestRetryUtil, 
ContentStreamTest, AddEvaluatorTest, FloorEvaluatorTest, 
SolrExampleStreamingBinaryTest, TestJsonRecordReader, GetByIdTest, 
ExclusiveOrEvaluatorTest, RoundEvaluatorTest, AnlysisResponseBaseTest, 
TestCloudSolrClientConnections, TestDelegationTokenResponse, ShardParamsTest, 
LengthEvaluatorTest, SolrParamTest, MultiplyEvaluatorTest, 
CloudSolrClientMultiConstructorTest, JdbcTest, TestCollectionStateWatchers, 
SelectWithEvaluatorsTest, HttpSolrClientSSLAuthConPoolTest, 
TestCollectionAdminRequest, GreaterThanEvaluatorTest, SchemaTest, 
LessThanEvaluatorTest, CosineEvaluatorTest, TestToleratedUpdateError, 
ClientUtilsTest, LargeVolumeJettyTest, CloudSolrClientBuilderTest, 
SolrExampleBinaryTest, StreamingTest, TestFastInputStream, TestPolicy, 
BasicHttpSolrClientTest, HttpSolrClientBuilderTest, GraphExpressionTest, 
JdbcDriverTest, StreamExpressionTest]
   [junit4] Completed [135/135 (1!)] on J1 in 123.84s, 94 tests, 1 error <<< 
FAILURES!

[...truncated 42474 lines...]

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Reply via email to