Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1109/

10 tests failed.
FAILED:  
org.apache.solr.cloud.CdcrReplicationDistributedZkTest.testResilienceWithDeleteByQueryOnTarget

Error Message:
Timeout waiting for CDCR replication to complete @source_collection:shard1

Stack Trace:
java.lang.RuntimeException: Timeout waiting for CDCR replication to complete 
@source_collection:shard1
        at 
__randomizedtesting.SeedInfo.seed([CAAE2B1DF4390888:6ACCB9872460CDCD]:0)
        at 
org.apache.solr.cloud.BaseCdcrDistributedZkTest.waitForReplicationToComplete(BaseCdcrDistributedZkTest.java:794)
        at 
org.apache.solr.cloud.CdcrReplicationDistributedZkTest.testResilienceWithDeleteByQueryOnTarget(CdcrReplicationDistributedZkTest.java:588)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at java.lang.Thread.run(Thread.java:745)


FAILED:  
org.apache.solr.cloud.CdcrReplicationDistributedZkTest.testBatchBoundaries

Error Message:
Timeout waiting for CDCR replication to complete @source_collection:shard2

Stack Trace:
java.lang.RuntimeException: Timeout waiting for CDCR replication to complete 
@source_collection:shard2
        at 
__randomizedtesting.SeedInfo.seed([CAAE2B1DF4390888:E881F2488D9DC96F]:0)
        at 
org.apache.solr.cloud.BaseCdcrDistributedZkTest.waitForReplicationToComplete(BaseCdcrDistributedZkTest.java:794)
        at 
org.apache.solr.cloud.CdcrReplicationDistributedZkTest.testBatchBoundaries(CdcrReplicationDistributedZkTest.java:558)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at java.lang.Thread.run(Thread.java:745)


FAILED:  
org.apache.solr.cloud.CdcrReplicationDistributedZkTest.testTargetCollectionNotAvailable

Error Message:
Timeout while trying to assert replication errors

Stack Trace:
java.lang.AssertionError: Timeout while trying to assert replication errors
        at 
__randomizedtesting.SeedInfo.seed([CAAE2B1DF4390888:2C3A716EFF6665A0]:0)
        at 
org.apache.solr.cloud.CdcrReplicationDistributedZkTest.testTargetCollectionNotAvailable(CdcrReplicationDistributedZkTest.java:145)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.AssertionError
        at org.junit.Assert.fail(Assert.java:92)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.junit.Assert.assertTrue(Assert.java:54)
        at 
org.apache.solr.cloud.CdcrReplicationDistributedZkTest.testTargetCollectionNotAvailable(CdcrReplicationDistributedZkTest.java:132)
        ... 41 more


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Timeout occured while waiting response from server at: https://127.0.0.1:58377

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting 
response from server at: https://127.0.0.1:58377
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:619)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:261)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:250)
        at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
        at 
org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:400)
        at 
org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:516)
        at 
org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:180)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.SocketTimeoutException: Read timed out
        at java.net.SocketInputStream.socketRead0(Native Method)
        at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
        at java.net.SocketInputStream.read(SocketInputStream.java:170)
        at java.net.SocketInputStream.read(SocketInputStream.java:141)
        at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
        at sun.security.ssl.InputRecord.read(InputRecord.java:503)
        at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973)
        at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:930)
        at sun.security.ssl.AppInputStream.read(AppInputStream.java:105)
        at 
org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139)
        at 
org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155)
        at 
org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284)
        at 
org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
        at 
org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
        at 
org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
        at 
org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165)
        at 
org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:167)
        at 
org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
        at 
org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
        at 
org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:271)
        at 
org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
        at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
        at 
org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
        at 
org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
        at 
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
        at 
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:513)
        ... 47 more


FAILED:  org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test

Error Message:
No live SolrServers available to handle this 
request:[http://127.0.0.1:52014/_/v/c8n_1x3_lf_shard1_replica1]

Stack Trace:
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: No live 
SolrServers available to handle this 
request:[http://127.0.0.1:52014/_/v/c8n_1x3_lf_shard1_replica1]
        at 
__randomizedtesting.SeedInfo.seed([CAAE2B1DF4390888:42FA14C75AC56570]:0)
        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:755)
        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1161)
        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1050)
        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:992)
        at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.sendDocsWithRetry(AbstractFullDistribZkTestBase.java:753)
        at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.sendDocsWithRetry(AbstractFullDistribZkTestBase.java:741)
        at 
org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.testRf3WithLeaderFailover(LeaderFailoverAfterPartitionTest.java:178)
        at 
org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test(LeaderFailoverAfterPartitionTest.java:57)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.solr.client.solrj.SolrServerException: No live 
SolrServers available to handle this 
request:[http://127.0.0.1:52014/_/v/c8n_1x3_lf_shard1_replica1]
        at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:394)
        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.lambda$directUpdate$0(CloudSolrClient.java:730)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        ... 1 more
Caused by: org.apache.solr.client.solrj.SolrServerException: Server refused 
connection at: http://127.0.0.1:52014/_/v/c8n_1x3_lf_shard1_replica1
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:615)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:261)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:250)
        at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:415)
        at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:383)
        ... 6 more
Caused by: org.apache.http.conn.HttpHostConnectException: Connect to 
127.0.0.1:52014 [/127.0.0.1] failed: Connection refused
        at 
org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151)
        at 
org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:353)
        at 
org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:380)
        at 
org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
        at 
org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
        at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
        at 
org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
        at 
org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
        at 
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
        at 
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:513)
        ... 10 more
Caused by: java.net.ConnectException: Connection refused
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at 
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
        at 
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
        at 
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
        at java.net.Socket.connect(Socket.java:589)
        at 
org.apache.http.conn.socket.PlainConnectionSocketFactory.connectSocket(PlainConnectionSocketFactory.java:74)
        at 
org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:134)
        ... 20 more


FAILED:  org.apache.solr.cloud.ForceLeaderTest.testReplicasInLIRNoLeader

Error Message:
No live SolrServers available to handle this 
request:[http://127.0.0.1:42892/_/kx/forceleader_test_collection_shard1_replica2]

Stack Trace:
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: No live 
SolrServers available to handle this 
request:[http://127.0.0.1:42892/_/kx/forceleader_test_collection_shard1_replica2]
        at 
__randomizedtesting.SeedInfo.seed([CAAE2B1DF4390888:2C391FDDCDBBF1E9]:0)
        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:755)
        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1161)
        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1050)
        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:992)
        at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.sendDocsWithRetry(AbstractFullDistribZkTestBase.java:753)
        at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.sendDocsWithRetry(AbstractFullDistribZkTestBase.java:741)
        at 
org.apache.solr.cloud.ForceLeaderTest.sendDoc(ForceLeaderTest.java:424)
        at 
org.apache.solr.cloud.ForceLeaderTest.testReplicasInLIRNoLeader(ForceLeaderTest.java:131)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.solr.client.solrj.SolrServerException: No live 
SolrServers available to handle this 
request:[http://127.0.0.1:42892/_/kx/forceleader_test_collection_shard1_replica2]
        at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:394)
        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.lambda$directUpdate$0(CloudSolrClient.java:730)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        ... 1 more
Caused by: org.apache.solr.client.solrj.SolrServerException: Server refused 
connection at: 
http://127.0.0.1:42892/_/kx/forceleader_test_collection_shard1_replica2
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:615)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:261)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:250)
        at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:415)
        at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:383)
        ... 6 more
Caused by: org.apache.http.conn.HttpHostConnectException: Connect to 
127.0.0.1:42892 [/127.0.0.1] failed: Connection refused
        at 
org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151)
        at 
org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:353)
        at 
org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:380)
        at 
org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
        at 
org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
        at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
        at 
org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
        at 
org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
        at 
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
        at 
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:513)
        ... 10 more
Caused by: java.net.ConnectException: Connection refused
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at 
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
        at 
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
        at 
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
        at java.net.Socket.connect(Socket.java:589)
        at 
org.apache.http.conn.socket.PlainConnectionSocketFactory.connectSocket(PlainConnectionSocketFactory.java:74)
        at 
org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:134)
        ... 20 more


FAILED:  org.apache.solr.handler.TestReplicationHandler.doTestStressReplication

Error Message:
timed out waiting for collection1 startAt time to exceed: Thu Sep 01 04:54:40 
CHUT 2016

Stack Trace:
java.lang.AssertionError: timed out waiting for collection1 startAt time to 
exceed: Thu Sep 01 04:54:40 CHUT 2016
        at 
__randomizedtesting.SeedInfo.seed([CAAE2B1DF4390888:11052BDBF111613B]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at 
org.apache.solr.handler.TestReplicationHandler.watchCoreStartAt(TestReplicationHandler.java:1508)
        at 
org.apache.solr.handler.TestReplicationHandler.doTestStressReplication(TestReplicationHandler.java:858)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at java.lang.Thread.run(Thread.java:745)


FAILED:  
org.apache.solr.cloud.CdcrReplicationDistributedZkTest.testReplicationAfterRestart

Error Message:
Timeout waiting for CDCR replication to complete @source_collection:shard1

Stack Trace:
java.lang.RuntimeException: Timeout waiting for CDCR replication to complete 
@source_collection:shard1
        at 
__randomizedtesting.SeedInfo.seed([CAAE2B1DF4390888:96B3CC9C3A19BCBE]:0)
        at 
org.apache.solr.cloud.BaseCdcrDistributedZkTest.waitForReplicationToComplete(BaseCdcrDistributedZkTest.java:794)
        at 
org.apache.solr.cloud.CdcrReplicationDistributedZkTest.testReplicationAfterRestart(CdcrReplicationDistributedZkTest.java:235)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at java.lang.Thread.run(Thread.java:745)


FAILED:  
junit.framework.TestSuite.org.apache.solr.cloud.ConcurrentDeleteAndCreateCollectionTest

Error Message:
ObjectTracker found 10 object(s) that were not released!!! [InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient]

Stack Trace:
java.lang.AssertionError: ObjectTracker found 10 object(s) that were not 
released!!! [InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient]
        at __randomizedtesting.SeedInfo.seed([CAAE2B1DF4390888]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.junit.Assert.assertNull(Assert.java:551)
        at 
org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:257)
        at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at java.lang.Thread.run(Thread.java:745)


FAILED:  
junit.framework.TestSuite.org.apache.solr.cloud.ConcurrentDeleteAndCreateCollectionTest

Error Message:
10 threads leaked from SUITE scope at 
org.apache.solr.cloud.ConcurrentDeleteAndCreateCollectionTest:     1) 
Thread[id=22062, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-ConcurrentDeleteAndCreateCollectionTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)    2) Thread[id=22057, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-ConcurrentDeleteAndCreateCollectionTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)    3) Thread[id=22082, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-ConcurrentDeleteAndCreateCollectionTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)    4) Thread[id=22052, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-ConcurrentDeleteAndCreateCollectionTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)    5) Thread[id=22067, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-ConcurrentDeleteAndCreateCollectionTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)    6) Thread[id=22037, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-ConcurrentDeleteAndCreateCollectionTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)    7) Thread[id=22072, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-ConcurrentDeleteAndCreateCollectionTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)    8) Thread[id=22042, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-ConcurrentDeleteAndCreateCollectionTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)    9) Thread[id=22047, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-ConcurrentDeleteAndCreateCollectionTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)   10) Thread[id=22077, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-ConcurrentDeleteAndCreateCollectionTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 10 threads leaked from 
SUITE scope at org.apache.solr.cloud.ConcurrentDeleteAndCreateCollectionTest: 
   1) Thread[id=22062, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-ConcurrentDeleteAndCreateCollectionTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
   2) Thread[id=22057, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-ConcurrentDeleteAndCreateCollectionTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
   3) Thread[id=22082, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-ConcurrentDeleteAndCreateCollectionTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
   4) Thread[id=22052, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-ConcurrentDeleteAndCreateCollectionTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
   5) Thread[id=22067, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-ConcurrentDeleteAndCreateCollectionTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
   6) Thread[id=22037, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-ConcurrentDeleteAndCreateCollectionTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
   7) Thread[id=22072, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-ConcurrentDeleteAndCreateCollectionTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
   8) Thread[id=22042, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-ConcurrentDeleteAndCreateCollectionTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
   9) Thread[id=22047, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-ConcurrentDeleteAndCreateCollectionTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
  10) Thread[id=22077, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-ConcurrentDeleteAndCreateCollectionTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
        at __randomizedtesting.SeedInfo.seed([CAAE2B1DF4390888]:0)




Build Log:
[...truncated 11613 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_CAAE2B1DF4390888-001/init-core-data-001
   [junit4]   2> 51675 INFO  
(SUITE-CollectionsAPIDistributedZkTest-seed#[CAAE2B1DF4390888]-worker) [    ] 
o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: 
@org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 51923 INFO  
(SUITE-CollectionsAPIDistributedZkTest-seed#[CAAE2B1DF4390888]-worker) [    ] 
o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 51949 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 51961 INFO  (Thread-35) [    ] o.a.s.c.ZkTestServer client 
port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 51961 INFO  (Thread-35) [    ] o.a.s.c.ZkTestServer Starting 
server
   [junit4]   2> 52132 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.c.ZkTestServer start zk server on port:44563
   [junit4]   2> 52133 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 52149 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 52160 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] 
o.a.z.s.NIOServerCnxn Exception causing close of session 0x0 due to 
java.io.IOException: ZooKeeperServer not running
   [junit4]   2> 53805 INFO  (zkCallback-32-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@7dc2bc1b 
name:ZooKeeperConnection Watcher:127.0.0.1:44563 got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 53805 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 53805 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 53805 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 53813 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 53835 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 53839 INFO  (zkCallback-33-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@42eff37a 
name:ZooKeeperConnection Watcher:127.0.0.1:44563/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 53839 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 53840 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 53840 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 53849 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 53859 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 53861 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 53862 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml
 to /configs/conf1/solrconfig.xml
   [junit4]   2> 53863 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 53867 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/schema.xml
 to /configs/conf1/schema.xml
   [junit4]   2> 53867 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 53869 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml
 to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 53869 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.c.c.SolrZkClient makePath: 
/configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 53885 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt
 to /configs/conf1/stopwords.txt
   [junit4]   2> 53885 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 53888 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt
 to /configs/conf1/protwords.txt
   [junit4]   2> 53888 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 53890 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml
 to /configs/conf1/currency.xml
   [junit4]   2> 53890 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 53892 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml
 to /configs/conf1/enumsConfig.xml
   [junit4]   2> 53892 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 53895 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json
 to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 53895 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 53897 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt
 to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 53897 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 53902 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt
 to /configs/conf1/old_synonyms.txt
   [junit4]   2> 53902 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 53904 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt
 to /configs/conf1/synonyms.txt
   [junit4]   2> 53904 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 53926 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 53960 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 53966 INFO  (zkCallback-35-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@6852de2 name:ZooKeeperConnection 
Watcher:127.0.0.1:44563/solr got event WatchedEvent state:SyncConnected 
type:None path:null path:null type:None
   [junit4]   2> 53966 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 53966 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 53968 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] 
o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from 
client sessionid 0x156e1d293e80002, likely client has closed socket
   [junit4]   2>        at 
org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2>        at 
org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 54163 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.SolrTestCaseJ4 Writing core.properties file to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_CAAE2B1DF4390888-001/control-001/cores/collection1
   [junit4]   2> 54247 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 54365 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@4fa6eb77{/,null,AVAILABLE}
   [junit4]   2> 55137 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.e.j.s.ServerConnector Started ServerConnector@44aab337{SSL,[ssl, 
http/1.1]}{127.0.0.1:49385}
   [junit4]   2> 55137 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.e.j.s.Server Started @59695ms
   [junit4]   2> 55137 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: 
{solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_CAAE2B1DF4390888-001/tempDir-001/control/data,
 hostContext=/, hostPort=49385, 
coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_CAAE2B1DF4390888-001/control-001/cores}
   [junit4]   2> 55140 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): 
sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 55144 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: 
'/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_CAAE2B1DF4390888-001/control-001'
   [junit4]   2> 55144 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 55144 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find 
system property or JNDI)
   [junit4]   2> 55146 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 55163 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 55187 INFO  (zkCallback-36-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@21fb6de1 
name:ZooKeeperConnection Watcher:127.0.0.1:44563/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 55187 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 55187 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 55208 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in 
ZooKeeper)
   [junit4]   2> 55209 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.c.SolrXmlConfig Loading container configuration from 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_CAAE2B1DF4390888-001/control-001/solr.xml
   [junit4]   2> 55221 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.c.CorePropertiesLocator Config-defined core root directory: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_CAAE2B1DF4390888-001/control-001/cores
   [junit4]   2> 55221 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.c.CoreContainer New CoreContainer 685098092
   [junit4]   2> 55221 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.c.CoreContainer Loading cores into CoreContainer 
[instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_CAAE2B1DF4390888-001/control-001]
   [junit4]   2> 55221 WARN  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.c.CoreContainer Couldn't add files from 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_CAAE2B1DF4390888-001/control-001/lib
 to classpath: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_CAAE2B1DF4390888-001/control-001/lib
   [junit4]   2> 55221 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme 
: ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 
10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 
5,sizeOfQueue : -1,fairnessPolicy : false,
   [junit4]   2> 55265 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.u.UpdateShardHandler Created UpdateShardHandler HTTP client with params: 
socketTimeout=340000&connTimeout=45000&maxConnections=100000&maxConnectionsPerHost=100000
   [junit4]   2> 55265 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 55265 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.l.LogWatcher Registering Log Listener [Log4j 
(org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 55265 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44563/solr
   [junit4]   2> 55265 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 55266 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 55282 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 55300 INFO  (zkCallback-39-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@72b01c8a 
name:ZooKeeperConnection Watcher:127.0.0.1:44563 got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 55302 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 55302 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 55310 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) 
[n:127.0.0.1:49385_    ] o.a.s.c.ZkController Added new OnReconnect listener 
org.apache.solr.cloud.ZkController$$Lambda$74/1552792430@56cbae7e
   [junit4]   2> 55328 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) 
[n:127.0.0.1:49385_    ] o.a.s.c.c.ConnectionManager Waiting for client to 
connect to ZooKeeper
   [junit4]   2> 55356 INFO  
(zkCallback-40-thread-1-processing-n:127.0.0.1:49385_) [n:127.0.0.1:49385_    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@621bdbb7 
name:ZooKeeperConnection Watcher:127.0.0.1:44563/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 55358 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) 
[n:127.0.0.1:49385_    ] o.a.s.c.c.ConnectionManager Client is connected to 
ZooKeeper
   [junit4]   2> 55360 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) 
[n:127.0.0.1:49385_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 55367 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) 
[n:127.0.0.1:49385_    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer/collection-queue-work
   [junit4]   2> 55370 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) 
[n:127.0.0.1:49385_    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer/collection-map-running
   [junit4]   2> 55373 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) 
[n:127.0.0.1:49385_    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer/collection-map-completed
   [junit4]   2> 55375 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) 
[n:127.0.0.1:49385_    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer/collection-map-failure
   [junit4]   2> 55376 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) 
[n:127.0.0.1:49385_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 55386 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) 
[n:127.0.0.1:49385_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 55388 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) 
[n:127.0.0.1:49385_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 55395 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) 
[n:127.0.0.1:49385_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 55396 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) 
[n:127.0.0.1:49385_    ] o.a.s.c.c.ZkStateReader Updating cluster state from 
ZooKeeper... 
   [junit4]   2> 55397 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) 
[n:127.0.0.1:49385_    ] o.a.s.c.c.ZkStateReader Loaded cluster properties: 
{urlScheme=https}
   [junit4]   2> 55397 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) 
[n:127.0.0.1:49385_    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (0)
   [junit4]   2> 55401 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) 
[n:127.0.0.1:49385_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 55402 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) 
[n:127.0.0.1:49385_    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer_elect/election
   [junit4]   2> 55403 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) 
[n:127.0.0.1:49385_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 55404 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) 
[n:127.0.0.1:49385_    ] o.a.s.c.LeaderElector Joined leadership election with 
path: /overseer_elect/election/96512736575881221-127.0.0.1:49385_-n_0000000000
   [junit4]   2> 55404 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) 
[n:127.0.0.1:49385_    ] o.a.s.c.OverseerElectionContext I am going to be the 
leader 127.0.0.1:49385_
   [junit4]   2> 55404 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) 
[n:127.0.0.1:49385_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 55406 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) 
[n:127.0.0.1:49385_    ] o.a.s.c.Overseer Overseer 
(id=96512736575881221-127.0.0.1:49385_-n_0000000000) starting
   [junit4]   2> 55408 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) 
[n:127.0.0.1:49385_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 55414 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) 
[n:127.0.0.1:49385_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting 
OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 
autoReplicaFailoverWaitAfterExpiration=10000 
autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 55425 INFO  
(OverseerStateUpdate-96512736575881221-127.0.0.1:49385_-n_0000000000) 
[n:127.0.0.1:49385_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 55436 INFO  
(OverseerStateUpdate-96512736575881221-127.0.0.1:49385_-n_0000000000) 
[n:127.0.0.1:49385_    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (0)
   [junit4]   2> 55444 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) 
[n:127.0.0.1:49385_    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:49385_
   [junit4]   2> 55444 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) 
[n:127.0.0.1:49385_    ] o.a.s.c.c.SolrZkClient makePath: 
/live_nodes/127.0.0.1:49385_
   [junit4]   2> 55446 INFO  
(zkCallback-40-thread-1-processing-n:127.0.0.1:49385_) [n:127.0.0.1:49385_    ] 
o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected 
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live 
nodes size: [0])
   [junit4]   2> 55446 INFO  
(OverseerCollectionConfigSetProcessor-96512736575881221-127.0.0.1:49385_-n_0000000000)
 [n:127.0.0.1:49385_    ] o.a.s.c.OverseerTaskProcessor Process current queue 
of overseer operations
   [junit4]   2> 55451 INFO  
(zkCallback-40-thread-1-processing-n:127.0.0.1:49385_) [n:127.0.0.1:49385_    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 55481 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) 
[n:127.0.0.1:49385_    ] o.a.s.c.CoreContainer Security conf doesn't exist. 
Skipping setup for authorization module.
   [junit4]   2> 55481 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) 
[n:127.0.0.1:49385_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 55482 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) 
[n:127.0.0.1:49385_    ] o.a.s.c.CorePropertiesLocator Looking for core 
definitions underneath 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_CAAE2B1DF4390888-001/control-001/cores
   [junit4]   2> 55483 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) 
[n:127.0.0.1:49385_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: 
{name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, 
loadOnStartup=true, configSetProperties=configsetprops.json, 
collection=control_collection, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 55484 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) 
[n:127.0.0.1:49385_    ] o.a.s.c.CorePropertiesLocator Found core collection1 
in 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_CAAE2B1DF4390888-001/control-001/cores/collection1
   [junit4]   2> 55484 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[CAAE2B1DF4390888]) 
[n:127.0.0.1:49385_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 55580 INFO  
(coreLoadExecutor-105-thread-1-processing-n:127.0.0.1:49385_) 
[n:127.0.0.1:49385_ c:control_collection   x:collection1] o.a.s.c.ZkController 
publishing state=down
   [junit4]   2> 55580 INFO  
(coreLoadExecutor-105-thread-1-processing-n:127.0.0.1:49385_) 
[n:127.0.0.1:49385_ c:control_collection   x:collection1] o.a.s.c.ZkController 
numShards not found on descriptor - reading it from system property
   [junit4]   2> 55582 INFO  
(coreLoadExecutor-105-thread-1-processing-n:127.0.0.1:49385_) 
[n:127.0.0.1:49385_ c:control_collection   x:collection1] o.a.s.c.ZkController 
Collection control_collection not visible yet, but flagging it so a watch is 
registered when it becomes visible
   [junit4]   2> 55583 INFO  
(OverseerStateUpdate-96512736575881221-127.0.0.1:49385_-n_0000000000) 
[n:127.0.0.1:49385_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message 
= {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:49385";,
   [junit4]   2>   "node_name":"127.0.0.1:49385_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 55583 INFO  
(OverseerStateUpdate-96512736575881221-127.0.0.1:49385_-n_0000000000) 
[n:127.0.0.1:49385_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 
message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:49385";,
   [junit4]   2>   "node_name":"127.0.0.1:49385_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 55584 INFO  
(OverseerStateUpdate-96512736575881221-127.0.0.1:49385_-n_0000000000) 
[n:127.0.0.1:49385_    ] o.a.s.c.o.ClusterStateMutator building a new cName: 
control_collection
   [junit4]   2> 55585 INFO  
(coreLoadExecutor-105-thread-1-processing-n:127.0.0.1:49385_) 
[n:127.0.0.1:49385_ c:control_collection   x:collection1] 
o.a.s.c.c.ZkStateReader Deleting data for [control_collection]
   [junit4]   2> 55586 INFO  
(OverseerStateUpdate-96512736575881221-127.0.0.1:49385_-n_0000000000) 
[n:127.0.0.1:49385_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard 
shard=shard1
   [junit4]   2> 55588 INFO  
(zkCallback-40-thread-1-processing-n:127.0.0.1:49385_) [n:127.0.0.1:49385_    ] 
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred 
- updating... (live nodes size: [1])
   [junit4]   2> 55611 INFO  
(coreLoadExecutor-105-thread-1-processing-n:127.0.0.1:49385_) 
[n:127.0.0.1:49385_ c:control_collection   x:collection1] o.a.s.c.ZkController 
look for our core node name
   [junit4]   2> 55611 INFO  
(coreLoadExecutor-105-thread-1-processing-n:127.0.0.1:49385_) 
[n:127.0.0.1:49385_ c:control_collection   x:collection1] o.a.s.c.ZkController 
waiting to find shard id in clusterstate for collection1
   [junit4]   2> 55611 INFO  
(coreLoadExecutor-105-thread-1-processing-n:127.0.0.1:49385_) 
[n:127.0.0.1:49385_ c:control_collection   x:collection1] o.a.s.c.ZkController 
Check for collection zkNode:control_collection
   [junit4]   2> 55611 INFO  
(coreLoadExecutor-105-thread-1-processing-n:127.0.0.1:49385_) 
[n:127.0.0.1:49385_ c:control_collection   x:collection1] o.a.s.c.ZkController 
Collection zkNode exists
   [junit4]   2> 55611 INFO  
(coreLoadExecutor-105-thread-1-processing-n:127.0.0.1:49385_) 
[n:127.0.0.1:49385_ c:control_collection   x:collection1] 
o.a.s.c.c.ZkStateReader Load collection config from: 
[/collections/control_collection]
   [junit4]   2> 55612 INFO  
(coreLoadExecutor-105-thread-1-processing-n:127.0.0.1:49385_) 
[n:127.0.0.1:49385_ c:control_collection   x:collection1] 
o.a.s.c.c.ZkStateReader path=[/collections/control_collection] 
[configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 55612 INFO  
(coreLoadExecutor-105-thread-1-processing-n:127.0.0.1:49385_) 
[n:127.0.0.1:49385_ c:control_collection   x:collection1] 
o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: 
'/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_CAAE2B1DF4390888-001/control-001/cores/collection1'
   [junit4]   2> 55612 INFO  
(coreLoadExecutor-105-thread-1-processing-n:127.0.0.1:49385_) 
[n:127.0.0.1:49385_ c:control_collection   x:collection1] 
o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 55612 INFO  
(coreLoadExecutor-105-thread-1-processing-n:127.0.0.1:49385_) 
[n:127.0.0.1:49385_ c:control_collection   x:collection1] 
o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find 
system property or JNDI)
   [junit4]   2

[...truncated too long message...]

est(LBHttpSolrClient.java:394)
   [junit4]    >        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.lambda$directUpdate$0(CloudSolrClient.java:730)
   [junit4]    >        at 
java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]    >        at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
   [junit4]    >        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4]    >        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >        ... 1 more
   [junit4]    > Caused by: org.apache.solr.client.solrj.SolrServerException: 
Server refused connection at: 
http://127.0.0.1:42892/_/kx/forceleader_test_collection_shard1_replica2
   [junit4]    >        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:615)
   [junit4]    >        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:261)
   [junit4]    >        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:250)
   [junit4]    >        at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:415)
   [junit4]    >        at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:383)
   [junit4]    >        ... 6 more
   [junit4]    > Caused by: org.apache.http.conn.HttpHostConnectException: 
Connect to 127.0.0.1:42892 [/127.0.0.1] failed: Connection refused
   [junit4]    >        at 
org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151)
   [junit4]    >        at 
org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:353)
   [junit4]    >        at 
org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:380)
   [junit4]    >        at 
org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
   [junit4]    >        at 
org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
   [junit4]    >        at 
org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
   [junit4]    >        at 
org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
   [junit4]    >        at 
org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
   [junit4]    >        at 
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
   [junit4]    >        at 
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
   [junit4]    >        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:513)
   [junit4]    >        ... 10 more
   [junit4]    > Caused by: java.net.ConnectException: Connection refused
   [junit4]    >        at java.net.PlainSocketImpl.socketConnect(Native Method)
   [junit4]    >        at 
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
   [junit4]    >        at 
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
   [junit4]    >        at 
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
   [junit4]    >        at 
java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
   [junit4]    >        at java.net.Socket.connect(Socket.java:589)
   [junit4]    >        at 
org.apache.http.conn.socket.PlainConnectionSocketFactory.connectSocket(PlainConnectionSocketFactory.java:74)
   [junit4]    >        at 
org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:134)
   [junit4]    >        ... 20 more
   [junit4]   2> 3357233 INFO  
(SUITE-ForceLeaderTest-seed#[CAAE2B1DF4390888]-worker) [    ] 
o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_CAAE2B1DF4390888-001
   [junit4]   2> Aug 31, 2016 7:13:15 PM 
com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked 
thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene62): 
{range_facet_l_dv=Lucene50(blocksize=128), _version_=PostingsFormat(name=Memory 
doPackFST= false), multiDefault=PostingsFormat(name=Direct), 
a_t=PostingsFormat(name=Direct), intDefault=PostingsFormat(name=Memory 
doPackFST= false), id=Lucene50(blocksize=128), 
range_facet_i_dv=PostingsFormat(name=Direct), 
text=PostingsFormat(name=LuceneVarGapFixedInterval), 
range_facet_l=PostingsFormat(name=Direct), 
timestamp=PostingsFormat(name=Direct)}, 
docValues:{range_facet_l_dv=DocValuesFormat(name=Direct), 
range_facet_i_dv=DocValuesFormat(name=Lucene54), 
timestamp=DocValuesFormat(name=Lucene54)}, maxPointsInLeafNode=1438, 
maxMBSortInHeap=7.336041322335511, sim=ClassicSimilarity, locale=nl-BE, 
timezone=Africa/Bamako
   [junit4]   2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 
1.8.0_102 (64-bit)/cpus=4,threads=1,free=312237992,total=513277952
   [junit4]   2> NOTE: All tests run in this JVM: [TestFoldingMultitermQuery, 
TestSubQueryTransformerDistrib, BlockJoinFacetRandomTest, CleanupOldIndexTest, 
CollectionsAPISolrJTest, StandardRequestHandlerTest, TestAnalyzedSuggestions, 
FileBasedSpellCheckerTest, BasicDistributedZkTest, TestDistribDocBasedVersion, 
TestSQLHandler, UUIDUpdateProcessorFallbackTest, XsltUpdateRequestHandlerTest, 
TestRawTransformer, AsyncCallRequestStatusResponseTest, DateRangeFieldTest, 
DocumentAnalysisRequestHandlerTest, TestDocTermOrds, CacheHeaderTest, 
DateFieldTest, TestSolrXml, HdfsWriteToMultipleCollectionsTest, 
TestLegacyNumericRangeQueryBuilder, TestReplicationHandlerBackup, 
BigEndianAscendingWordDeserializerTest, SolrCloudExampleTest, TestDocSet, 
JSONWriterTest, TestHighlightDedupGrouping, ConnectionReuseTest, TestTrieFacet, 
TestDynamicFieldResource, TestPhraseSuggestions, TestSolr4Spatial, 
TestNumericTerms32, TestManagedSchema, EchoParamsTest, DataDrivenBlockJoinTest, 
OutputWriterTest, TestRemoteStreaming, QueryElevationComponentTest, 
CursorPagingTest, WrapperMergePolicyFactoryTest, 
TestRandomCollapseQParserPlugin, ZkStateReaderTest, ReturnFieldsTest, 
DeleteInactiveReplicaTest, TestPKIAuthenticationPlugin, TestRecoveryHdfs, 
TestCopyFieldCollectionResource, IndexSchemaRuntimeFieldTest, 
TestTolerantUpdateProcessorCloud, SolrRequestParserTest, 
TestFieldCacheSanityChecker, TestBadConfig, HttpPartitionTest, 
TestValueSourceCache, TestComponentsName, TestStressLucene, ZkNodePropsTest, 
QueryResultKeyTest, ResponseHeaderTest, TestUseDocValuesAsStored, 
OutOfBoxZkACLAndCredentialsProvidersTest, AtomicUpdatesTest, ExplicitHLLTest, 
DefaultValueUpdateProcessorTest, TestJsonFacets, 
TriLevelCompositeIdRoutingTest, CloudMLTQParserTest, 
DocExpirationUpdateProcessorFactoryTest, CollectionStateFormat2Test, 
CopyFieldTest, AnalysisAfterCoreReloadTest, 
TestSolrCloudWithSecureImpersonation, DistributedDebugComponentTest, 
HdfsThreadLeakTest, TestRealTimeGet, TestFaceting, TestDFISimilarityFactory, 
AddBlockUpdateTest, TestOnReconnectListenerSupport, TestMergePolicyConfig, 
DistributedFacetPivotLongTailTest, SynonymTokenizerTest, PolyFieldTest, 
SuggestComponentContextFilterQueryTest, TestRandomFaceting, 
RAMDirectoryFactoryTest, JavabinLoaderTest, HdfsRecoveryZkTest, 
SaslZkACLProviderTest, QueryEqualityTest, ExternalFileFieldSortTest, 
TestHdfsCloudBackupRestore, ResourceLoaderTest, TestChildDocTransformer, 
PathHierarchyTokenizerFactoryTest, TestDynamicLoading, TestNoOpRegenerator, 
PreAnalyzedUpdateProcessorTest, TestSchemaResource, SyncSliceTest, 
TestLazyCores, ChaosMonkeySafeLeaderTest, UnloadDistributedZkTest, 
OverseerTest, LeaderElectionIntegrationTest, ShardRoutingTest, BasicZkTest, 
TestReplicationHandler, TestDistributedGrouping, 
TermVectorComponentDistributedTest, TestJoin, TestStressVersions, 
TestCoreContainer, SolrCoreTest, StatsComponentTest, TestFiltering, 
SimplePostToolTest, TestCoreDiscovery, TestExtendedDismaxParser, DocValuesTest, 
DisMaxRequestHandlerTest, DirectSolrSpellCheckerTest, 
TestReversedWildcardFilterFactory, TestQueryTypes, TermsComponentTest, 
RequestHandlersTest, TermVectorComponentTest, TestArbitraryIndexDir, 
LoggingHandlerTest, TestJmxIntegration, BinaryUpdateRequestHandlerTest, 
TestPHPSerializedResponseWriter, SearchHandlerTest, UpdateParamsTest, 
TestSolrIndexConfig, BadComponentTest, TestStressRecovery, SolrTestCaseJ4Test, 
TestCursorMarkWithoutUniqueKey, TestSimpleTrackingShardHandler, 
BaseCdcrDistributedZkTest, CdcrBootstrapTest, CloudExitableDirectoryReaderTest, 
CollectionsAPIAsyncDistributedZkTest, ConcurrentDeleteAndCreateCollectionTest, 
DeleteShardTest, DistribCursorPagingTest, ForceLeaderTest]
   [junit4] Completed [378/632 (6!)] on J0 in 115.74s, 3 tests, 1 error, 1 
skipped <<< FAILURES!

[...truncated 828 lines...]
   [junit4] JVM J0: stdout was not empty, see: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/temp/junit4-J0-20160831_181712_490.sysout
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] Dumping heap to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/heapdumps/java_pid13708.hprof
 ...
   [junit4] Heap dump file created [533401820 bytes in 2.740 secs]
   [junit4] <<< JVM J0: EOF ----

[...truncated 11041 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/build.xml:771:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/build.xml:715:
 Some of the tests produced a heap dump, but did not fail. Maybe a suppressed 
OutOfMemoryError? Dumps created:
* java_pid13708.hprof

Total time: 342 minutes 11 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to