Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2272/

6 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.solr.analytics.OverallAnalyticsTest

Error Message:
1 thread leaked from SUITE scope at 
org.apache.solr.analytics.OverallAnalyticsTest:     1) Thread[id=85, 
name=qtp557035741-85, state=TIMED_WAITING, group=TGRP-OverallAnalyticsTest]     
    at sun.misc.Unsafe.park(Native Method)         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
         at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
         at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) 
        at java.lang.Thread.run(Thread.java:748)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE 
scope at org.apache.solr.analytics.OverallAnalyticsTest: 
   1) Thread[id=85, name=qtp557035741-85, state=TIMED_WAITING, 
group=TGRP-OverallAnalyticsTest]
        at sun.misc.Unsafe.park(Native Method)
        at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
        at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
        at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
        at java.lang.Thread.run(Thread.java:748)
        at __randomizedtesting.SeedInfo.seed([2D4C3C614787FA5E]:0)


FAILED:  
junit.framework.TestSuite.org.apache.solr.analytics.OverallAnalyticsTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=85, 
name=qtp557035741-85, state=TIMED_WAITING, group=TGRP-OverallAnalyticsTest]     
    at sun.misc.Unsafe.park(Native Method)         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
         at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
         at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) 
        at java.lang.Thread.run(Thread.java:748)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie 
threads that couldn't be terminated:
   1) Thread[id=85, name=qtp557035741-85, state=TIMED_WAITING, 
group=TGRP-OverallAnalyticsTest]
        at sun.misc.Unsafe.park(Native Method)
        at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
        at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
        at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
        at java.lang.Thread.run(Thread.java:748)
        at __randomizedtesting.SeedInfo.seed([2D4C3C614787FA5E]:0)


FAILED:  
org.apache.solr.cloud.MoveReplicaHDFSFailoverTest.testOldReplicaIsDeleted

Error Message:
Could not load collection from ZK: movereplicatest_coll3

Stack Trace:
org.apache.solr.common.SolrException: Could not load collection from ZK: 
movereplicatest_coll3
        at 
__randomizedtesting.SeedInfo.seed([57F4D5455C2641A7:186A31985D2C0EC5]:0)
        at 
org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1108)
        at 
org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:647)
        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.getDocCollection(CloudSolrClient.java:1206)
        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:849)
        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
        at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:173)
        at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:138)
        at 
org.apache.solr.cloud.MoveReplicaHDFSFailoverTest.addDocs(MoveReplicaHDFSFailoverTest.java:203)
        at 
org.apache.solr.cloud.MoveReplicaHDFSFailoverTest.testOldReplicaIsDeleted(MoveReplicaHDFSFailoverTest.java:140)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: 
KeeperErrorCode = Session expired for 
/collections/movereplicatest_coll3/state.json
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
        at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1212)
        at 
org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:339)
        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
        at 
org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:339)
        at 
org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:1120)
        at 
org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1106)
        ... 48 more


FAILED:  
org.apache.solr.cloud.MoveReplicaHDFSFailoverTest.testOldReplicaIsDeletedInRaceCondition

Error Message:


Stack Trace:
java.lang.AssertionError
        at 
__randomizedtesting.SeedInfo.seed([57F4D5455C2641A7:5DA45A339C262006]:0)
        at org.junit.Assert.fail(Assert.java:92)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.junit.Assert.assertTrue(Assert.java:54)
        at 
org.apache.solr.cloud.MoveReplicaHDFSFailoverTest.testOldReplicaIsDeletedInRaceCondition(MoveReplicaHDFSFailoverTest.java:182)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.MoveReplicaHDFSTest.testNormalMove

Error Message:
Collection not found: MoveReplicaHDFSTest_coll_false

Stack Trace:
org.apache.solr.common.SolrException: Collection not found: 
MoveReplicaHDFSTest_coll_false
        at 
__randomizedtesting.SeedInfo.seed([57F4D5455C2641A7:F12C62E2D9C783BD]:0)
        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:851)
        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
        at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:173)
        at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:138)
        at 
org.apache.solr.cloud.MoveReplicaTest.addDocs(MoveReplicaTest.java:393)
        at org.apache.solr.cloud.MoveReplicaTest.test(MoveReplicaTest.java:119)
        at 
org.apache.solr.cloud.MoveReplicaHDFSTest.testNormalMove(MoveReplicaHDFSTest.java:63)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testNodeLost

Error Message:


Stack Trace:
java.util.concurrent.TimeoutException
        at 
__randomizedtesting.SeedInfo.seed([57F4D5455C2641A7:E8E11BBBDFCC2421]:0)
        at 
org.apache.solr.cloud.autoscaling.sim.SimSolrCloudTestCase.waitForState(SimSolrCloudTestCase.java:271)
        at 
org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.doTestNodeLost(TestLargeCluster.java:453)
        at 
org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testNodeLost(TestLargeCluster.java:295)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 12025 lines...]
   [junit4] Suite: org.apache.solr.cloud.MoveReplicaHDFSTest
   [junit4]   2> 692125 INFO  
(SUITE-MoveReplicaHDFSTest-seed#[57F4D5455C2641A7]-worker) [    ] 
o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: 
test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_57F4D5455C2641A7-001/init-core-data-001
   [junit4]   2> 692126 WARN  
(SUITE-MoveReplicaHDFSTest-seed#[57F4D5455C2641A7]-worker) [    ] 
o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=17 numCloses=17
   [junit4]   2> 692126 INFO  
(SUITE-MoveReplicaHDFSTest-seed#[57F4D5455C2641A7]-worker) [    ] 
o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) 
w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 692127 INFO  
(SUITE-MoveReplicaHDFSTest-seed#[57F4D5455C2641A7]-worker) [    ] 
o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: 
@org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 692128 INFO  
(SUITE-MoveReplicaHDFSTest-seed#[57F4D5455C2641A7]-worker) [    ] 
o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_57F4D5455C2641A7-001/tempDir-001
   [junit4]   2> 692129 INFO  
(SUITE-MoveReplicaHDFSTest-seed#[57F4D5455C2641A7]-worker) [    ] 
o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 692129 INFO  (Thread-545) [    ] o.a.s.c.ZkTestServer client 
port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 692129 INFO  (Thread-545) [    ] o.a.s.c.ZkTestServer Starting 
server
   [junit4]   2> 692136 ERROR (Thread-545) [    ] o.a.z.s.ZooKeeperServer 
ZKShutdownHandler is not registered, so ZooKeeper server won't take any action 
on ERROR or SHUTDOWN server state changes
   [junit4]   2> 692229 INFO  
(SUITE-MoveReplicaHDFSTest-seed#[57F4D5455C2641A7]-worker) [    ] 
o.a.s.c.ZkTestServer start zk server on port:33010
   [junit4]   2> 692231 INFO  (zkConnectionManagerCallback-668-thread-1) [    ] 
o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 692234 INFO  (jetty-launcher-665-thread-2) [    ] 
o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 
2017-11-22T05:27:37+08:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 692234 INFO  (jetty-launcher-665-thread-3) [    ] 
o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 
2017-11-22T05:27:37+08:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 692234 INFO  (jetty-launcher-665-thread-4) [    ] 
o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 
2017-11-22T05:27:37+08:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 692235 INFO  (jetty-launcher-665-thread-1) [    ] 
o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 
2017-11-22T05:27:37+08:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 692235 INFO  (jetty-launcher-665-thread-2) [    ] 
o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 692235 INFO  (jetty-launcher-665-thread-2) [    ] 
o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 692235 INFO  (jetty-launcher-665-thread-2) [    ] 
o.e.j.s.session Scavenging every 600000ms
   [junit4]   2> 692235 INFO  (jetty-launcher-665-thread-2) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@f4d2672{/solr,null,AVAILABLE}
   [junit4]   2> 692237 INFO  (jetty-launcher-665-thread-2) [    ] 
o.e.j.s.AbstractConnector Started ServerConnector@10c98503{SSL,[ssl, 
http/1.1]}{127.0.0.1:43284}
   [junit4]   2> 692237 INFO  (jetty-launcher-665-thread-2) [    ] 
o.e.j.s.Server Started @700675ms
   [junit4]   2> 692237 INFO  (jetty-launcher-665-thread-2) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, 
hostPort=43284}
   [junit4]   2> 692237 ERROR (jetty-launcher-665-thread-2) [    ] 
o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 692237 INFO  (jetty-launcher-665-thread-2) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 
8.0.0
   [junit4]   2> 692237 INFO  (jetty-launcher-665-thread-2) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 692237 INFO  (jetty-launcher-665-thread-2) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 692237 INFO  (jetty-launcher-665-thread-2) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2018-01-24T00:45:24.309Z
   [junit4]   2> 692238 INFO  (zkConnectionManagerCallback-670-thread-1) [    ] 
o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 692238 INFO  (jetty-launcher-665-thread-2) [    ] 
o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 692244 INFO  (jetty-launcher-665-thread-1) [    ] 
o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 692245 INFO  (jetty-launcher-665-thread-1) [    ] 
o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 692245 INFO  (jetty-launcher-665-thread-1) [    ] 
o.e.j.s.session Scavenging every 600000ms
   [junit4]   2> 692245 INFO  (jetty-launcher-665-thread-2) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33010/solr
   [junit4]   2> 692246 INFO  (jetty-launcher-665-thread-1) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@4c13894d{/solr,null,AVAILABLE}
   [junit4]   2> 692246 INFO  (jetty-launcher-665-thread-4) [    ] 
o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 692246 INFO  (jetty-launcher-665-thread-4) [    ] 
o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 692246 INFO  (jetty-launcher-665-thread-4) [    ] 
o.e.j.s.session Scavenging every 660000ms
   [junit4]   2> 692247 INFO  (jetty-launcher-665-thread-4) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@72d842fe{/solr,null,AVAILABLE}
   [junit4]   2> 692247 INFO  (jetty-launcher-665-thread-1) [    ] 
o.e.j.s.AbstractConnector Started ServerConnector@54b5afca{SSL,[ssl, 
http/1.1]}{127.0.0.1:34953}
   [junit4]   2> 692247 INFO  (jetty-launcher-665-thread-4) [    ] 
o.e.j.s.AbstractConnector Started ServerConnector@2fc74c23{SSL,[ssl, 
http/1.1]}{127.0.0.1:46604}
   [junit4]   2> 692247 INFO  (jetty-launcher-665-thread-1) [    ] 
o.e.j.s.Server Started @700686ms
   [junit4]   2> 692247 INFO  (jetty-launcher-665-thread-4) [    ] 
o.e.j.s.Server Started @700686ms
   [junit4]   2> 692247 INFO  (jetty-launcher-665-thread-1) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, 
hostPort=34953}
   [junit4]   2> 692247 INFO  (jetty-launcher-665-thread-4) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, 
hostPort=46604}
   [junit4]   2> 692247 INFO  (jetty-launcher-665-thread-3) [    ] 
o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 692247 INFO  (jetty-launcher-665-thread-3) [    ] 
o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 692247 INFO  (jetty-launcher-665-thread-3) [    ] 
o.e.j.s.session Scavenging every 600000ms
   [junit4]   2> 692247 ERROR (jetty-launcher-665-thread-1) [    ] 
o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 692247 ERROR (jetty-launcher-665-thread-4) [    ] 
o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 692247 INFO  (jetty-launcher-665-thread-1) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 
8.0.0
   [junit4]   2> 692248 INFO  (jetty-launcher-665-thread-4) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 
8.0.0
   [junit4]   2> 692248 INFO  (jetty-launcher-665-thread-1) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 692248 INFO  (jetty-launcher-665-thread-4) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 692248 INFO  (jetty-launcher-665-thread-1) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 692248 INFO  (jetty-launcher-665-thread-4) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 692248 INFO  (jetty-launcher-665-thread-3) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@650a778c{/solr,null,AVAILABLE}
   [junit4]   2> 692248 INFO  (jetty-launcher-665-thread-1) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2018-01-24T00:45:24.320Z
   [junit4]   2> 692248 INFO  (jetty-launcher-665-thread-4) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2018-01-24T00:45:24.320Z
   [junit4]   2> 692247 INFO  (zkConnectionManagerCallback-674-thread-1) [    ] 
o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 692248 INFO  (jetty-launcher-665-thread-3) [    ] 
o.e.j.s.AbstractConnector Started ServerConnector@603e5874{SSL,[ssl, 
http/1.1]}{127.0.0.1:33508}
   [junit4]   2> 692248 INFO  (jetty-launcher-665-thread-3) [    ] 
o.e.j.s.Server Started @700687ms
   [junit4]   2> 692248 INFO  (jetty-launcher-665-thread-3) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, 
hostPort=33508}
   [junit4]   2> 692249 ERROR (jetty-launcher-665-thread-3) [    ] 
o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 692249 INFO  (jetty-launcher-665-thread-3) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 
8.0.0
   [junit4]   2> 692249 INFO  (jetty-launcher-665-thread-3) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 692249 INFO  (jetty-launcher-665-thread-3) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 692249 INFO  (jetty-launcher-665-thread-3) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2018-01-24T00:45:24.321Z
   [junit4]   2> 692250 INFO  (zkConnectionManagerCallback-677-thread-1) [    ] 
o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 692250 INFO  (zkConnectionManagerCallback-680-thread-1) [    ] 
o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 692250 INFO  (jetty-launcher-665-thread-4) [    ] 
o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 692250 INFO  (jetty-launcher-665-thread-3) [    ] 
o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 692256 INFO  (zkConnectionManagerCallback-678-thread-1) [    ] 
o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 692258 DEBUG (jetty-launcher-665-thread-2) 
[n:127.0.0.1:43284_solr    ] o.a.s.c.ZkController Added new OnReconnect 
listener org.apache.solr.cloud.ZkController$$Lambda$253/628261647@386a8eec
   [junit4]   2> 692258 INFO  (jetty-launcher-665-thread-1) [    ] 
o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 692269 INFO  (jetty-launcher-665-thread-3) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33010/solr
   [junit4]   2> 692269 INFO  (jetty-launcher-665-thread-4) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33010/solr
   [junit4]   2> 692271 INFO  (jetty-launcher-665-thread-1) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33010/solr
   [junit4]   2> 692271 INFO  
(zkConnectionManagerCallback-683-thread-1-processing-n:127.0.0.1:43284_solr) 
[n:127.0.0.1:43284_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 692271 INFO  (zkConnectionManagerCallback-687-thread-1) [    ] 
o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 692272 INFO  (zkConnectionManagerCallback-690-thread-1) [    ] 
o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 692278 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] 
o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from 
client sessionid 0x16125a0656a0006, likely client has closed socket
   [junit4]   2>        at 
org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
   [junit4]   2>        at 
org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 692279 DEBUG (jetty-launcher-665-thread-3) 
[n:127.0.0.1:33508_solr    ] o.a.s.c.ZkController Added new OnReconnect 
listener org.apache.solr.cloud.ZkController$$Lambda$253/628261647@f8df5a9
   [junit4]   2> 692279 DEBUG (jetty-launcher-665-thread-4) 
[n:127.0.0.1:46604_solr    ] o.a.s.c.ZkController Added new OnReconnect 
listener org.apache.solr.cloud.ZkController$$Lambda$253/628261647@17baa91f
   [junit4]   2> 692280 INFO  (zkConnectionManagerCallback-694-thread-1) [    ] 
o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 692280 INFO  
(zkConnectionManagerCallback-698-thread-1-processing-n:127.0.0.1:46604_solr) 
[n:127.0.0.1:46604_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 692283 INFO  
(zkConnectionManagerCallback-696-thread-1-processing-n:127.0.0.1:33508_solr) 
[n:127.0.0.1:33508_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 692283 DEBUG (jetty-launcher-665-thread-1) 
[n:127.0.0.1:34953_solr    ] o.a.s.c.ZkController Added new OnReconnect 
listener org.apache.solr.cloud.ZkController$$Lambda$253/628261647@169aa53f
   [junit4]   2> 692292 INFO  
(zkConnectionManagerCallback-700-thread-1-processing-n:127.0.0.1:34953_solr) 
[n:127.0.0.1:34953_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 692413 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: data: /solr/clusterprops.json
   [junit4]   2> 692413 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: children: /solr/live_nodes
   [junit4]   2> 692418 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: data: /solr/clusterstate.json
   [junit4]   2> 692418 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: children: /solr/collections
   [junit4]   2> 692418 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: create/delete: /solr/aliases.json
   [junit4]   2> 692419 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: data: /solr/clusterprops.json
   [junit4]   2> 692419 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: create/delete: /solr/security.json
   [junit4]   2> 692419 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: children: /solr/live_nodes
   [junit4]   2> 692420 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: data: /solr/clusterstate.json
   [junit4]   2> 692420 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: children: /solr/collections
   [junit4]   2> 692420 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: create/delete: /solr/aliases.json
   [junit4]   2> 692420 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: create/delete: /solr/security.json
   [junit4]   2> 692427 INFO  (jetty-launcher-665-thread-4) 
[n:127.0.0.1:46604_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 692427 INFO  (jetty-launcher-665-thread-2) 
[n:127.0.0.1:43284_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 692428 DEBUG (jetty-launcher-665-thread-2) 
[n:127.0.0.1:43284_solr    ] o.a.s.c.LeaderElector Joined leadership election 
with path: 
/overseer_elect/election/99402037605302279-127.0.0.1:43284_solr-n_0000000001
   [junit4]   2> 692428 DEBUG (jetty-launcher-665-thread-4) 
[n:127.0.0.1:46604_solr    ] o.a.s.c.LeaderElector Joined leadership election 
with path: 
/overseer_elect/election/99402037605302282-127.0.0.1:46604_solr-n_0000000000
   [junit4]   2> 692429 INFO  (jetty-launcher-665-thread-4) 
[n:127.0.0.1:46604_solr    ] o.a.s.c.OverseerElectionContext I am going to be 
the leader 127.0.0.1:46604_solr
   [junit4]   2> 692429 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: data: 
/solr/overseer_elect/election/99402037605302282-127.0.0.1:46604_solr-n_0000000000
   [junit4]   2> 692429 DEBUG (jetty-launcher-665-thread-2) 
[n:127.0.0.1:43284_solr    ] o.a.s.c.LeaderElector Watching path 
/overseer_elect/election/99402037605302282-127.0.0.1:46604_solr-n_0000000000 to 
know if I could be the leader
   [junit4]   2> 692430 INFO  (jetty-launcher-665-thread-4) 
[n:127.0.0.1:46604_solr    ] o.a.s.c.Overseer Overseer 
(id=99402037605302282-127.0.0.1:46604_solr-n_0000000000) starting
   [junit4]   2> 692431 INFO  (jetty-launcher-665-thread-2) 
[n:127.0.0.1:43284_solr    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:43284_solr
   [junit4]   2> 692431 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
fired: children: /solr/live_nodes
   [junit4]   2> 692431 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
fired: children: /solr/live_nodes
   [junit4]   2> 692431 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: data: /solr/clusterprops.json
   [junit4]   2> 692439 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: children: /solr/live_nodes
   [junit4]   2> 692439 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: children: /solr/live_nodes
   [junit4]   2> 692439 INFO  (jetty-launcher-665-thread-1) 
[n:127.0.0.1:34953_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (1)
   [junit4]   2> 692439 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: children: /solr/live_nodes
   [junit4]   2> 692439 INFO  
(zkCallback-697-thread-1-processing-n:127.0.0.1:46604_solr) 
[n:127.0.0.1:46604_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (1)
   [junit4]   2> 692439 INFO  
(zkCallback-681-thread-1-processing-n:127.0.0.1:43284_solr) 
[n:127.0.0.1:43284_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (1)
   [junit4]   2> 692447 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: data: /solr/clusterstate.json
   [junit4]   2> 692447 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: children: /solr/collections
   [junit4]   2> 692447 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: create/delete: /solr/aliases.json
   [junit4]   2> 692448 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: create/delete: /solr/security.json
   [junit4]   2> 692449 INFO  (jetty-launcher-665-thread-1) 
[n:127.0.0.1:34953_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 692449 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: data: /solr/clusterprops.json
   [junit4]   2> 692449 DEBUG (jetty-launcher-665-thread-1) 
[n:127.0.0.1:34953_solr    ] o.a.s.c.LeaderElector Joined leadership election 
with path: 
/overseer_elect/election/99402037605302284-127.0.0.1:34953_solr-n_0000000002
   [junit4]   2> 692449 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: children: /solr/live_nodes
   [junit4]   2> 692449 INFO  (jetty-launcher-665-thread-3) 
[n:127.0.0.1:33508_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (1)
   [junit4]   2> 692450 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: data: /solr/clusterstate.json
   [junit4]   2> 692450 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: data: 
/solr/overseer_elect/election/99402037605302279-127.0.0.1:43284_solr-n_0000000001
   [junit4]   2> 692450 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: children: /solr/collections
   [junit4]   2> 692450 DEBUG (jetty-launcher-665-thread-1) 
[n:127.0.0.1:34953_solr    ] o.a.s.c.LeaderElector Watching path 
/overseer_elect/election/99402037605302279-127.0.0.1:43284_solr-n_0000000001 to 
know if I could be the leader
   [junit4]   2> 692450 DEBUG (jetty-launcher-665-thread-1) 
[n:127.0.0.1:34953_solr    ] o.a.s.c.ZkController Publish 
node=127.0.0.1:34953_solr as DOWN
   [junit4]   2> 692450 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: create/delete: /solr/aliases.json
   [junit4]   2> 692451 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: create/delete: /solr/security.json
   [junit4]   2> 692452 DEBUG 
(OverseerStateUpdate-99402037605302282-127.0.0.1:46604_solr-n_0000000000) 
[n:127.0.0.1:46604_solr    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 692452 DEBUG 
(OverseerCollectionConfigSetProcessor-99402037605302282-127.0.0.1:46604_solr-n_0000000000)
 [n:127.0.0.1:46604_solr    ] o.a.s.c.OverseerTaskProcessor Process current 
queue of overseer operations
   [junit4]   2> 692454 INFO  (jetty-launcher-665-thread-1) 
[n:127.0.0.1:34953_solr    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:34953_solr
   [junit4]   2> 692454 DEBUG (jetty-launcher-665-thread-4) 
[n:127.0.0.1:46604_solr    ] o.a.s.c.ZkController Publish 
node=127.0.0.1:46604_solr as DOWN
   [junit4]   2> 692454 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
fired: children: /solr/live_nodes
   [junit4]   2> 692454 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
fired: children: /solr/live_nodes
   [junit4]   2> 692454 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
fired: children: /solr/live_nodes
   [junit4]   2> 692454 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
fired: children: /solr/live_nodes
   [junit4]   2> 692455 DEBUG 
(OverseerAutoScalingTriggerThread-99402037605302282-127.0.0.1:46604_solr-n_0000000000)
 [n:127.0.0.1:46604_solr    ] o.a.s.c.a.OverseerTriggerThread Adding 
.autoAddReplicas trigger
   [junit4]   2> 692459 INFO  
(OverseerStateUpdate-99402037605302282-127.0.0.1:46604_solr-n_0000000000) 
[n:127.0.0.1:46604_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (1) -> (2)
   [junit4]   2> 692459 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: children: /solr/live_nodes
   [junit4]   2> 692459 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: children: /solr/live_nodes
   [junit4]   2> 692459 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: children: /solr/live_nodes
   [junit4]   2> 692459 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: data: /solr/autoscaling.json
   [junit4]   2> 692459 INFO  
(zkCallback-681-thread-1-processing-n:127.0.0.1:43284_solr) 
[n:127.0.0.1:43284_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (1) -> (2)
   [junit4]   2> 692459 DEBUG 
(OverseerAutoScalingTriggerThread-99402037605302282-127.0.0.1:46604_solr-n_0000000000)
 [n:127.0.0.1:46604_solr    ] o.a.s.c.a.OverseerTriggerThread Refreshing 
/autoscaling.json with znode version 1
   [junit4]   2> 692460 INFO  (jetty-launcher-665-thread-4) 
[n:127.0.0.1:46604_solr    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:46604_solr
   [junit4]   2> 692460 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: children: /solr/overseer/queue-work
   [junit4]   2> 692460 DEBUG 
(OverseerCollectionConfigSetProcessor-99402037605302282-127.0.0.1:46604_solr-n_0000000000)
 [n:127.0.0.1:46604_solr    ] o.a.s.c.OverseerTaskProcessor Cleaning up 
work-queue. #Running tasks: 0
   [junit4]   2> 692459 INFO  (jetty-launcher-665-thread-3) 
[n:127.0.0.1:33508_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 692460 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: children: /solr/overseer/queue
   [junit4]   2> 692460 DEBUG 
(OverseerCollectionConfigSetProcessor-99402037605302282-127.0.0.1:46604_solr-n_0000000000)
 [n:127.0.0.1:46604_solr    ] o.a.s.c.OverseerTaskProcessor RunningTasks: []
   [junit4]   2> 692460 DEBUG 
(OverseerCollectionConfigSetProcessor-99402037605302282-127.0.0.1:46604_solr-n_0000000000)
 [n:127.0.0.1:46604_solr    ] o.a.s.c.OverseerTaskProcessor BlockedTasks: []
   [junit4]   2> 692460 DEBUG 
(OverseerCollectionConfigSetProcessor-99402037605302282-127.0.0.1:46604_solr-n_0000000000)
 [n:127.0.0.1:46604_solr    ] o.a.s.c.OverseerTaskProcessor CompletedTasks: []
   [junit4]   2> 692460 DEBUG 
(OverseerCollectionConfigSetProcessor-99402037605302282-127.0.0.1:46604_solr-n_0000000000)
 [n:127.0.0.1:46604_solr    ] o.a.s.c.OverseerTaskProcessor RunningZKTasks: []
   [junit4]   2> 692460 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
fired: children: /solr/live_nodes
   [junit4]   2> 692460 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
fired: children: /solr/live_nodes
   [junit4]   2> 692460 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
fired: children: /solr/live_nodes
   [junit4]   2> 692460 DEBUG (jetty-launcher-665-thread-3) 
[n:127.0.0.1:33508_solr    ] o.a.s.c.LeaderElector Joined leadership election 
with path: 
/overseer_elect/election/99402037605302283-127.0.0.1:33508_solr-n_0000000003
   [junit4]   2> 692461 DEBUG 
(OverseerCollectionConfigSetProcessor-99402037605302282-127.0.0.1:46604_solr-n_0000000000)
 [n:127.0.0.1:46604_solr    ] o.a.s.c.OverseerTaskQueue Peeking for top 100 
elements. ExcludeSet: []
   [junit4]   2> 692461 DEBUG 
(OverseerAutoScalingTriggerThread-99402037605302282-127.0.0.1:46604_solr-n_0000000000)
 [n:127.0.0.1:46604_solr    ] o.a.s.c.a.NodeLostTrigger Initial livenodes: 
[127.0.0.1:43284_solr, 127.0.0.1:34953_solr]
   [junit4]   2> 692461 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: data: 
/solr/overseer_elect/election/99402037605302284-127.0.0.1:34953_solr-n_0000000002
   [junit4]   2> 692462 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: children: /solr/live_nodes
   [junit4]   2> 692462 DEBUG (jetty-launcher-665-thread-3) 
[n:127.0.0.1:33508_solr    ] o.a.s.c.LeaderElector Watching path 
/overseer_elect/election/99402037605302284-127.0.0.1:34953_solr-n_0000000002 to 
know if I could be the leader
   [junit4]   2> 692462 DEBUG 
(OverseerAutoScalingTriggerThread-99402037605302282-127.0.0.1:46604_solr-n_0000000000)
 [n:127.0.0.1:46604_solr    ] o.a.s.c.a.OverseerTriggerThread Current 
znodeVersion 1, lastZnodeVersion -1
   [junit4]   2> 692462 DEBUG (jetty-launcher-665-thread-3) 
[n:127.0.0.1:33508_solr    ] o.a.s.c.ZkController Publish 
node=127.0.0.1:33508_solr as DOWN
   [junit4]   2> 692462 DEBUG 
(OverseerAutoScalingTriggerThread-99402037605302282-127.0.0.1:46604_solr-n_0000000000)
 [n:127.0.0.1:46604_solr    ] o.a.s.c.a.OverseerTriggerThread Processed trigger 
updates upto znodeVersion 1
   [junit4]   2> 692462 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
fired: children: /solr/overseer/queue
   [junit4]   2> 692462 INFO  (jetty-launcher-665-thread-3) 
[n:127.0.0.1:33508_solr    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:33508_solr
   [junit4]   2> 692463 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
fired: children: /solr/live_nodes
   [junit4]   2> 692463 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: children: /solr/live_nodes
   [junit4]   2> 692463 INFO  
(zkCallback-697-thread-1-processing-n:127.0.0.1:46604_solr) 
[n:127.0.0.1:46604_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (2) -> (3)
   [junit4]   2> 692463 INFO  
(zkCallback-681-thread-1-processing-n:127.0.0.1:43284_solr) 
[n:127.0.0.1:43284_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (2) -> (4)
   [junit4]   2> 692464 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: children: /solr/overseer/collection-queue-work
   [junit4]   2> 692467 DEBUG 
(OverseerStateUpdate-99402037605302282-127.0.0.1:46604_solr-n_0000000000) 
[n:127.0.0.1:46604_solr    ] o.a.s.c.Overseer processMessage: queueSize: 2, 
message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:34953_solr"} current state version: 0
   [junit4]   2> 692467 DEBUG 
(OverseerStateUpdate-99402037605302282-127.0.0.1:46604_solr-n_0000000000) 
[n:127.0.0.1:46604_solr    ] o.a.s.c.o.NodeMutator DownNode state invoked for 
node: 127.0.0.1:34953_solr
   [junit4]   2> 692467 DEBUG 
(OverseerStateUpdate-99402037605302282-127.0.0.1:46604_solr-n_0000000000) 
[n:127.0.0.1:46604_solr    ] o.a.s.c.Overseer processMessage: queueSize: 2, 
message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:46604_solr"} current state version: 0
   [junit4]   2> 692467 DEBUG 
(OverseerStateUpdate-99402037605302282-127.0.0.1:46604_solr-n_0000000000) 
[n:127.0.0.1:46604_solr    ] o.a.s.c.o.NodeMutator DownNode state invoked for 
node: 127.0.0.1:46604_solr
   [junit4]   2> 692467 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: children: /solr/overseer/queue
   [junit4]   2> 692468 DEBUG 
(OverseerStateUpdate-99402037605302282-127.0.0.1:46604_solr-n_0000000000) 
[n:127.0.0.1:46604_solr    ] o.a.s.c.Overseer processMessage: queueSize: 3, 
message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:33508_solr"} current state version: 0
   [junit4]   2> 692468 DEBUG 
(OverseerStateUpdate-99402037605302282-127.0.0.1:46604_solr-n_0000000000) 
[n:127.0.0.1:46604_solr    ] o.a.s.c.o.NodeMutator DownNode state invoked for 
node: 127.0.0.1:33508_solr
   [junit4]   2> 692491 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: children: /solr/live_nodes
   [junit4]   2> 692502 DEBUG 
(OverseerAutoScalingTriggerThread-99402037605302282-127.0.0.1:46604_solr-n_0000000000)
 [n:127.0.0.1:46604_solr    ] o.a.s.c.a.OverseerTriggerThread -- clean old 
nodeAdded markers
   [junit4]   2> 692503 INFO  (jetty-launcher-665-thread-2) 
[n:127.0.0.1:43284_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_43284.solr.node' (registry 'solr.node') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@1686d525
   [junit4]   2> 692505 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: children: /solr/autoscaling/events/.auto_add_replicas
   [junit4]   2> 692505 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: children: /solr/live_nodes
   [junit4]   2> 692505 DEBUG 
(OverseerAutoScalingTriggerThread-99402037605302282-127.0.0.1:46604_solr-n_0000000000)
 [n:127.0.0.1:46604_solr    ] o.a.s.c.a.OverseerTriggerThread Current 
znodeVersion 1, lastZnodeVersion 1
   [junit4]   2> 692505 INFO  
(zkCallback-697-thread-2-processing-n:127.0.0.1:46604_solr) 
[n:127.0.0.1:46604_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (3) -> (4)
   [junit4]   2> 692523 INFO  
(zkCallback-695-thread-1-processing-n:127.0.0.1:33508_solr) 
[n:127.0.0.1:33508_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (1) -> (2)
   [junit4]   2> 692523 INFO  
(zkCallback-699-thread-1-processing-n:127.0.0.1:34953_solr) 
[n:127.0.0.1:34953_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (1) -> (4)
   [junit4]   2> 692505 DEBUG (ScheduledTrigger-1247-thread-1) 
[n:127.0.0.1:46604_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: 
.auto_add_replicas with currently live nodes: 4
   [junit4]   2> 692531 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: children: /solr/live_nodes
   [junit4]   2> 692531 INFO  
(zkCallback-695-thread-2-processing-n:127.0.0.1:33508_solr) 
[n:127.0.0.1:33508_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (2) -> (4)
   [junit4]   2> 692533 INFO  (jetty-launcher-665-thread-2) 
[n:127.0.0.1:43284_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_43284.solr.jvm' (registry 'solr.jvm') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@1686d525
   [junit4]   2> 692534 INFO  (jetty-launcher-665-thread-2) 
[n:127.0.0.1:43284_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_43284.solr.jetty' (registry 'solr.jetty') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@1686d525
   [junit4]   2> 692539 INFO  (jetty-launcher-665-thread-2) 
[n:127.0.0.1:43284_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core 
definitions underneath 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_57F4D5455C2641A7-001/tempDir-001/node2/.
   [junit4]   2> 692545 INFO  (jetty-launcher-665-thread-3) 
[n:127.0.0.1:33508_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_33508.solr.node' (registry 'solr.node') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@1686d525
   [junit4]   2> 692574 INFO  (jetty-launcher-665-thread-1) 
[n:127.0.0.1:34953_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_34953.solr.node' (registry 'solr.node') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@1686d525
   [junit4]   2> 692578 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
fired: children: /solr/overseer/queue
   [junit4]   2> 692579 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: children: /solr/overseer/queue
   [junit4]   2> 692595 INFO  (jetty-launcher-665-thread-3) 
[n:127.0.0.1:33508_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_33508.solr.jvm' (registry 'solr.jvm') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@1686d525
   [junit4]   2> 692595 INFO  (jetty-launcher-665-thread-3) 
[n:127.0.0.1:33508_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_33508.solr.jetty' (registry 'solr.jetty') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@1686d525
   [junit4]   2> 692606 INFO  (jetty-launcher-665-thread-3) 
[n:127.0.0.1:33508_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core 
definitions underneath 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_57F4D5455C2641A7-001/tempDir-001/node3/.
   [junit4]   2> 692609 INFO  (jetty-launcher-665-thread-1) 
[n:127.0.0.1:34953_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_34953.solr.jvm' (registry 'solr.jvm') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@1686d525
   [junit4]   2> 692609 INFO  (jetty-launcher-665-thread-1) 
[n:127.0.0.1:34953_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_34953.solr.jetty' (registry 'solr.jetty') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@1686d525
   [junit4]   2> 692611 INFO  (jetty-launcher-665-thread-1) 
[n:127.0.0.1:34953_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core 
definitions underneath 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_57F4D5455C2641A7-001/tempDir-001/node1/.
   [junit4]   2> 692631 INFO  (jetty-launcher-665-thread-4) 
[n:127.0.0.1:46604_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_46604.solr.node' (registry 'solr.node') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@1686d525
   [junit4]   2> 692657 INFO  (jetty-launcher-665-thread-4) 
[n:127.0.0.1:46604_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_46604.solr.jvm' (registry 'solr.jvm') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@1686d525
   [junit4]   2> 692657 INFO  (jetty-launcher-665-thread-4) 
[n:127.0.0.1:46604_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_46604.solr.jetty' (registry 'solr.jetty') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@1686d525
   [junit4]   2> 692658 INFO  (jetty-launcher-665-thread-4) 
[n:127.0.0.1:46604_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core 
definitions underneath 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_57F4D5455C2641A7-001/tempDir-001/node4/.
   [junit4]   2> 692759 INFO  (zkConnectionManagerCallback-710-thread-1) [    ] 
o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 692762 INFO  (zkConnectionManagerCallback-714-thread-1) [    ] 
o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 692762 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: data: /solr/clusterprops.json
   [junit4]   2> 692762 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: children: /solr/live_nodes
   [junit4]   2> 692762 INFO  
(SUITE-MoveReplicaHDFSTest-seed#[57F4D5455C2641A7]-worker) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 692762 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: data: /solr/clusterstate.json
   [junit4]   2> 692762 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: children: /solr/collections
   [junit4]   2> 692762 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: create/delete: /solr/aliases.json
   [junit4]   2> 692763 INFO  
(SUITE-MoveReplicaHDFSTest-seed#[57F4D5455C2641A7]-worker) [    ] 
o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:33010/solr ready
   [junit4]   2> 692790 INFO  (qtp349227651-3341) [n:127.0.0.1:34953_solr    ] 
o.a.s.h.a.CollectionsHandler Invoked Collection Action :overseerstatus with 
params action=OVERSEERSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 692790 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
fired: children: /solr/overseer/collection-queue-work
   [junit4]   2> 692790 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: create/delete: /solr/overseer/collection-queue-work/qnr-0000000000
   [junit4]   2> 692790 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: children: /solr/overseer/collection-queue-work
   [junit4]   2> 692790 DEBUG 
(OverseerCollectionConfigSetProcessor-99402037605302282-127.0.0.1:46604_solr-n_0000000000)
 [n:127.0.0.1:46604_solr    ] o.a.s.c.ZkDistributedQueue Found child node with 
improper name: qnr-0000000000
   [junit4]   2> 692790 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
fired: children: /solr/overseer/collection-queue-work
   [junit4]   2> 692791 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: children: /solr/overseer/collection-queue-work
   [junit4]   2> 692791 DEBUG 
(OverseerCollectionConfigSetProcessor-99402037605302282-127.0.0.1:46604_solr-n_0000000000)
 [n:127.0.0.1:46604_solr    ] o.a.s.c.ZkDistributedQueue Found child node with 
improper name: qnr-0000000000
   [junit4]   2> 692791 DEBUG 
(OverseerCollectionConfigSetProcessor-99402037605302282-127.0.0.1:46604_solr-n_0000000000)
 [n:127.0.0.1:46604_solr    ] o.a.s.c.OverseerTaskQueue Returning topN 
elements: [/overseer/collection-queue-work/qn-0000000000, ]
   [junit4]   2> 692791 DEBUG 
(OverseerCollectionConfigSetProcessor-99402037605302282-127.0.0.1:46604_solr-n_0000000000)
 [n:127.0.0.1:46604_solr    ] o.a.s.c.OverseerTaskProcessor Got 1 tasks from 
work-queue : [[org.apache.solr.cloud.OverseerTaskQueue$QueueEvent@2e684ecf]]
   [junit4]   2> 692791 DEBUG 
(OverseerCollectionConfigSetProcessor-99402037605302282-127.0.0.1:46604_solr-n_0000000000)
 [n:127.0.0.1:46604_solr    ] o.a.s.c.OverseerTaskProcessor Marked task 
[/overseer/collection-queue-work/qn-0000000000] as running
   [junit4]   2> 692791 DEBUG 
(OverseerCollectionConfigSetProcessor-99402037605302282-127.0.0.1:46604_solr-n_0000000000)
 [n:127.0.0.1:46604_solr    ] o.a.s.c.OverseerTaskProcessor Overseer Collection 
Message Handler: Get the message 
id:/overseer/collection-queue-work/qn-0000000000 
message:{"operation":"overseerstatus"}
   [junit4]   2> 692793 DEBUG 
(OverseerThreadFactory-1250-thread-1-processing-n:127.0.0.1:46604_solr) 
[n:127.0.0.1:46604_solr    ] o.a.s.c.OverseerTaskProcessor Runner processing 
/overseer/collection-queue-work/qn-0000000000
   [junit4]   2> 692793 DEBUG 
(OverseerCollectionConfigSetProcessor-99402037605302282-127.0.0.1:46604_solr-n_0000000000)
 [n:127.0.0.1:46604_solr    ] o.a.s.c.OverseerTaskProcessor Cleaning up 
work-queue. #Running tasks: 1
   [junit4]   2> 692793 DEBUG 
(OverseerThreadFactory-1250-thread-1-processing-n:127.0.0.1:46604_solr) 
[n:127.0.0.1:46604_solr    ] o.a.s.c.a.c.OverseerCollectionMessageHandler 
OverseerCollectionMessageHandler.processMessage : overseerstatus , 
{"operation":"overseerstatus"}
   [junit4]   2> 692793 DEBUG 
(OverseerCollectionConfigSetProcessor-99402037605302282-127.0.0.1:46604_solr-n_0000000000)
 [n:127.0.0.1:46604_solr    ] o.a.s.c.OverseerTaskProcessor RunningTasks: 
[/overseer/collection-queue-work/qn-0000000000]
   [junit4]   2> 692793 DEBUG 
(OverseerCollectionConfigSetProcessor-99402037605302282-127.0.0.1:46604_solr-n_0000000000)
 [n:127.0.0.1:46604_solr    ] o.a.s.c.OverseerTaskProcessor BlockedTasks: []
   [junit4]   2> 692793 DEBUG 
(OverseerCollectionConfigSetProcessor-99402037605302282-127.0.0.1:46604_solr-n_0000000000)
 [n:127.0.0.1:46604_solr    ] o.a.s.c.OverseerTaskProcessor CompletedTasks: []
   [junit4]   2> 692793 DEBUG 
(OverseerCollectionConfigSetProcessor-99402037605302282-127.0.0.1:46604_solr-n_0000000000)
 [n:127.0.0.1:46604_solr    ] o.a.s.c.OverseerTaskProcessor RunningZKTasks: 
[/overseer/collection-queue-work/qn-0000000000]
   [junit4]   2> 692793 DEBUG 
(OverseerCollectionConfigSetProcessor-99402037605302282-127.0.0.1:46604_solr-n_0000000000)
 [n:127.0.0.1:46604_solr    ] o.a.s.c.OverseerTaskQueue Peeking for top 99 
elements. ExcludeSet: [/overseer/collection-queue-work/qn-0000000000],[]
   [junit4]   2> 692798 DEBUG 
(OverseerThreadFactory-1250-thread-1-processing-n:127.0.0.1:46604_solr) 
[n:127.0.0.1:46604_solr    ] o.a.s.c.OverseerTaskProcessor Completed 
task:[/overseer/collection-queue-work/qn-0000000000]
   [junit4]   2> 692799 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
fired: data: /solr/overseer/collection-queue-work/qnr-0000000000
   [junit4]   2> 692799 DEBUG 
(zkCallback-699-thread-1-processing-n:127.0.0.1:34953_solr) 
[n:127.0.0.1:34953_solr    ] o.a.s.c.OverseerTaskQueue NodeDataChanged fired on 
path /overseer/collection-queue-work/qnr-0000000000 state SyncConnected 
latchEventType null
   [junit4]   2> 692799 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
fired: children: /solr/overseer/collection-queue-work
   [junit4]   2> 692799 DEBUG 
(OverseerThreadFactory-1250-thread-1-processing-n:127.0.0.1:46604_solr) 
[n:127.0.0.1:46604_solr    ] o.a.s.c.OverseerTaskProcessor Marked task 
[/overseer/collection-queue-work/qn-0000000000] as completed.
   [junit4]   2> 692799 DEBUG 
(OverseerThreadFactory-1250-thread-1-processing-n:127.0.0.1:46604_solr) 
[n:127.0.0.1:46604_solr    ] o.a.s.c.OverseerTaskProcessor RunningTasks: []
   [junit4]   2> 692799 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: children: /solr/overseer/collection-queue-work
   [junit4]   2> 692799 DEBUG 
(OverseerCollectionConfigSetProcessor-99402037605302282-127.0.0.1:46604_solr-n_0000000000)
 [n:127.0.0.1:46604_solr    ] o.a.s.c.ZkDistributedQueue Found child node with 
improper name: qnr-0000000000
   [junit4]   2> 692799 DEBUG 
(OverseerThreadFactory-1250-thread-1-processing-n:127.0.0.1:46604_solr) 
[n:127.0.0.1:46604_solr    ] o.a.s.c.OverseerTaskProcessor BlockedTasks: []
   [junit4]   2> 692799 DEBUG 
(OverseerThreadFactory-1250-thread-1-processing-n:127.0.0.1:46604_solr) 
[n:127.0.0.1:46604_solr    ] o.a.s.c.OverseerTaskProcessor CompletedTasks: 
[/overseer/collection-queue-work/qn-0000000000]
   [junit4]   2> 692799 DEBUG 
(OverseerThreadFactory-1250-thread-1-processing-n:127.0.0.1:46604_solr) 
[n:127.0.0.1:46604_solr    ] o.a.s.c.OverseerTaskProcessor RunningZKTasks: 
[/overseer/collection-queue-work/qn-0000000000]
   [junit4]   2> 692799 DEBUG 
(OverseerThreadFactory-1250-thread-1-processing-n:127.0.0.1:46604_solr) 
[n:127.0.0.1:46604_solr    ] o.a.s.c.OverseerTaskProcessor Overseer Collection 
Message Handler: Message id:/overseer/collection-queue-work/qn-0000000000 
complete, 
response:{leader=127.0.0.1:46604_solr,overseer_queue_size=0,overseer_work_queue_size=0,overseer_collection_queue_size=2,overseer_operations={am_i_leader={requests=3,errors=0,avgRequestsPerSecond=8.725852399652894,5minRateRequestsPerSecond=0.0,15minRateRequestsPerSecond=0.0,avgTimePerRequest=0.7657733333333333,medianRequestTime=0.356973,75thPcRequestTime=1.805921,95thPcRequestTime=1.805921,99thPcRequestTime=1.805921,999thPcRequestTime=1.805921},downnode={requests=3,errors=0,avgRequestsPerSecond=9.129854421580127,5minRateRequestsPerSecond=0.0,15minRateRequestsPerSecond=0.0,avgTimePerRequest=0.02617533333333333,medianRequestTime=0.025772,75thPcRequestTime=0.030854,95thPcRequestTime=0.030854,99thPcRequestTime=0.030854,999thPcRequestTime=0.030854}},collection_operations={am_i_leader={requests=3,errors=0,avgRequestsPerSecond=8.777257953777807,5minRateRequestsPerSecond=0.0,15minRateRequestsPerSecond=0.0,avgTimePerRequest=0.2737306666666666,medianRequestTime=0.298219,75thPcRequestTime=0.384,95thPcRequestTime=0.384,99thPcRequestTime=0.384,999thPcRequestTime=0.384},overseerstatus={requests=0,errors=0,avgRequestsPerSecond=0.0,5minRateRequestsPerSecond=0.0,15minRateRequestsPerSecond=0.0,avgTimePerRequest=0.0,medianRequestTime=0.0,75thPcRequestTime=0.0,95thPcRequestTime=0.0,99thPcRequestTime=0.0,999thPcRequestTime=0.0}},overseer_queue={},overseer_internal_queue={peek={avgRequestsPerSecond=2.9753281180691125,5minRateRequestsPerSecond=0.0,15minRateRequestsPerSecond=0.0,avgTimePerRequest=0.361412,medianRequestTime=0.361412,75thPcRequestTime=0.361412,95thPcRequestTime=0.361412,99thPcRequestTime=0.361412,999thPcRequestTime=0.361412}},collection_queue={peektopn_wait2000={avgRequestsPerSecond=2.993599370218539,5minRateRequestsPerSecond=0.0,15minRateRequestsPerSecond=0.0,avgTimePerRequest=329.481271,medianRequestTime=329.481271,75thPcRequestTime=329.481271,95thPcRequestTime=329.481271,99thPcRequestTime=329.481271,999thPcRequestTime=329.481271}}}
   [junit4]   2> 692803 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
fired: children: /solr/overseer/collection-queue-work
   [junit4]   2> 692803 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: children: /solr/overseer/collection-queue-work
   [junit4]   2> 692806 INFO  (qtp349227651-3341) [n:127.0.0.1:34953_solr    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections 
params={action=OVERSEERSTATUS&wt=javabin&version=2} status=0 QTime=16
   [junit4]   2> 693307 WARN  
(SUITE-MoveReplicaHDFSTest-seed#[57F4D5455C2641A7]-worker) [    ] 
o.a.h.u.NativeCodeLoader Unable to load native-hadoop library for your 
platform... using builtin-java classes where applicable
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 693532 DEBUG (ScheduledTrigger-1247-thread-1) 
[n:127.0.0.1:46604_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: 
.auto_add_replicas with currently live nodes: 4
   [junit4]   2> 693797 WARN  
(SUITE-MoveReplicaHDFSTest-seed#[57F4D5455C2641A7]-worker) [    ] 
o.a.h.m.i.MetricsConfig Cannot locate configuration: tried 
hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 693924 INFO  
(SUITE-MoveReplicaHDFSTest-seed#[57F4D5455C2641A7]-worker) [    ] o.m.log 
Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via 
org.mortbay.log.Slf4jLog
   [junit4]   2> 693943 WARN  
(SUITE-MoveReplicaHDFSTest-seed#[57F4D5455C2641A7]-worker) [    ] 
o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 694196 INFO  
(SUITE-MoveReplicaHDFSTest-seed#[57F4D5455C2641A7]-worker) [    ] o.m.log 
jetty-6.1.26
   [junit4]   2> 694230 INFO  
(SUITE-MoveReplicaHDFSTest-seed#[57F4D5455C2641A7]-worker) [    ] o.m.log 
Extract 
jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/hdfs
 to ./temp/Jetty_lucene2.us.west_apache_org_33312_hdfs____gmfvdq/webapp
   [junit4]   2> 694533 DEBUG (ScheduledTrigger-1247-thread-2) 
[n:127.0.0.1:46604_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: 
.auto_add_replicas with currently live nodes: 4
   [junit4]   2> 694794 DEBUG 
(OverseerCollectionConfigSetProcessor-99402037605302282-127.0.0.1:46604_solr-n_0000000000)
 [n:127.0.0.1:46604_solr    ] o.a.s.c.OverseerTaskProcessor Got 0 tasks from 
work-queue : [[]]
   [junit4]   2> 694822 DEBUG 
(OverseerCollectionConfigSetProcessor-99402037605302282-127.0.0.1:46604_solr-n_0000000000)
 [n:127.0.0.1:46604_solr    ] o.a.s.c.OverseerTaskProcessor Cleaning up 
work-queue. #Running tasks: 0
   [junit4]   2> 694823 INFO  
(OverseerCollectionConfigSetProcessor-99402037605302282-127.0.0.1:46604_solr-n_0000000000)
 [n:127.0.0.1:46604_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: 
/overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may 
have disconnected from ZooKeeper
   [junit4]   2> 694823 DEBUG 
(OverseerCollectionConfigSetProcessor-99402037605302282-127.0.0.1:46604_solr-n_0000000000)
 [n:127.0.0.1:46604_solr    ] o.a.s.c.OverseerTaskProcessor RunningTasks: []
   [junit4]   2> 694823 DEBUG 
(OverseerCollectionConfigSetProcessor-99402037605302282-127.0.0.1:46604_solr-n_0000000000)
 [n:127.0.0.1:46604_solr    ] o.a.s.c.OverseerTaskProcessor BlockedTasks: []
   [junit4]   2> 694823 DEBUG 
(OverseerCollectionConfigSetProcessor-99402037605302282-127.0.0.1:46604_solr-n_0000000000)
 [n:127.0.0.1:46604_solr    ] o.a.s.c.OverseerTaskProcessor CompletedTasks: []
   [junit4]   2> 694823 DEBUG 
(OverseerCollectionConfigSetProcessor-99402037605302282-127.0.0.1:46604_solr-n_0000000000)
 [n:127.0.0.1:46604_solr    ] o.a.s.c.OverseerTaskProcessor RunningZKTasks: []
   [junit4]   2> 694823 DEBUG 
(OverseerCollectionConfigSetProcessor-99402037605302282-127.0.0.1:46604_solr-n_0000000000)
 [n:127.0.0.1:46604_solr    ] o.a.s.c.OverseerTaskQueue Peeking for top 100 
elements. ExcludeSet: []
   [junit4]   2> 695339 INFO  
(SUITE-MoveReplicaHDFSTest-seed#[57F4D5455C2641A7]-worker) [    ] o.m.log 
Started 
HttpServer2$selectchannelconnectorwithsafestar...@lucene2-us-west.apache.org:33312
   [junit4]   2> 695533 DEBUG (ScheduledTrigger-1247-thread-1) 
[n:127.0.0.1:46604_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: 
.auto_add_replicas with currently live nodes: 4
   [junit4]   2> 695943 WARN  
(SUITE-MoveReplicaHDFSTest-seed#[57F4D5455C2641A7]-worker) [    ] 
o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 695946 INFO  
(SUITE-MoveReplicaHDFSTest-seed#[57F4D5455C2641A7]-worker) [    ] o.m.log 
jetty-6.1.26
   [junit4]   2> 695963 INFO  
(SUITE-MoveReplicaHDFSTest-seed#[57F4D5455C2641A7]-worker) [    ] o.m.log 
Extract 
jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode
 to ./temp/Jetty_localhost_35649_datanode____l0pc00/webapp
   [junit4]   2> 696282 INFO  
(SUITE-MoveReplicaHDFSTest-seed#[57F4D5455C2641A7]-worker) [    ] o.m.log 
Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:35649
   [junit4]   2> 696537 DEBUG (ScheduledTrigger-1247-thread-1) 
[n:127.0.0.1:46604_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: 
.auto_add_replicas with currently live nodes: 4
   [junit4]   2> 696646 WARN  
(SUITE-MoveReplicaHDFSTest-seed#[57F4D5455C2641A7]-worker) [    ] 
o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 696647 INFO  
(SUITE-MoveReplicaHDFSTest-seed#[57F4D5455C2641A7]-worker) [    ] o.m.log 
jetty-6.1.26
   [junit4]   2> 696656 INFO  
(SUITE-MoveReplicaHDFSTest-seed#[57F4D5455C2641A7]-worker) [    ] o.m.log 
Extract 
jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode
 to ./temp/Jetty_localhost_44224_datanode____.6wfbfp/webapp
   [junit4]   2> 696823 DEBUG 
(OverseerCollectionConfigSetProcessor-99402037605302282-127.0.0.1:46604_solr-n_0000000000)
 [n:127.0.0.1:46604_solr    ] o.a.s.c.OverseerTaskProcessor Got 0 tasks from 
work-queue : [[]]
   [junit4]   2> 696824 DEBUG 
(OverseerCollectionConfigSetProcessor-99402037605302282-127.0.0.1:46604_solr-n_0000000000)
 [n:127.0.0.1:46604_solr    ] o.a.s.c.OverseerTaskProcessor Cleaning up 
work-queue. #Running tasks: 0
   [junit4]   2> 696824 DEBUG 
(OverseerCollectionConfigSetProcessor-99402037605302282-127.0.0.1:46604_solr-n_0000000000)
 [n:127.0.0.1:46604_solr    ] o.a.s.c.OverseerTaskProcessor RunningTasks: []
   [junit4]   2> 696824 DEBUG 
(OverseerCollectionConfigSetProcessor-99402037605302282-127.0.0.1:46604_solr-n_0000000000)
 [n:127.0.0.1:46604_solr    ] o.a.s.c.OverseerTaskProcessor BlockedTasks: []
   [junit4]   2> 696824 DEBUG 
(OverseerCollectionConfigSetProcessor-99402037605302282-127.0.0.1:46604_solr-n_0000000000)
 [n:127.0.0.1:46604_solr    ] o.a.s.c.OverseerTaskProcessor CompletedTasks: []
   [junit4]   2> 696824 DEBUG 
(OverseerCollectionConfigSetProcessor-99402037605302282-127.0.0.1:46604_solr-n_0000000000)
 [n:127.0.0.1:46604_solr    ] o.a.s.c.OverseerTaskProcessor RunningZKTasks: []
   [junit4]   2> 696824 DEBUG 
(OverseerCollectionConfigSetProcessor-99402037605302282-127.0.0.1:46604_solr-n_0000000000)
 [n:127.0.0.1:46604_solr    ] o.a.s.c.OverseerTaskQueue Peeking for top 100 
elements. ExcludeSet: []
   [junit4]   2> 697016 INFO  
(SUITE-MoveReplicaHDFSTest-seed#[57F4D5455C2641A7]-worker) [    ] o.m.log 
Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:44224
   [junit4]   2> 697155 ERROR (DataNode: 
[[[DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_57F4D5455C2641A7-001/tempDir-002/hdfsBaseDir/data/data1/,
 
[DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_57F4D5455C2641A7-001/tempDir-002/hdfsBaseDir/data/data2/]]
  heartbeating to lucene2-us-west.apache.org/127.0.0.1:33378) [    ] 
o.a.h.h.s.d.DirectoryScanner 
dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 
ms/sec. Assuming default value of 1000
   [junit4]   2> 697269 ERROR (DataNode: 
[[[DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_57F4D5455C2641A7-001/tempDir-002/hdfsBaseDir/data/data3/,
 
[DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_57F4D5455C2641A7-001/tempDir-002/hdfsBaseDir/data/data4/]]
  heartbeating to lucene2-us-west.apache.org/127.0.0.1:33378) [    ] 
o.a.h.h.s.d.DirectoryScanner 
dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 
ms/sec. Assuming default value of 1000
   [junit4]   2> 697320 INFO  (Block report processor) [    ] BlockStateChange 
BLOCK* processReport 0x44ba2899ec99c: from storage 
DS-303225c4-c948-4e58-8d56-50735a8e5f68 node 
DatanodeRegistration(127.0.0.1:35420, 
datanodeUuid=97597ef9-e5c4-409a-8491-e00de70eebc5, infoPort=42337, 
infoSecurePort=0, ipcPort=44618, 
storageInfo=lv=-56;cid=testClusterID;nsid=716216287;c=0), blocks: 0, 
hasStaleStorage: true, processing time: 2 msecs
   [junit4]   2> 697320 INFO  (Block report processor) [    ] BlockStateChange 
BLOCK* processReport 0x44ba2899ec99c: from storage 
DS-7a614ec1-4607-4fc0-bc85-c7c9f6c65aaf node 
DatanodeRegistration(127.0.0.1:40027, 
datanodeUuid=85bfcbaf-197c-4613-b4af-d45beba10f3f, infoPort=46329, 
infoSecurePort=0, ipcPort=39261, 
storageInfo=lv=-56;cid=testClusterID;nsid=716216287;c=0), blocks: 0, 
hasStaleStorage: true, processing time: 0 msecs
   [junit4]   2> 697320 INFO  (Block report processor) [    ] BlockStateChange 
BLOCK* processReport 0x44ba2899ec99c: from storage 
DS-82bf9cc6-450d-4e9e-97da-f98993c416d0 node 
DatanodeRegistration(127.0.0.1:35420, 
datanodeUuid=97597ef9-e5c4-409a-8491-e00de70eebc5, infoPort=42337, 
infoSecurePort=0, ipcPort=44618, 
storageInfo=lv=-56;cid=testClusterID;nsid=716216287;c=0), blocks: 0, 
hasStaleStorage: false, processing time: 0 msecs
   [junit4]   2> 697321 INFO  (Block report processor) [    ] BlockStateChange 
BLOCK* processReport 0x44ba2899ec99c: from storage 
DS-69cec72d-debc-4cd6-ae68-ac409ca623ba node 
DatanodeRegistration(127.0.0.1:40027, 
datanodeUuid=85bfcbaf-197c-4613-b4af-d45beba10f3f, infoPort=46329, 
infoSecurePort=0, ipcPort=39261, 
storageInfo=lv=-56;cid=testClusterID;nsid=716216287;c=0), blocks: 0, 
hasStaleStorage: false, processing time: 0 msecs
   [junit4]   2> 697537 DEBUG (ScheduledTrigger-1247-thread-2) 
[n:127.0.0.1:46604_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: 
.auto_add_replicas with currently live nodes: 4
   [junit4]   2> 697682 INFO  
(TEST-MoveReplicaHDFSTest.testNormalFailedMove-seed#[57F4D5455C2641A7]) [    ] 
o.a.s.SolrTestCaseJ4 ###Starting testNormalFailedMove
   [junit4]   2> 697683 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: data: /solr/clusterprops.json
   [junit4]   2> 697683 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: children: /solr/live_nodes
   [junit4]   2> 697683 INFO  
(TEST-MoveReplicaHDFSTest.testNormalFailedMove-seed#[57F4D5455C2641A7]) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 697683 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: data: /solr/clusterstate.json
   [junit4]   2> 697683 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: children: /solr/collections
   [junit4]   2> 697684 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: create/delete: /solr/aliases.json
   [junit4]   2> 697685 INFO  
(TEST-MoveReplicaHDFSTest.testNormalFailedMove-seed#[57F4D5455C2641A7]) [    ] 
o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 
2017-11-22T05:27:37+08:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 697686 INFO  
(TEST-MoveReplicaHDFSTest.testNormalFailedMove-seed#[57F4D5455C2641A7]) [    ] 
o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 697686 INFO  
(TEST-MoveReplicaHDFSTest.testNormalFailedMove-seed#[57F4D5455C2641A7]) [    ] 
o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 697686 INFO  
(TEST-MoveReplicaHDFSTest.testNormalFailedMove-seed#[57F4D5455C2641A7]) [    ] 
o.e.j.s.session Scavenging every 660000ms
   [junit4]   2> 697686 INFO  
(TEST-MoveReplicaHDFSTest.testNormalFailedMove-seed#[57F4D5455C2641A7]) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@600379d8{/solr,null,AVAILABLE}
   [junit4]   2> 697687 INFO  
(TEST-MoveReplicaHDFSTest.testNormalFailedMove-seed#[57F4D5455C2641A7]) [    ] 
o.e.j.s.AbstractConnector Started ServerConnector@571f2b9e{SSL,[ssl, 
http/1.1]}{127.0.0.1:41922}
   [junit4]   2> 697687 INFO  
(TEST-MoveReplicaHDFSTest.testNormalFailedMove-seed#[57F4D5455C2641A7]) [    ] 
o.e.j.s.Server Started @706126ms
   [junit4]   2> 697687 INFO  
(TEST-MoveReplicaHDFSTest.testNormalFailedMove-seed#[57F4D5455C2641A7]) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, 
hostPort=41922}
   [junit4]   2> 697687 ERROR 
(TEST-MoveReplicaHDFSTest.testNormalFailedMove-seed#[57F4D5455C2641A7]) [    ] 
o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 697687 INFO  
(TEST-MoveReplicaHDFSTest.testNormalFailedMove-seed#[57F4D5455C2641A7]) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 
8.0.0
   [junit4]   2> 697687 INFO  
(TEST-MoveReplicaHDFSTest.testNormalFailedMove-seed#[57F4D5455C2641A7]) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 697687 INFO  
(TEST-MoveReplicaHDFSTest.testNormalFailedMove-seed#[57F4D5455C2641A7]) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 697687 INFO  
(TEST-MoveReplicaHDFSTest.testNormalFailedMove-seed#[57F4D5455C2641A7]) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2018-01-24T00:45:29.759Z
   [junit4]   2> 697692 INFO  (zkConnectionManagerCallback-717-thread-1) [    ] 
o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 697692 INFO  
(TEST-MoveReplicaHDFSTest.testNormalFailedMove-seed#[57F4D5455C2641A7]) [    ] 
o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 697713 INFO  
(TEST-MoveReplicaHDFSTest.testNormalFailedMove-seed#[57F4D5455C2641A7]) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33010/solr
   [junit4]   2> 697714 INFO  (zkConnectionManagerCallback-721-thread-1) [    ] 
o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 697716 DEBUG 
(TEST-MoveReplicaHDFSTest.testNormalFailedMove-seed#[57F4D5455C2641A7]) 
[n:127.0.0.1:41922_solr    ] o.a.s.c.ZkController Added new OnReconnect 
listener org.apache.solr.cloud.ZkController$$Lambda$253/628261647@2fcf69c9
   [junit4]   2> 697717 INFO  
(zkConnectionManagerCallback-723-thread-1-processing-n:127.0.0.1:41922_solr) 
[n:127.0.0.1:41922_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 697721 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: data: /solr/clusterprops.json
   [junit4]   2> 697721 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: children: /solr/live_nodes
   [junit4]   2> 697721 INFO  
(TEST-MoveReplicaHDFSTest.testNormalFailedMove-seed#[57F4D5455C2641A7]) 
[n:127.0.0.1:41922_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (4)
   [junit4]   2> 697721 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: data: /solr/clusterstate.json
   [junit4]   2> 697721 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: children: /solr/collections
   [junit4]   2> 697721 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: create/delete: /solr/aliases.json
   [junit4]   2> 697721 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: create/delete: /solr/security.json
   [junit4]   2> 697722 INFO  
(TEST-MoveReplicaHDFSTest.testNormalFailedMove-seed#[57F4D5455C2641A7]) 
[n:127.0.0.1:41922_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 697722 DEBUG 
(TEST-MoveReplicaHDFSTest.testNormalFailedMove-seed#[57F4D5455C2641A7]) 
[n:127.0.0.1:41922_solr    ] o.a.s.c.LeaderElector Joined leadership election 
with path: 
/overseer_elect/election/99402037605302289-127.0.0.1:41922_solr-n_0000000004
   [junit4]   2> 697723 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: data: 
/solr/overseer_elect/election/99402037605302283-127.0.0.1:33508_solr-n_0000000003
   [junit4]   2> 697723 DEBUG 
(TEST-MoveReplicaHDFSTest.testNormalFailedMove-seed#[57F4D5455C2641A7]) 
[n:127.0.0.1:41922_solr    ] o.a.s.c.LeaderElector Watching path 
/overseer_elect/election/99402037605302283-127.0.0.1:33508_solr-n_0000000003 to 
know if I could be the leader
   [junit4]   2> 697723 DEBUG 
(TEST-MoveReplicaHDFSTest.testNormalFailedMove-seed#[57F4D5455C2641A7]) 
[n:127.0.0.1:41922_solr    ] o.a.s.c.ZkController Publish 
node=127.0.0.1:41922_solr as DOWN
   [junit4]   2> 697724 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
fired: children: /solr/overseer/queue
   [junit4]   2> 697724 INFO  
(TEST-MoveReplicaHDFSTest.testNormalFailedMove-seed#[57F4D5455C2641A7]) 
[n:127.0.0.1:41922_solr    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:41922_solr
   [junit4]   2> 697724 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: children: /solr/overseer/queue
   [junit4]   2> 697724 DEBUG 
(OverseerStateUpdate-99402037605302282-127.0.0.1:46604_solr-n_0000000000) 
[n:127.0.0.1:46604_solr    ] o.a.s.c.Overseer processMessage: queueSize: 1, 
message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:41922_solr"} current state version: 0
   [junit4]   2> 697724 DEBUG 
(OverseerStateUpdate-99402037605302282-127.0.0.1:46604_solr-n_0000000000) 
[n:127.0.0.1:46604_solr    ] o.a.s.c.o.NodeMutator DownNode state invoked for 
node: 127.0.0.1:41922_solr
   [junit4]   2> 697725 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
fired: children: /solr/live_nodes
   [junit4]   2> 697725 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
fired: children: /solr/live_nodes
   [junit4]   2> 697725 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
fired: children: /solr/live_nodes
   [junit4]   2> 697725 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
fired: children: /solr/live_nodes
   [junit4]   2> 697725 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
fired: children: /solr/live_nodes
   [junit4]   2> 697725 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
fired: children: /solr/live_nodes
   [junit4]   2> 697725 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: children: /solr/live_nodes
   [junit4]   2> 697725 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: children: /solr/live_nodes
   [junit4]   2> 697725 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: children: /solr/live_nodes
   [junit4]   2> 697725 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: children: /solr/live_nodes
   [junit4]   2> 697725 INFO  
(zkCallback-695-thread-2-processing-n:127.0.0.1:33508_solr) 
[n:127.0.0.1:33508_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (4) -> (5)
   [junit4]   2> 697725 INFO  
(zkCallback-699-thread-1-processing-n:127.0.0.1:34953_solr) 
[n:127.0.0.1:34953_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (4) -> (5)
   [junit4]   2> 697726 INFO  (zkCallback-713-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 697727 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: children: /solr/live_nodes
   [junit4]   2> 697727 INFO  
(zkCallback-697-thread-2-processing-n:127.0.0.1:46604_solr) 
[n:127.0.0.1:46604_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (4) -> (5)
   [junit4]   2> 697727 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: children: /solr/live_nodes
   [junit4]   2> 697727 INFO  (zkCallback-713-thread-2) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 697727 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch 
added: children: /solr/live_nodes
   [junit4]   2> 697727 INFO  
(zkCallback-681-thread-1-processing-n:127.0.0.1:43284_solr) 
[n:127.0.0.1:43284_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (4) -> (5)
   [junit4]   2> 697735 INFO  
(zkCallback-722-thread-1-processing-n:127.0.0.1:41922_solr) 
[n:127.0.0.1:41922_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (4) -> (5)
   [junit4]   2> 697777 INFO  
(TEST-MoveReplicaHDFSTest.testNormalFailedMove-seed#[57F4D5455C2641A7]) 
[n:127.0.0.1:41922_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_41922.solr.node' (registry 'solr.node') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@1686d525
   [junit4]   2> 697795 INFO  
(TEST-MoveReplicaHDFSTest.testNormalFailedMove-seed#[57F4D5455C2641A7]) 
[n:127.0.0.1:41922_solr 

[...truncated too long message...]

tLegacyNumericUtils, BadComponentTest, TestPayloadScoreQParserPlugin, 
TestInitParams, BasicFunctionalityTest, TestPointFields, RemoteQueryErrorTest, 
DocExpirationUpdateProcessorFactoryTest, TestSolrXml, 
DistributedMLTComponentTest, UpdateLogTest, TestCloudSchemaless, SuggesterTest, 
SolrCoreTest, AutoScalingHandlerTest, TestReversedWildcardFilterFactory, 
TestConfigSetsAPIExclusivity, QueryElevationComponentTest, DOMUtilTest, 
RuleEngineTest, TestSolrConfigHandlerConcurrent, TestXIncludeConfig, 
MissingSegmentRecoveryTest, DocValuesMissingTest, AnalyticsQueryTest, 
TestFieldSortValues, DistribDocExpirationUpdateProcessorTest, 
TestOnReconnectListenerSupport, DistributedFacetSimpleRefinementLongTailTest, 
ZkControllerTest, BasicDistributedZk2Test, UnloadDistributedZkTest, 
SyncSliceTest, OpenCloseCoreStressTest, OverseerTest, BasicZkTest, 
FullSolrCloudDistribCmdsTest, TestReplicationHandler, TestRandomFaceting, 
TestDistributedGrouping, TestReload, TestCoreContainer, SimpleFacetsTest, 
SolrCmdDistributorTest, BadIndexSchemaTest, ConvertedLegacyTest, TestSort, 
TestLazyCores, DistributedQueryElevationComponentTest, SolrIndexSplitterTest, 
SimplePostToolTest, AnalysisAfterCoreReloadTest, DocValuesTest, 
TestStressLucene, TestTrie, SpatialFilterTest, SuggesterWFSTTest, 
WordBreakSolrSpellCheckerTest, SchemaVersionSpecificBehaviorTest, 
SolrCoreCheckLockOnStartupTest, TestUpdate, TestWordDelimiterFilterFactory, 
QueryEqualityTest, StatelessScriptUpdateProcessorFactoryTest, 
SortByFunctionTest, TestRemoteStreaming, DistanceFunctionTest, 
TestSolrDeletionPolicy1, SolrInfoBeanTest, DebugComponentTest, CacheHeaderTest, 
IndexBasedSpellCheckerTest, TestQueryUtils, DirectSolrSpellCheckerTest, 
PrimitiveFieldTypeTest, TestOmitPositions, TermsComponentTest, 
XmlUpdateRequestHandlerTest, TestValueSourceCache, 
PathHierarchyTokenizerFactoryTest, TermVectorComponentTest, 
MoreLikeThisHandlerTest, LoggingHandlerTest, ReturnFieldsTest, 
TestCSVResponseWriter, QueryParsingTest, UniqFieldsUpdateProcessorFactoryTest, 
JsonLoaderTest, BinaryUpdateRequestHandlerTest, TestPartialUpdateDeduplication, 
PingRequestHandlerTest, SearchHandlerTest, UpdateParamsTest, 
AlternateDirectoryTest, ResponseLogComponentTest, SolrIndexConfigTest, 
TestStressRecovery, TestMergePolicyConfig, TestSolrDeletionPolicy2, 
MultiTermTest, SampleTest, TestDocSet, NumericFieldsTest, 
TestElisionMultitermQuery, TestPostingsSolrHighlighter, 
DirectSolrConnectionTest, NotRequiredUniqueKeyTest, 
SpellPossibilityIteratorTest, TestCharFilters, TestDFRSimilarityFactory, 
OpenExchangeRatesOrgProviderTest, ChaosMonkeyNothingIsSafeTest, 
CollectionStateFormat2Test, ConnectionManagerTest, CreateCollectionCleanupTest, 
DeleteInactiveReplicaTest, DeleteStatusTest, DocValuesNotIndexedTest, 
LeaderFailoverAfterPartitionTest, OverseerModifyCollectionTest, 
OverseerStatusTest, RecoveryAfterSoftCommitTest, 
SharedFSAutoReplicaFailoverTest, SolrCLIZkUtilsTest, SolrXmlInZkTest, 
TestCloudDeleteByQuery, TestConfigSetsAPIZkFailure, 
TestDeleteCollectionOnDownNodes, TestDistribDocBasedVersion, 
TestLeaderElectionZkExpiry, TestRequestForwarding, TestSkipOverseerOperations, 
TestStressCloudBlindAtomicUpdates, TestTolerantUpdateProcessorRandomCloud, 
TestUtilizeNode, TriLevelCompositeIdRoutingTest, 
VMParamsZkACLAndCredentialsProvidersTest, AssignTest, 
ConcurrentDeleteAndCreateCollectionTest, CustomCollectionTest, 
TestCollectionAPI, TestHdfsCloudBackupRestore, ComputePlanActionTest, 
TestDistribStateManager, TestExecutePlanAction, TestLargeCluster]
   [junit4] Completed [586/768 (3!)] on J1 in 62.38s, 4 tests, 1 error, 1 
skipped <<< FAILURES!

[...truncated 3687 lines...]
   [junit4] Suite: org.apache.solr.analytics.OverallAnalyticsTest
   [junit4]   2> Creating dataDir: 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/contrib/solr-analytics/test/J2/temp/solr.analytics.OverallAnalyticsTest_2D4C3C614787FA5E-001/init-core-data-001
   [junit4]   2> Jan 24, 2018 1:53:36 AM 
com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked 
thread(s).
   [junit4]   2> Jan 24, 2018 1:53:56 AM 
com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> SEVERE: 1 thread leaked from SUITE scope at 
org.apache.solr.analytics.OverallAnalyticsTest: 
   [junit4]   2>    1) Thread[id=85, name=qtp557035741-85, state=TIMED_WAITING, 
group=TGRP-OverallAnalyticsTest]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2>         at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
   [junit4]   2>         at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
   [junit4]   2>         at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
   [junit4]   2>         at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
   [junit4]   2>         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> Jan 24, 2018 1:53:56 AM 
com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> INFO: Starting to interrupt leaked threads:
   [junit4]   2>    1) Thread[id=85, name=qtp557035741-85, state=TIMED_WAITING, 
group=TGRP-OverallAnalyticsTest]
   [junit4]   2> Jan 24, 2018 1:53:59 AM 
com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> SEVERE: There are still zombie threads that couldn't be 
terminated:
   [junit4]   2>    1) Thread[id=85, name=qtp557035741-85, state=TIMED_WAITING, 
group=TGRP-OverallAnalyticsTest]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2>         at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
   [junit4]   2>         at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
   [junit4]   2>         at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
   [junit4]   2>         at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
   [junit4]   2>         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> NOTE: test params are: codec=Lucene70, 
sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@e47a78b),
 locale=en-MT, timezone=Asia/Dushanbe
   [junit4]   2> NOTE: Linux 4.4.0-104-generic amd64/Oracle Corporation 
1.8.0_152 (64-bit)/cpus=4,threads=2,free=267843056,total=321388544
   [junit4]   2> NOTE: All tests run in this JVM: [LegacyFieldFacetExtrasTest, 
OverallAnalyticsTest]
   [junit4]   2> NOTE: reproduce with: ant test  
-Dtestcase=OverallAnalyticsTest -Dtests.seed=2D4C3C614787FA5E 
-Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=en-MT 
-Dtests.timezone=Asia/Dushanbe -Dtests.asserts=true 
-Dtests.file.encoding=US-ASCII
   [junit4] ERROR   0.00s J2 | OverallAnalyticsTest (suite) <<<
   [junit4]    > Throwable #1: 
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE 
scope at org.apache.solr.analytics.OverallAnalyticsTest: 
   [junit4]    >    1) Thread[id=85, name=qtp557035741-85, state=TIMED_WAITING, 
group=TGRP-OverallAnalyticsTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
   [junit4]    >         at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
   [junit4]    >         at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
   [junit4]    >         at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
   [junit4]    >         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    >        at 
__randomizedtesting.SeedInfo.seed([2D4C3C614787FA5E]:0)Throwable #2: 
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie 
threads that couldn't be terminated:
   [junit4]    >    1) Thread[id=85, name=qtp557035741-85, state=TIMED_WAITING, 
group=TGRP-OverallAnalyticsTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
   [junit4]    >         at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
   [junit4]    >         at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
   [junit4]    >         at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
   [junit4]    >         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    >        at 
__randomizedtesting.SeedInfo.seed([2D4C3C614787FA5E]:0)
   [junit4] Completed [29/78 (1!)] on J2 in 91.43s, 1 test, 2 errors <<< 
FAILURES!

[...truncated 45043 lines...]
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Reply via email to