Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1469/

4 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest

Error Message:
ObjectTracker found 1 object(s) that were not released!!! [NRTCachingDirectory] 
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: 
org.apache.lucene.store.NRTCachingDirectory  at 
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
  at 
org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)
  at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:351)  at 
org.apache.solr.core.SolrCore.cleanupOldIndexDirectories(SolrCore.java:3031)  
at org.apache.solr.core.SolrCore.close(SolrCore.java:1604)  at 
org.apache.solr.core.SolrCore.<init>(SolrCore.java:1000)  at 
org.apache.solr.core.SolrCore.<init>(SolrCore.java:863)  at 
org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1043)
  at org.apache.solr.core.CoreContainer.lambda$load$13(CoreContainer.java:644)  
at 
com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
  at java.util.concurrent.FutureTask.run(FutureTask.java:266)  at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
  at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 
 at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
 at java.lang.Thread.run(Thread.java:748)  

Stack Trace:
java.lang.AssertionError: ObjectTracker found 1 object(s) that were not 
released!!! [NRTCachingDirectory]
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: 
org.apache.lucene.store.NRTCachingDirectory
        at 
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
        at 
org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)
        at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:351)
        at 
org.apache.solr.core.SolrCore.cleanupOldIndexDirectories(SolrCore.java:3031)
        at org.apache.solr.core.SolrCore.close(SolrCore.java:1604)
        at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1000)
        at org.apache.solr.core.SolrCore.<init>(SolrCore.java:863)
        at 
org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1043)
        at 
org.apache.solr.core.CoreContainer.lambda$load$13(CoreContainer.java:644)
        at 
com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)


        at __randomizedtesting.SeedInfo.seed([77E9F6DCC7BA209B]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.junit.Assert.assertNull(Assert.java:551)
        at 
org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:295)
        at sun.reflect.GeneratedMethodAccessor58.invoke(Unknown Source)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:897)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest.testHistory

Error Message:
expected:<5> but was:<0>

Stack Trace:
java.lang.AssertionError: expected:<5> but was:<0>
        at 
__randomizedtesting.SeedInfo.seed([77E9F6DCC7BA209B:1A1552217DF2DF9C]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.failNotEquals(Assert.java:647)
        at org.junit.Assert.assertEquals(Assert.java:128)
        at org.junit.Assert.assertEquals(Assert.java:472)
        at org.junit.Assert.assertEquals(Assert.java:456)
        at 
org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest.testHistory(AutoscalingHistoryHandlerTest.java:244)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.uninverting.TestDocTermOrds.testTriggerUnInvertLimit

Error Message:
Java heap space

Stack Trace:
java.lang.OutOfMemoryError: Java heap space
        at 
__randomizedtesting.SeedInfo.seed([77E9F6DCC7BA209B:445BDE18CA0DFA2C]:0)
        at org.apache.lucene.store.RAMFile.newBuffer(RAMFile.java:78)
        at org.apache.lucene.store.RAMFile.addBuffer(RAMFile.java:51)
        at 
org.apache.lucene.store.RAMOutputStream.switchCurrentBuffer(RAMOutputStream.java:164)
        at 
org.apache.lucene.store.RAMOutputStream.writeBytes(RAMOutputStream.java:150)
        at 
org.apache.lucene.store.MockIndexOutputWrapper.writeBytes(MockIndexOutputWrapper.java:141)
        at 
org.apache.lucene.codecs.lucene50.Lucene50PostingsWriter.addPosition(Lucene50PostingsWriter.java:320)
        at 
org.apache.lucene.codecs.PushPostingsWriterBase.writeTerm(PushPostingsWriterBase.java:163)
        at 
org.apache.lucene.codecs.blocktree.BlockTreeTermsWriter$TermsWriter.write(BlockTreeTermsWriter.java:865)
        at 
org.apache.lucene.codecs.blocktree.BlockTreeTermsWriter.write(BlockTreeTermsWriter.java:344)
        at 
org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:105)
        at 
org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsWriter.merge(PerFieldPostingsFormat.java:165)
        at 
org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:243)
        at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:138)
        at 
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4441)
        at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4081)
        at 
org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
        at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2245)
        at 
org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:5096)
        at 
org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1730)
        at 
org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1462)
        at 
org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:171)
        at 
org.apache.solr.uninverting.TestDocTermOrds.testTriggerUnInvertLimit(TestDocTermOrds.java:167)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)


FAILED:  org.apache.solr.cloud.AliasIntegrationTest.testModifyMetadataCAR

Error Message:


Stack Trace:
java.lang.AssertionError
        at 
__randomizedtesting.SeedInfo.seed([77E9F6DCC7BA209B:6BBBB7977A6180C3]:0)
        at org.junit.Assert.fail(Assert.java:92)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.junit.Assert.assertTrue(Assert.java:54)
        at 
org.apache.solr.cloud.AliasIntegrationTest.checkFooAndBarMeta(AliasIntegrationTest.java:283)
        at 
org.apache.solr.cloud.AliasIntegrationTest.testModifyMetadataCAR(AliasIntegrationTest.java:262)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 13045 lines...]
   [junit4] Suite: org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest
   [junit4]   2> Creating dataDir: 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_77E9F6DCC7BA209B-001/init-core-data-001
   [junit4]   2> 163429 WARN  
(SUITE-AutoscalingHistoryHandlerTest-seed#[77E9F6DCC7BA209B]-worker) [    ] 
o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=9 numCloses=9
   [junit4]   2> 163429 INFO  
(SUITE-AutoscalingHistoryHandlerTest-seed#[77E9F6DCC7BA209B]-worker) [    ] 
o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) 
w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 163431 INFO  
(SUITE-AutoscalingHistoryHandlerTest-seed#[77E9F6DCC7BA209B]-worker) [    ] 
o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: 
@org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 164241 INFO  
(SUITE-AutoscalingHistoryHandlerTest-seed#[77E9F6DCC7BA209B]-worker) [    ] 
o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: 
test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 164242 INFO  
(SUITE-AutoscalingHistoryHandlerTest-seed#[77E9F6DCC7BA209B]-worker) [    ] 
o.a.s.c.MiniSolrCloudCluster Starting cluster of 3 servers in 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_77E9F6DCC7BA209B-001/tempDir-001
   [junit4]   2> 164242 INFO  
(SUITE-AutoscalingHistoryHandlerTest-seed#[77E9F6DCC7BA209B]-worker) [    ] 
o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 164242 INFO  (Thread-259) [    ] o.a.s.c.ZkTestServer client 
port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 164242 INFO  (Thread-259) [    ] o.a.s.c.ZkTestServer Starting 
server
   [junit4]   2> 164256 ERROR (Thread-259) [    ] o.a.z.s.ZooKeeperServer 
ZKShutdownHandler is not registered, so ZooKeeper server won't take any action 
on ERROR or SHUTDOWN server state changes
   [junit4]   2> 164349 INFO  
(SUITE-AutoscalingHistoryHandlerTest-seed#[77E9F6DCC7BA209B]-worker) [    ] 
o.a.s.c.ZkTestServer start zk server on port:39309
   [junit4]   2> 164426 INFO  (zkConnectionManagerCallback-292-thread-1) [    ] 
o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 164575 INFO  (jetty-launcher-289-thread-3) [    ] 
o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 
2017-11-21T15:27:37-06:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 164588 INFO  (jetty-launcher-289-thread-1) [    ] 
o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 
2017-11-21T15:27:37-06:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 164730 INFO  (jetty-launcher-289-thread-3) [    ] 
o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 164730 INFO  (jetty-launcher-289-thread-3) [    ] 
o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 164730 INFO  (jetty-launcher-289-thread-3) [    ] 
o.e.j.s.session Scavenging every 600000ms
   [junit4]   2> 164730 INFO  (jetty-launcher-289-thread-3) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@7e8838c{/solr,null,AVAILABLE}
   [junit4]   2> 164795 INFO  (jetty-launcher-289-thread-2) [    ] 
o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 
2017-11-21T15:27:37-06:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 164835 INFO  (jetty-launcher-289-thread-1) [    ] 
o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 164835 INFO  (jetty-launcher-289-thread-1) [    ] 
o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 164835 INFO  (jetty-launcher-289-thread-1) [    ] 
o.e.j.s.session Scavenging every 660000ms
   [junit4]   2> 164854 INFO  (jetty-launcher-289-thread-1) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@6b21689c{/solr,null,AVAILABLE}
   [junit4]   2> 164921 INFO  (jetty-launcher-289-thread-2) [    ] 
o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 164921 INFO  (jetty-launcher-289-thread-2) [    ] 
o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 164921 INFO  (jetty-launcher-289-thread-2) [    ] 
o.e.j.s.session Scavenging every 660000ms
   [junit4]   2> 164921 INFO  (jetty-launcher-289-thread-2) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@1128210b{/solr,null,AVAILABLE}
   [junit4]   2> 164927 INFO  (jetty-launcher-289-thread-1) [    ] 
o.e.j.s.AbstractConnector Started ServerConnector@3797b7fa{SSL,[ssl, 
http/1.1]}{127.0.0.1:39631}
   [junit4]   2> 164927 INFO  (jetty-launcher-289-thread-1) [    ] 
o.e.j.s.Server Started @176751ms
   [junit4]   2> 164927 INFO  (jetty-launcher-289-thread-1) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, 
hostPort=39631}
   [junit4]   2> 164928 ERROR (jetty-launcher-289-thread-1) [    ] 
o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 164928 INFO  (jetty-launcher-289-thread-1) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 
8.0.0
   [junit4]   2> 164928 INFO  (jetty-launcher-289-thread-1) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 164928 INFO  (jetty-launcher-289-thread-1) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 164928 INFO  (jetty-launcher-289-thread-1) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2018-02-04T14:34:48.881Z
   [junit4]   2> 164945 INFO  (jetty-launcher-289-thread-2) [    ] 
o.e.j.s.AbstractConnector Started ServerConnector@6db6f06b{SSL,[ssl, 
http/1.1]}{127.0.0.1:40121}
   [junit4]   2> 164945 INFO  (jetty-launcher-289-thread-2) [    ] 
o.e.j.s.Server Started @176768ms
   [junit4]   2> 164945 INFO  (jetty-launcher-289-thread-2) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, 
hostPort=40121}
   [junit4]   2> 164945 ERROR (jetty-launcher-289-thread-2) [    ] 
o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 164945 INFO  (jetty-launcher-289-thread-2) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 
8.0.0
   [junit4]   2> 164945 INFO  (jetty-launcher-289-thread-2) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 164945 INFO  (jetty-launcher-289-thread-2) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 164945 INFO  (jetty-launcher-289-thread-2) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2018-02-04T14:34:48.898Z
   [junit4]   2> 164946 INFO  (jetty-launcher-289-thread-3) [    ] 
o.e.j.s.AbstractConnector Started ServerConnector@72612d6a{SSL,[ssl, 
http/1.1]}{127.0.0.1:36041}
   [junit4]   2> 164946 INFO  (jetty-launcher-289-thread-3) [    ] 
o.e.j.s.Server Started @176769ms
   [junit4]   2> 164946 INFO  (jetty-launcher-289-thread-3) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, 
hostPort=36041}
   [junit4]   2> 164946 ERROR (jetty-launcher-289-thread-3) [    ] 
o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 164946 INFO  (jetty-launcher-289-thread-3) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 
8.0.0
   [junit4]   2> 164946 INFO  (jetty-launcher-289-thread-3) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 164946 INFO  (jetty-launcher-289-thread-3) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 164946 INFO  (jetty-launcher-289-thread-3) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2018-02-04T14:34:48.899Z
   [junit4]   2> 165022 INFO  (zkConnectionManagerCallback-298-thread-1) [    ] 
o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 165042 INFO  (jetty-launcher-289-thread-3) [    ] 
o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 165109 INFO  (zkConnectionManagerCallback-296-thread-1) [    ] 
o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 165150 INFO  (jetty-launcher-289-thread-2) [    ] 
o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 165165 INFO  (zkConnectionManagerCallback-294-thread-1) [    ] 
o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 165274 INFO  (jetty-launcher-289-thread-1) [    ] 
o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 165322 INFO  (jetty-launcher-289-thread-3) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39309/solr
   [junit4]   2> 165440 INFO  (jetty-launcher-289-thread-2) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39309/solr
   [junit4]   2> 165508 INFO  (zkConnectionManagerCallback-302-thread-1) [    ] 
o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 165573 INFO  (jetty-launcher-289-thread-1) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39309/solr
   [junit4]   2> 165649 INFO  (zkConnectionManagerCallback-306-thread-1) [    ] 
o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 165693 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] 
o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 
0x1000580386d0004, likely client has closed socket
   [junit4]   2> 165710 INFO  (zkConnectionManagerCallback-310-thread-1) [    ] 
o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 165789 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] 
o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 
0x1000580386d0006, likely client has closed socket
   [junit4]   2> 165803 INFO  
(zkConnectionManagerCallback-312-thread-1-processing-n:127.0.0.1:36041_solr) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 165803 INFO  
(zkConnectionManagerCallback-314-thread-1-processing-n:127.0.0.1:40121_solr) 
[n:127.0.0.1:40121_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 165897 INFO  
(zkConnectionManagerCallback-316-thread-1-processing-n:127.0.0.1:39631_solr) 
[n:127.0.0.1:39631_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 166151 INFO  (jetty-launcher-289-thread-3) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 166152 INFO  (jetty-launcher-289-thread-3) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.OverseerElectionContext I am going to be 
the leader 127.0.0.1:36041_solr
   [junit4]   2> 166152 INFO  (jetty-launcher-289-thread-3) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.Overseer Overseer 
(id=72063642298548231-127.0.0.1:36041_solr-n_0000000000) starting
   [junit4]   2> 166205 DEBUG 
(OverseerStateUpdate-72063642298548231-127.0.0.1:36041_solr-n_0000000000) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 166222 DEBUG 
(OverseerAutoScalingTriggerThread-72063642298548231-127.0.0.1:36041_solr-n_0000000000)
 [n:127.0.0.1:36041_solr    ] o.a.s.c.a.OverseerTriggerThread Adding 
.autoAddReplicas trigger
   [junit4]   2> 166222 INFO  (jetty-launcher-289-thread-3) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:36041_solr
   [junit4]   2> 166223 DEBUG 
(OverseerAutoScalingTriggerThread-72063642298548231-127.0.0.1:36041_solr-n_0000000000)
 [n:127.0.0.1:36041_solr    ] o.a.s.c.a.OverseerTriggerThread Refreshing 
/autoscaling.json with znode version 1
   [junit4]   2> 166242 DEBUG 
(OverseerAutoScalingTriggerThread-72063642298548231-127.0.0.1:36041_solr-n_0000000000)
 [n:127.0.0.1:36041_solr    ] o.a.s.c.a.NodeLostTrigger Initial livenodes: []
   [junit4]   2> 166242 DEBUG 
(OverseerAutoScalingTriggerThread-72063642298548231-127.0.0.1:36041_solr-n_0000000000)
 [n:127.0.0.1:36041_solr    ] o.a.s.c.a.OverseerTriggerThread Current 
znodeVersion 1, lastZnodeVersion -1
   [junit4]   2> 166242 DEBUG 
(OverseerAutoScalingTriggerThread-72063642298548231-127.0.0.1:36041_solr-n_0000000000)
 [n:127.0.0.1:36041_solr    ] o.a.s.c.a.OverseerTriggerThread Processed trigger 
updates upto znodeVersion 1
   [junit4]   2> 166279 INFO  
(zkCallback-311-thread-1-processing-n:127.0.0.1:36041_solr) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (1)
   [junit4]   2> 166331 DEBUG 
(OverseerAutoScalingTriggerThread-72063642298548231-127.0.0.1:36041_solr-n_0000000000)
 [n:127.0.0.1:36041_solr    ] o.a.s.c.a.OverseerTriggerThread -- clean old 
nodeAdded markers
   [junit4]   2> 166331 DEBUG 
(OverseerAutoScalingTriggerThread-72063642298548231-127.0.0.1:36041_solr-n_0000000000)
 [n:127.0.0.1:36041_solr    ] o.a.s.c.a.OverseerTriggerThread Current 
znodeVersion 1, lastZnodeVersion 1
   [junit4]   2> 166368 DEBUG (ScheduledTrigger-347-thread-1) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: 
.auto_add_replicas with currently live nodes: 1
   [junit4]   2> 166570 INFO  (jetty-launcher-289-thread-3) 
[n:127.0.0.1:36041_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_36041.solr.node' (registry 'solr.node') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@79633248
   [junit4]   2> 166602 INFO  (jetty-launcher-289-thread-3) 
[n:127.0.0.1:36041_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_36041.solr.jvm' (registry 'solr.jvm') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@79633248
   [junit4]   2> 166619 INFO  (jetty-launcher-289-thread-3) 
[n:127.0.0.1:36041_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_36041.solr.jetty' (registry 'solr.jetty') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@79633248
   [junit4]   2> 166620 INFO  (jetty-launcher-289-thread-3) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core 
definitions underneath 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_77E9F6DCC7BA209B-001/tempDir-001/node3/.
   [junit4]   2> 166961 INFO  (jetty-launcher-289-thread-2) 
[n:127.0.0.1:40121_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (1)
   [junit4]   2> 166963 INFO  (jetty-launcher-289-thread-2) 
[n:127.0.0.1:40121_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 166989 INFO  (jetty-launcher-289-thread-2) 
[n:127.0.0.1:40121_solr    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:40121_solr
   [junit4]   2> 166990 DEBUG 
(OverseerStateUpdate-72063642298548231-127.0.0.1:36041_solr-n_0000000000) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.Overseer processMessage: queueSize: 1, 
message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:40121_solr"} current state version: 0
   [junit4]   2> 166990 DEBUG 
(OverseerStateUpdate-72063642298548231-127.0.0.1:36041_solr-n_0000000000) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.o.NodeMutator DownNode state invoked for 
node: 127.0.0.1:40121_solr
   [junit4]   2> 167037 INFO  
(zkCallback-311-thread-1-processing-n:127.0.0.1:36041_solr) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (1) -> (2)
   [junit4]   2> 167038 INFO  
(zkCallback-313-thread-1-processing-n:127.0.0.1:40121_solr) 
[n:127.0.0.1:40121_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (1) -> (2)
   [junit4]   2> 167070 INFO  (jetty-launcher-289-thread-1) 
[n:127.0.0.1:39631_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (2)
   [junit4]   2> 167072 INFO  (jetty-launcher-289-thread-1) 
[n:127.0.0.1:39631_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 167275 INFO  (jetty-launcher-289-thread-1) 
[n:127.0.0.1:39631_solr    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:39631_solr
   [junit4]   2> 167320 DEBUG 
(OverseerStateUpdate-72063642298548231-127.0.0.1:36041_solr-n_0000000000) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.Overseer processMessage: queueSize: 1, 
message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:39631_solr"} current state version: 0
   [junit4]   2> 167320 DEBUG 
(OverseerStateUpdate-72063642298548231-127.0.0.1:36041_solr-n_0000000000) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.o.NodeMutator DownNode state invoked for 
node: 127.0.0.1:39631_solr
   [junit4]   2> 167385 DEBUG (ScheduledTrigger-347-thread-1) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: 
.auto_add_replicas with currently live nodes: 2
   [junit4]   2> 167386 INFO  
(zkCallback-311-thread-1-processing-n:127.0.0.1:36041_solr) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (2) -> (3)
   [junit4]   2> 167519 INFO  
(zkCallback-313-thread-1-processing-n:127.0.0.1:40121_solr) 
[n:127.0.0.1:40121_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (2) -> (3)
   [junit4]   2> 167632 INFO  
(zkCallback-315-thread-1-processing-n:127.0.0.1:39631_solr) 
[n:127.0.0.1:39631_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (2) -> (3)
   [junit4]   2> 167834 INFO  (jetty-launcher-289-thread-2) 
[n:127.0.0.1:40121_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_40121.solr.node' (registry 'solr.node') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@79633248
   [junit4]   2> 167848 INFO  (jetty-launcher-289-thread-1) 
[n:127.0.0.1:39631_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_39631.solr.node' (registry 'solr.node') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@79633248
   [junit4]   2> 167884 INFO  (jetty-launcher-289-thread-1) 
[n:127.0.0.1:39631_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_39631.solr.jvm' (registry 'solr.jvm') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@79633248
   [junit4]   2> 167884 INFO  (jetty-launcher-289-thread-1) 
[n:127.0.0.1:39631_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_39631.solr.jetty' (registry 'solr.jetty') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@79633248
   [junit4]   2> 167885 INFO  (jetty-launcher-289-thread-1) 
[n:127.0.0.1:39631_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core 
definitions underneath 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_77E9F6DCC7BA209B-001/tempDir-001/node1/.
   [junit4]   2> 167942 INFO  (jetty-launcher-289-thread-2) 
[n:127.0.0.1:40121_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_40121.solr.jvm' (registry 'solr.jvm') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@79633248
   [junit4]   2> 167942 INFO  (jetty-launcher-289-thread-2) 
[n:127.0.0.1:40121_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_40121.solr.jetty' (registry 'solr.jetty') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@79633248
   [junit4]   2> 167943 INFO  (jetty-launcher-289-thread-2) 
[n:127.0.0.1:40121_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core 
definitions underneath 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_77E9F6DCC7BA209B-001/tempDir-001/node2/.
   [junit4]   2> 168329 INFO  (zkConnectionManagerCallback-324-thread-1) [    ] 
o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 168439 DEBUG (ScheduledTrigger-347-thread-2) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: 
.auto_add_replicas with currently live nodes: 3
   [junit4]   2> 168549 INFO  (zkConnectionManagerCallback-328-thread-1) [    ] 
o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 168598 INFO  
(SUITE-AutoscalingHistoryHandlerTest-seed#[77E9F6DCC7BA209B]-worker) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 168601 INFO  
(SUITE-AutoscalingHistoryHandlerTest-seed#[77E9F6DCC7BA209B]-worker) [    ] 
o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:39309/solr ready
   [junit4]   2> 169305 INFO  (qtp1933083322-1089) [n:127.0.0.1:40121_solr    ] 
o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params 
replicationFactor=3&name=AutoscalingHistoryHandlerTest_collection&nrtReplicas=3&action=CREATE&numShards=1&wt=javabin&version=2
 and sendToOCPQueue=true
   [junit4]   2> 169320 INFO  
(OverseerThreadFactory-349-thread-1-processing-n:127.0.0.1:36041_solr) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.a.c.CreateCollectionCmd Create collection 
AutoscalingHistoryHandlerTest_collection
   [junit4]   2> 169320 INFO  
(OverseerThreadFactory-349-thread-1-processing-n:127.0.0.1:36041_solr) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node 
/configs/_default/protwords.txt to 
/configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/protwords.txt
   [junit4]   2> 169322 INFO  
(OverseerThreadFactory-349-thread-1-processing-n:127.0.0.1:36041_solr) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node 
/configs/_default/managed-schema to 
/configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/managed-schema
   [junit4]   2> 169323 INFO  
(OverseerThreadFactory-349-thread-1-processing-n:127.0.0.1:36041_solr) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node 
/configs/_default/solrconfig.xml to 
/configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/solrconfig.xml
   [junit4]   2> 169360 INFO  
(OverseerThreadFactory-349-thread-1-processing-n:127.0.0.1:36041_solr) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node 
/configs/_default/synonyms.txt to 
/configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/synonyms.txt
   [junit4]   2> 169361 INFO  
(OverseerThreadFactory-349-thread-1-processing-n:127.0.0.1:36041_solr) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node 
/configs/_default/stopwords.txt to 
/configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/stopwords.txt
   [junit4]   2> 169362 INFO  
(OverseerThreadFactory-349-thread-1-processing-n:127.0.0.1:36041_solr) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node 
/configs/_default/params.json to 
/configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/params.json
   [junit4]   2> 169377 INFO  
(OverseerThreadFactory-349-thread-1-processing-n:127.0.0.1:36041_solr) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node 
/configs/_default/lang/contractions_it.txt to 
/configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/contractions_it.txt
   [junit4]   2> 169440 DEBUG (ScheduledTrigger-347-thread-1) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: 
.auto_add_replicas with currently live nodes: 3
   [junit4]   2> 169441 INFO  
(OverseerThreadFactory-349-thread-1-processing-n:127.0.0.1:36041_solr) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node 
/configs/_default/lang/stopwords_no.txt to 
/configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_no.txt
   [junit4]   2> 169482 INFO  
(OverseerThreadFactory-349-thread-1-processing-n:127.0.0.1:36041_solr) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node 
/configs/_default/lang/stopwords_hy.txt to 
/configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_hy.txt
   [junit4]   2> 169483 INFO  
(OverseerThreadFactory-349-thread-1-processing-n:127.0.0.1:36041_solr) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node 
/configs/_default/lang/stemdict_nl.txt to 
/configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stemdict_nl.txt
   [junit4]   2> 169484 INFO  
(OverseerThreadFactory-349-thread-1-processing-n:127.0.0.1:36041_solr) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node 
/configs/_default/lang/contractions_ca.txt to 
/configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/contractions_ca.txt
   [junit4]   2> 169494 INFO  
(OverseerThreadFactory-349-thread-1-processing-n:127.0.0.1:36041_solr) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node 
/configs/_default/lang/stopwords_id.txt to 
/configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_id.txt
   [junit4]   2> 169531 INFO  
(OverseerThreadFactory-349-thread-1-processing-n:127.0.0.1:36041_solr) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node 
/configs/_default/lang/stopwords_ga.txt to 
/configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ga.txt
   [junit4]   2> 169534 INFO  
(OverseerThreadFactory-349-thread-1-processing-n:127.0.0.1:36041_solr) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node 
/configs/_default/lang/stopwords_da.txt to 
/configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_da.txt
   [junit4]   2> 169567 INFO  
(OverseerThreadFactory-349-thread-1-processing-n:127.0.0.1:36041_solr) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node 
/configs/_default/lang/stopwords_hi.txt to 
/configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_hi.txt
   [junit4]   2> 169570 INFO  
(OverseerThreadFactory-349-thread-1-processing-n:127.0.0.1:36041_solr) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node 
/configs/_default/lang/stopwords_ja.txt to 
/configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ja.txt
   [junit4]   2> 169588 INFO  
(OverseerThreadFactory-349-thread-1-processing-n:127.0.0.1:36041_solr) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node 
/configs/_default/lang/stopwords_pt.txt to 
/configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_pt.txt
   [junit4]   2> 169590 INFO  
(OverseerThreadFactory-349-thread-1-processing-n:127.0.0.1:36041_solr) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node 
/configs/_default/lang/stopwords_hu.txt to 
/configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_hu.txt
   [junit4]   2> 169702 INFO  
(OverseerThreadFactory-349-thread-1-processing-n:127.0.0.1:36041_solr) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node 
/configs/_default/lang/stopwords_el.txt to 
/configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_el.txt
   [junit4]   2> 169703 INFO  
(OverseerThreadFactory-349-thread-1-processing-n:127.0.0.1:36041_solr) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node 
/configs/_default/lang/stopwords_ru.txt to 
/configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ru.txt
   [junit4]   2> 169730 INFO  
(OverseerThreadFactory-349-thread-1-processing-n:127.0.0.1:36041_solr) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node 
/configs/_default/lang/stopwords_tr.txt to 
/configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_tr.txt
   [junit4]   2> 169732 INFO  
(OverseerThreadFactory-349-thread-1-processing-n:127.0.0.1:36041_solr) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node 
/configs/_default/lang/stopwords_ar.txt to 
/configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ar.txt
   [junit4]   2> 169798 INFO  
(OverseerThreadFactory-349-thread-1-processing-n:127.0.0.1:36041_solr) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node 
/configs/_default/lang/userdict_ja.txt to 
/configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/userdict_ja.txt
   [junit4]   2> 169799 INFO  
(OverseerThreadFactory-349-thread-1-processing-n:127.0.0.1:36041_solr) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node 
/configs/_default/lang/stopwords_it.txt to 
/configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_it.txt
   [junit4]   2> 169801 INFO  
(OverseerThreadFactory-349-thread-1-processing-n:127.0.0.1:36041_solr) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node 
/configs/_default/lang/stopwords_eu.txt to 
/configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_eu.txt
   [junit4]   2> 169907 INFO  
(OverseerThreadFactory-349-thread-1-processing-n:127.0.0.1:36041_solr) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node 
/configs/_default/lang/stopwords_cz.txt to 
/configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_cz.txt
   [junit4]   2> 169950 INFO  
(OverseerThreadFactory-349-thread-1-processing-n:127.0.0.1:36041_solr) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node 
/configs/_default/lang/stoptags_ja.txt to 
/configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stoptags_ja.txt
   [junit4]   2> 169951 INFO  
(OverseerThreadFactory-349-thread-1-processing-n:127.0.0.1:36041_solr) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node 
/configs/_default/lang/contractions_fr.txt to 
/configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/contractions_fr.txt
   [junit4]   2> 169952 INFO  
(OverseerThreadFactory-349-thread-1-processing-n:127.0.0.1:36041_solr) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node 
/configs/_default/lang/stopwords_fa.txt to 
/configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_fa.txt
   [junit4]   2> 169953 INFO  
(OverseerThreadFactory-349-thread-1-processing-n:127.0.0.1:36041_solr) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node 
/configs/_default/lang/stopwords_de.txt to 
/configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_de.txt
   [junit4]   2> 169954 INFO  
(OverseerThreadFactory-349-thread-1-processing-n:127.0.0.1:36041_solr) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node 
/configs/_default/lang/contractions_ga.txt to 
/configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/contractions_ga.txt
   [junit4]   2> 169955 INFO  
(OverseerThreadFactory-349-thread-1-processing-n:127.0.0.1:36041_solr) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node 
/configs/_default/lang/stopwords_ca.txt to 
/configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ca.txt
   [junit4]   2> 169968 INFO  
(OverseerThreadFactory-349-thread-1-processing-n:127.0.0.1:36041_solr) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node 
/configs/_default/lang/stopwords_nl.txt to 
/configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_nl.txt
   [junit4]   2> 169969 INFO  
(OverseerThreadFactory-349-thread-1-processing-n:127.0.0.1:36041_solr) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node 
/configs/_default/lang/stopwords_bg.txt to 
/configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_bg.txt
   [junit4]   2> 169970 INFO  
(OverseerThreadFactory-349-thread-1-processing-n:127.0.0.1:36041_solr) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node 
/configs/_default/lang/stopwords_es.txt to 
/configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_es.txt
   [junit4]   2> 169984 INFO  
(OverseerThreadFactory-349-thread-1-processing-n:127.0.0.1:36041_solr) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node 
/configs/_default/lang/stopwords_en.txt to 
/configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_en.txt
   [junit4]   2> 169986 INFO  
(OverseerThreadFactory-349-thread-1-processing-n:127.0.0.1:36041_solr) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node 
/configs/_default/lang/stopwords_lv.txt to 
/configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_lv.txt
   [junit4]   2> 169987 INFO  
(OverseerThreadFactory-349-thread-1-processing-n:127.0.0.1:36041_solr) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node 
/configs/_default/lang/stopwords_th.txt to 
/configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_th.txt
   [junit4]   2> 169988 INFO  
(OverseerThreadFactory-349-thread-1-processing-n:127.0.0.1:36041_solr) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node 
/configs/_default/lang/stopwords_ro.txt to 
/configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ro.txt
   [junit4]   2> 170006 INFO  
(OverseerThreadFactory-349-thread-1-processing-n:127.0.0.1:36041_solr) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node 
/configs/_default/lang/stopwords_fr.txt to 
/configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_fr.txt
   [junit4]   2> 170007 INFO  
(OverseerThreadFactory-349-thread-1-processing-n:127.0.0.1:36041_solr) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node 
/configs/_default/lang/stopwords_sv.txt to 
/configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_sv.txt
   [junit4]   2> 170020 INFO  
(OverseerThreadFactory-349-thread-1-processing-n:127.0.0.1:36041_solr) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node 
/configs/_default/lang/stopwords_fi.txt to 
/configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_fi.txt
   [junit4]   2> 170021 INFO  
(OverseerThreadFactory-349-thread-1-processing-n:127.0.0.1:36041_solr) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node 
/configs/_default/lang/hyphenations_ga.txt to 
/configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/hyphenations_ga.txt
   [junit4]   2> 170023 INFO  
(OverseerThreadFactory-349-thread-1-processing-n:127.0.0.1:36041_solr) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node 
/configs/_default/lang/stopwords_gl.txt to 
/configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_gl.txt
   [junit4]   2> 170044 DEBUG 
(OverseerStateUpdate-72063642298548231-127.0.0.1:36041_solr-n_0000000000) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.Overseer processMessage: queueSize: 1, 
message = {
   [junit4]   2>   "name":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "fromApi":"true",
   [junit4]   2>   "replicationFactor":"3",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "nrtReplicas":"3",
   [junit4]   2>   "stateFormat":"2",
   [junit4]   2>   "operation":"create"} current state version: 0
   [junit4]   2> 170044 DEBUG 
(OverseerStateUpdate-72063642298548231-127.0.0.1:36041_solr-n_0000000000) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.o.ClusterStateMutator building a new 
cName: AutoscalingHistoryHandlerTest_collection
   [junit4]   2> 170044 DEBUG 
(OverseerStateUpdate-72063642298548231-127.0.0.1:36041_solr-n_0000000000) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.o.ZkStateWriter going to create_collection 
/collections/AutoscalingHistoryHandlerTest_collection/state.json
   [junit4]   2> 170177 DEBUG 
(OverseerStateUpdate-72063642298548231-127.0.0.1:36041_solr-n_0000000000) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.Overseer processMessage: queueSize: 1, 
message = {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   
"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:36041/solr";,
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} current state version: 0
   [junit4]   2> 170177 INFO  
(OverseerStateUpdate-72063642298548231-127.0.0.1:36041_solr-n_0000000000) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   
"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:36041/solr";,
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 170184 DEBUG 
(OverseerStateUpdate-72063642298548231-127.0.0.1:36041_solr-n_0000000000) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.o.SliceMutator Old Slice: shard1:{
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "replicas":{}}
   [junit4]   2> 170184 DEBUG 
(OverseerStateUpdate-72063642298548231-127.0.0.1:36041_solr-n_0000000000) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.o.SliceMutator New Slice: shard1:{
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "replicas":{"core_node3":{
   [junit4]   2>       
"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>       "base_url":"https://127.0.0.1:36041/solr";,
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"}}}
   [junit4]   2> 170203 DEBUG 
(OverseerStateUpdate-72063642298548231-127.0.0.1:36041_solr-n_0000000000) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.Overseer processMessage: queueSize: 2, 
message = {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   
"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:40121/solr";,
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} current state version: 0
   [junit4]   2> 170203 INFO  
(OverseerStateUpdate-72063642298548231-127.0.0.1:36041_solr-n_0000000000) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   
"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:40121/solr";,
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 170204 DEBUG 
(OverseerStateUpdate-72063642298548231-127.0.0.1:36041_solr-n_0000000000) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.o.SliceMutator Old Slice: shard1:{
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "replicas":{"core_node3":{
   [junit4]   2>       
"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>       "base_url":"https://127.0.0.1:36041/solr";,
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"}}}
   [junit4]   2> 170204 DEBUG 
(OverseerStateUpdate-72063642298548231-127.0.0.1:36041_solr-n_0000000000) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.o.SliceMutator New Slice: shard1:{
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       
"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>       "base_url":"https://127.0.0.1:36041/solr";,
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"},
   [junit4]   2>     "core_node5":{
   [junit4]   2>       
"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
   [junit4]   2>       "base_url":"https://127.0.0.1:40121/solr";,
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"}}}
   [junit4]   2> 170205 DEBUG 
(OverseerStateUpdate-72063642298548231-127.0.0.1:36041_solr-n_0000000000) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.Overseer processMessage: queueSize: 3, 
message = {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   
"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:39631/solr";,
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} current state version: 0
   [junit4]   2> 170205 INFO  
(OverseerStateUpdate-72063642298548231-127.0.0.1:36041_solr-n_0000000000) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   
"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:39631/solr";,
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 170205 DEBUG 
(OverseerStateUpdate-72063642298548231-127.0.0.1:36041_solr-n_0000000000) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.o.SliceMutator Old Slice: shard1:{
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       
"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>       "base_url":"https://127.0.0.1:36041/solr";,
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"},
   [junit4]   2>     "core_node5":{
   [junit4]   2>       
"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
   [junit4]   2>       "base_url":"https://127.0.0.1:40121/solr";,
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"}}}
   [junit4]   2> 170205 DEBUG 
(OverseerStateUpdate-72063642298548231-127.0.0.1:36041_solr-n_0000000000) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.o.SliceMutator New Slice: shard1:{
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       
"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>       "base_url":"https://127.0.0.1:36041/solr";,
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"},
   [junit4]   2>     "core_node5":{
   [junit4]   2>       
"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
   [junit4]   2>       "base_url":"https://127.0.0.1:40121/solr";,
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"},
   [junit4]   2>     "core_node6":{
   [junit4]   2>       
"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
   [junit4]   2>       "base_url":"https://127.0.0.1:39631/solr";,
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"}}}
   [junit4]   2> 170312 DEBUG 
(OverseerStateUpdate-72063642298548231-127.0.0.1:36041_solr-n_0000000000) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.o.ZkStateWriter going to update_collection 
/collections/AutoscalingHistoryHandlerTest_collection/state.json version: 0
   [junit4]   2> 170440 DEBUG (ScheduledTrigger-347-thread-1) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: 
.auto_add_replicas with currently live nodes: 3
   [junit4]   2> 170872 INFO  (qtp1933083322-1092) [n:127.0.0.1:40121_solr    ] 
o.a.s.h.a.CoreAdminOperation core create command 
qt=/admin/cores&coreNodeName=core_node5&collection.configName=AutoscalingHistoryHandlerTest_collection.AUTOCREATED&newCollection=true&name=AutoscalingHistoryHandlerTest_collection_shard1_replica_n2&action=CREATE&numShards=1&collection=AutoscalingHistoryHandlerTest_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 170872 INFO  (qtp1933083322-1092) [n:127.0.0.1:40121_solr    ] 
o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 
transient cores
   [junit4]   2> 170874 INFO  (qtp1540563842-1082) [n:127.0.0.1:39631_solr    ] 
o.a.s.h.a.CoreAdminOperation core create command 
qt=/admin/cores&coreNodeName=core_node6&collection.configName=AutoscalingHistoryHandlerTest_collection.AUTOCREATED&newCollection=true&name=AutoscalingHistoryHandlerTest_collection_shard1_replica_n4&action=CREATE&numShards=1&collection=AutoscalingHistoryHandlerTest_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 170875 INFO  (qtp1540563842-1082) [n:127.0.0.1:39631_solr    ] 
o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 
transient cores
   [junit4]   2> 170888 INFO  (qtp1876312837-1071) [n:127.0.0.1:36041_solr    ] 
o.a.s.h.a.CoreAdminOperation core create command 
qt=/admin/cores&coreNodeName=core_node3&collection.configName=AutoscalingHistoryHandlerTest_collection.AUTOCREATED&newCollection=true&name=AutoscalingHistoryHandlerTest_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=AutoscalingHistoryHandlerTest_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 170888 INFO  (qtp1876312837-1071) [n:127.0.0.1:36041_solr    ] 
o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 
transient cores
   [junit4]   2> 170909 DEBUG 
(OverseerStateUpdate-72063642298548231-127.0.0.1:36041_solr-n_0000000000) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.Overseer processMessage: queueSize: 3, 
message = {
   [junit4]   2>   
"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
   [junit4]   2>   "core_node_name":"core_node6",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:39631/solr";,
   [junit4]   2>   "node_name":"127.0.0.1:39631_solr",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 170909 DEBUG 
(OverseerStateUpdate-72063642298548231-127.0.0.1:36041_solr-n_0000000000) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 
message={
   [junit4]   2>   
"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
   [junit4]   2>   "core_node_name":"core_node6",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:39631/solr";,
   [junit4]   2>   "node_name":"127.0.0.1:39631_solr",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 170909 DEBUG 
(OverseerStateUpdate-72063642298548231-127.0.0.1:36041_solr-n_0000000000) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.o.ReplicaMutator Will update state for 
replica: 
core_node6:{"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4","base_url":"https://127.0.0.1:39631/solr","node_name":"127.0.0.1:39631_solr","state":"down","type":"NRT"}
   [junit4]   2> 170910 DEBUG 
(OverseerStateUpdate-72063642298548231-127.0.0.1:36041_solr-n_0000000000) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.o.ReplicaMutator Collection is now: 
DocCollection(AutoscalingHistoryHandlerTest_collection//collections/AutoscalingHistoryHandlerTest_collection/state.json/1)={
   [junit4]   2>   "pullReplicas":"0",
   [junit4]   2>   "replicationFactor":"3",
   [junit4]   2>   "router":{"name":"compositeId"},
   [junit4]   2>   "maxShardsPerNode":"1",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"3",
   [junit4]   2>   "tlogReplicas":"0",
   [junit4]   2>   "shards":{"shard1":{
   [junit4]   2>       "range":"80000000-7fffffff",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node3":{
   [junit4]   2>           
"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>           "base_url":"https://127.0.0.1:36041/solr";,
   [junit4]   2>           "state":"down",
   [junit4]   2>           "node_name":null,
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node5":{
   [junit4]   2>           
"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
   [junit4]   2>           "base_url":"https://127.0.0.1:40121/solr";,
   [junit4]   2>           "state":"down",
   [junit4]   2>           "node_name":null,
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node6":{
   [junit4]   2>           
"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
   [junit4]   2>           "base_url":"https://127.0.0.1:39631/solr";,
   [junit4]   2>           "node_name":"127.0.0.1:39631_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"}}}}}
   [junit4]   2> 170910 DEBUG 
(OverseerStateUpdate-72063642298548231-127.0.0.1:36041_solr-n_0000000000) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.Overseer processMessage: queueSize: 3, 
message = {
   [junit4]   2>   
"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
   [junit4]   2>   "core_node_name":"core_node5",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:40121/solr";,
   [junit4]   2>   "node_name":"127.0.0.1:40121_solr",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 170910 DEBUG 
(OverseerStateUpdate-72063642298548231-127.0.0.1:36041_solr-n_0000000000) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 
message={
   [junit4]   2>   
"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
   [junit4]   2>   "core_node_name":"core_node5",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:40121/solr";,
   [junit4]   2>   "node_name":"127.0.0.1:40121_solr",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 170910 DEBUG 
(OverseerStateUpdate-72063642298548231-127.0.0.1:36041_solr-n_0000000000) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.o.ReplicaMutator Will update state for 
replica: 
core_node5:{"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2","base_url":"https://127.0.0.1:40121/solr","node_name":"127.0.0.1:40121_solr","state":"down","type":"NRT"}
   [junit4]   2> 170910 DEBUG 
(OverseerStateUpdate-72063642298548231-127.0.0.1:36041_solr-n_0000000000) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.o.ReplicaMutator Collection is now: 
DocCollection(AutoscalingHistoryHandlerTest_collection//collections/AutoscalingHistoryHandlerTest_collection/state.json/1)={
   [junit4]   2>   "pullReplicas":"0",
   [junit4]   2>   "replicationFactor":"3",
   [junit4]   2>   "router":{"name":"compositeId"},
   [junit4]   2>   "maxShardsPerNode":"1",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"3",
   [junit4]   2>   "tlogReplicas":"0",
   [junit4]   2>   "shards":{"shard1":{
   [junit4]   2>       "range":"80000000-7fffffff",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node3":{
   [junit4]   2>           
"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>           "base_url":"https://127.0.0.1:36041/solr";,
   [junit4]   2>           "state":"down",
   [junit4]   2>           "node_name":null,
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node5":{
   [junit4]   2>           
"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
   [junit4]   2>           "base_url":"https://127.0.0.1:40121/solr";,
   [junit4]   2>           "node_name":"127.0.0.1:40121_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node6":{
   [junit4]   2>           
"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
   [junit4]   2>           "base_url":"https://127.0.0.1:39631/solr";,
   [junit4]   2>           "node_name":"127.0.0.1:39631_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"}}}}}
   [junit4]   2> 170910 DEBUG 
(OverseerStateUpdate-72063642298548231-127.0.0.1:36041_solr-n_0000000000) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.Overseer processMessage: queueSize: 3, 
message = {
   [junit4]   2>   
"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>   "core_node_name":"core_node3",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:36041/solr";,
   [junit4]   2>   "node_name":"127.0.0.1:36041_solr",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 170910 DEBUG 
(OverseerStateUpdate-72063642298548231-127.0.0.1:36041_solr-n_0000000000) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 
message={
   [junit4]   2>   
"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>   "core_node_name":"core_node3",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:36041/solr";,
   [junit4]   2>   "node_name":"127.0.0.1:36041_solr",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 170910 DEBUG 
(OverseerStateUpdate-72063642298548231-127.0.0.1:36041_solr-n_0000000000) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.o.ReplicaMutator Will update state for 
replica: 
core_node3:{"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1","base_url":"https://127.0.0.1:36041/solr","node_name":"127.0.0.1:36041_solr","state":"down","type":"NRT"}
   [junit4]   2> 170910 DEBUG 
(OverseerStateUpdate-72063642298548231-127.0.0.1:36041_solr-n_0000000000) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.o.ReplicaMutator Collection is now: 
DocCollection(AutoscalingHistoryHandlerTest_collection//collections/AutoscalingHistoryHandlerTest_collection/state.json/1)={
   [junit4]   2>   "pullReplicas":"0",
   [junit4]   2>   "replicationFactor":"3",
   [junit4]   2>   "router":{"name":"compositeId"},
   [junit4]   2>   "maxShardsPerNode":"1",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"3",
   [junit4]   2>   "tlogReplicas":"0",
   [junit4]   2>   "shards":{"shard1":{
   [junit4]   2>       "range":"80000000-7fffffff",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node3":{
   [junit4]   2>           
"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>           "base_url":"https://127.0.0.1:36041/solr";,
   [junit4]   2>           "node_name":"127.0.0.1:36041_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node5":{
   [junit4]   2>           
"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
   [junit4]   2>           "base_url":"https://127.0.0.1:40121/solr";,
   [junit4]   2>           "node_name":"127.0.0.1:40121_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node6":{
   [junit4]   2>           
"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
   [junit4]   2>           "base_url":"https://127.0.0.1:39631/solr";,
   [junit4]   2>           "node_name":"127.0.0.1:39631_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"}}}}}
   [junit4]   2> 171035 DEBUG 
(OverseerStateUpdate-72063642298548231-127.0.0.1:36041_solr-n_0000000000) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.o.ZkStateWriter going to update_collection 
/collections/AutoscalingHistoryHandlerTest_collection/state.json version: 1
   [junit4]   2> 171035 INFO  
(zkCallback-311-thread-1-processing-n:127.0.0.1:36041_solr) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for 
collection [AutoscalingHistoryHandlerTest_collection] has occurred - 
updating... (live nodes size: [3])
   [junit4]   2> 171062 INFO  
(zkCallback-313-thread-1-processing-n:127.0.0.1:40121_solr) 
[n:127.0.0.1:40121_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for 
collection [AutoscalingHistoryHandlerTest_collection] has occurred - 
updating... (live nodes size: [3])
   [junit4]   2> 171062 INFO  
(zkCallback-315-thread-1-processing-n:127.0.0.1:39631_solr) 
[n:127.0.0.1:39631_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for 
collection [AutoscalingHistoryHandlerTest_collection] has occurred - 
updating... (live nodes size: [3])
   [junit4]   2> 171091 INFO  
(zkCallback-311-thread-2-processing-n:127.0.0.1:36041_solr) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for 
collection [AutoscalingHistoryHandlerTest_collection] has occurred - 
updating... (live nodes size: [3])
   [junit4]   2> 171091 INFO  
(zkCallback-315-thread-2-processing-n:127.0.0.1:39631_solr) 
[n:127.0.0.1:39631_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for 
collection [AutoscalingHistoryHandlerTest_collection] has occurred - 
updating... (live nodes size: [3])
   [junit4]   2> 171107 INFO  
(zkCallback-313-thread-2-processing-n:127.0.0.1:40121_solr) 
[n:127.0.0.1:40121_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for 
collection [AutoscalingHistoryHandlerTest_collection] has occurred - 
updating... (live nodes size: [3])
   [junit4]   2> 171455 DEBUG (ScheduledTrigger-347-thread-1) 
[n:127.0.0.1:36041_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: 
.auto_add_replicas with currently live nodes: 3
   [junit4]   2> 172052 INFO  (qtp1933083322-1092) [n:127.0.0.1:40121_solr 
c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 
x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] 
o.a.s.c.RequestParams conf resource params.json loaded . version : 0 
   [junit4]   2> 172087 INFO  (qtp1933083322-1092) [n:127.0.0.1:40121_solr 
c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 
x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] 
o.a.s.c.RequestParams request params refreshed to version 0
   [junit4]   2> 172088 WARN  (qtp1933083322-1092) [n:127.0.0.1:40121_solr 
c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 
x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] 
o.a.s.c.SolrConfig Couldn't add files from 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_77E9F6DCC7BA209B-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/extraction/lib
 filtered by .*\.jar to classpath: 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_77E9F6DCC7BA209B-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/extraction/lib
   [junit4]   2> 172144 WARN  (qtp1933083322-1092) [n:127.0.0.1:40121_solr 
c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 
x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] 
o.a.s.c.SolrConfig Couldn't add files from 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_77E9F6DCC7BA209B-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist
 filtered by solr-cell-\d.*\.jar to classpath: 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_77E9F6DCC7BA209B-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist
   [junit4]   2> 172144 WARN  (qtp1933083322-1092) [n:127.0.0.1:40121_solr 
c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 
x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] 
o.a.s.c.SolrConfig Couldn't add files from 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_77E9F6DCC7BA209B-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/clustering/lib
 filtered by .*\.jar to classpath: 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_77E9F6DCC7BA209B-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/clustering/lib
   [junit4]   2> 172145 WARN  (qtp1933083322-1092) [n:127.0.0.1:40121_solr 
c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 
x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] 
o.a.s.c.SolrConfig Couldn't add files from 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_77E9F6DCC7BA209B-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist
 filtered by solr-clustering-\d.*\.jar to classpath: 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_77E9F6DCC7BA209B-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist
   [junit4]   2> 172145 WARN  (qtp1933083322-1092) [n:127.0.0.1:40121_solr 
c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 
x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] 
o.a.s.c.SolrConfig Couldn't add files from 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_77E9F6DCC7BA209B-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/langid/lib
 filtered by .*\.jar to classpath: 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_77E9F6DCC7BA209B-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/langid/lib
   [junit4]   2> 172145 WARN  (qtp1933083322-1092) [n:127.0.0.1:40121_solr 
c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 
x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] 
o.a.s.c.SolrConfig Couldn't add files from 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_77E9F6DCC7BA209B-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist
 filtered by solr-langid-\d.*\.jar to classpath: 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_77E9F6DCC7BA209B-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist
   [junit4]   2> 172145 WARN  (qtp1933083322-1092) [n:127.0.0.1:40121_solr 
c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 
x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] 
o.a.s.c.SolrConfig Couldn't add files from 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_77E9F6DCC7BA209B-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/velocity/lib
 filtered by .*\.jar to classpath: 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_77E9F6DCC7BA209B-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/velocity/lib
   [junit4]   2> 172145 WARN  (qtp1933083322-1092) [n:127.0.0.1:40121_solr 
c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 
x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] 
o.a.s.c.SolrConfig Couldn't add files from 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_77E9F6DCC7BA209B-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist
 filtered by solr-velocity-\d.*\.jar to classpath: 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_77E9F6DCC7BA209B-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist
   [junit4]   2> 172158 INFO  (qtp1876312837-1071) [n:127.0.0.1:36041_solr 
c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 
x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] 
o.a.s.c.RequestParams conf resource params.json loaded . version : 0 
   [junit4]   2> 172158 INFO  (qtp1876312837-1071) [n:127.0.0.1:36041_solr 
c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 
x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] 
o.a.s.c.RequestParams request params refreshed to version 0
   [junit4]   2> 172159 WARN  (qtp1876312837-1071) [n:127.0.0.1:36041_solr 
c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 
x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] 
o.a.s.c.SolrConfig Couldn't add files from 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_77E9F6DCC7BA209B-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/extraction/lib
 filtered by .*\.jar to classpath: 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_77E9F6DCC7BA209B-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/extraction/lib
   [junit4]   2> 172159 WARN  (qtp1876312837-1071) [n:127.0.0.1:36041_solr 
c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 
x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] 
o.a.s.c.SolrConfig Couldn't add files from 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_77E9F6DCC7BA209B-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist
 filtered by solr-cell-\d.*\.jar to classpath: 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_77E9F6DCC7BA209B-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist
   [junit4]   2> 172159 WARN  (qtp1876312837-1071) [n:127.0.0.1:36041_solr 
c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 
x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] 
o.a.s.c.SolrConfig Couldn't add files from 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_77E9F6DCC7BA209B-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/clustering/lib
 filtered by .*\.jar to classpath: 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_77E9F6DCC7BA209B-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/clustering/lib
   [junit4]   2> 172159 WARN  (qtp1876312837-1071) [n:127.0.0.1:36041_solr 
c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 
x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] 
o.a.s.c.SolrConfig Couldn't add files from 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_77E9F6DCC7BA209B-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist
 filtered by solr-clustering-\d.*\.jar to classpath: 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_77E9F6DCC7BA209B-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist
   [junit4]   2> 172160 INFO  (qtp1540563842-1082) [n:127.0.0.1:39631_solr 
c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 
x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] 
o.a.s.c.RequestParams conf resource params.json loaded . version : 0 
   [junit4]   2> 172192 INFO  (qtp1540563842-1082) [n:127.0.0.1:39631_solr 
c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 
x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] 
o.a.s.c.RequestParams request params refreshed to version 0
   [junit4]   2> 172193 WARN  (qtp1540563842-1082) [n:127.0.0.1:39631_solr 
c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 
x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] 
o.a.s.c.SolrConfig Couldn't add files from 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_77E9F6DCC7BA209B-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../contrib/extraction/lib
 filtered by .*\.jar to classpath: 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_77E9F6DCC7BA209B-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../contrib/extraction/lib
   [junit4]   2> 172193 WARN  (qtp1540563842-1082) [n:127.0.0.1:39631_solr 
c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 
x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] 
o.a.s.c.SolrConfig Couldn't add files from 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-

[...truncated too long message...]

zone=Iceland
   [junit4]   2> NOTE: Linux 4.4.0-112-generic amd64/Oracle Corporation 
1.8.0_152 (64-bit)/cpus=4,threads=2,free=276749968,total=529006592
   [junit4]   2> NOTE: All tests run in this JVM: [DistanceUnitsTest, 
ZkSolrClientTest, TestSolrConfigHandler, TestUnifiedSolrHighlighter, 
OverseerStatusTest, DocumentAnalysisRequestHandlerTest, TestCollationField, 
TestFieldTypeCollectionResource, ExitableDirectoryReaderTest, 
TestCoreAdminApis, TestLeaderElectionWithEmptyReplica, SolrIndexConfigTest, 
BasicDistributedZkTest, TestDistribDocBasedVersion, 
TestPHPSerializedResponseWriter, TestMissingGroups, 
TestLMDirichletSimilarityFactory, SolrRequestParserTest, 
TestIndexingPerformance, TestSimDistributedQueue, V2ApiIntegrationTest, 
SystemLogListenerTest, TestSubQueryTransformer, TestQuerySenderNoQuery, 
TestAddFieldRealTimeGet, TestSolrQueryParser, SolrGraphiteReporterTest, 
OverseerCollectionConfigSetProcessorTest, IndexBasedSpellCheckerTest, 
DefaultValueUpdateProcessorTest, ReplicaListTransformerTest, 
TestExtendedDismaxParser, TestDistributedSearch, 
TestEmbeddedSolrServerSchemaAPI, HdfsLockFactoryTest, 
SpellingQueryConverterTest, AddBlockUpdateTest, SpellCheckCollatorTest, 
ClassificationUpdateProcessorTest, TestExpandComponent, TestStandardQParsers, 
OutputWriterTest, TestUtilizeNode, VersionInfoTest, TestBinaryResponseWriter, 
ZkCLITest, HealthCheckHandlerTest, ChaosMonkeyNothingIsSafeTest, 
DirectUpdateHandlerTest, TestScoreJoinQPNoScore, 
DistributedSuggestComponentTest, ConfigureRecoveryStrategyTest, 
TestSolrCoreSnapshots, TestConfigSetProperties, TestHdfsCloudBackupRestore, 
TestPrepRecovery, TestDynamicFieldResource, TestClusterStateMutator, 
TestNumericTerms32, NodeAddedTriggerTest, TestNodeAddedTrigger, 
RAMDirectoryFactoryTest, TestManagedSchemaThreadSafety, 
TestFreeTextSuggestions, CustomCollectionTest, SolrMetricReporterTest, 
SimpleFacetsTest, TestHashPartitioner, TestMergePolicyConfig, PluginInfoTest, 
TestHdfsBackupRestoreCore, UUIDFieldTest, InfoHandlerTest, 
TestClusterProperties, TestRandomFaceting, DateRangeFieldTest, 
CdcrReplicationHandlerTest, TestPseudoReturnFields, TestRequestForwarding, 
StatsComponentTest, TestOrdValues, TestLockTree, XmlUpdateRequestHandlerTest, 
TestOmitPositions, TestSolr4Spatial2, SolrXmlInZkTest, TestFilteredDocIdSet, 
DistributedFacetPivotWhiteBoxTest, TestComponentsName, 
CoreAdminCreateDiscoverTest, TestFieldCacheReopen, TestPushWriter, 
WrapperMergePolicyFactoryTest, DistributedExpandComponentTest, TestTlogReplica, 
SecurityConfHandlerTest, TestGeoJSONResponseWriter, TestLuceneMatchVersion, 
SynonymTokenizerTest, TestWordDelimiterFilterFactory, 
TestStressCloudBlindAtomicUpdates, DistributedFacetPivotLongTailTest, 
BlockDirectoryTest, LIRRollingUpdatesTest, TestBlobHandler, 
SimpleCollectionCreateDeleteTest, TestApiFramework, TestSolrDeletionPolicy1, 
CreateCollectionCleanupTest, TestExactSharedStatsCache, TestInitQParser, 
ManagedSchemaRoundRobinCloudTest, HdfsChaosMonkeyNothingIsSafeTest, 
TestPhraseSuggestions, TestBulkSchemaAPI, TestCloudManagedSchema, 
TestJsonFacetRefinement, TestTestInjection, MigrateRouteKeyTest, 
OpenCloseCoreStressTest, AtomicUpdatesTest, FullSolrCloudDistribCmdsTest, 
TestReloadAndDeleteDocs, TestCSVLoader, ResponseHeaderTest, 
ChaosMonkeySafeLeaderTest, BasicDistributedZk2Test, SyncSliceTest, 
ShardRoutingTest, TestZkChroot, TestRandomDVFaceting, TestFaceting, 
TestRecovery, TermVectorComponentDistributedTest, TestJoin, TestReload, 
TestStressVersions, DistributedTermsComponentTest, TestSolr4Spatial, 
TestGroupingSearch, PeerSyncTest, BadIndexSchemaTest, ConvertedLegacyTest, 
TestSort, TestFiltering, TestFunctionQuery, TestLazyCores, TestBadConfig, 
TestIndexSearcher, ShowFileRequestHandlerTest, 
DistributedQueryElevationComponentTest, SolrIndexSplitterTest, 
SimplePostToolTest, AnalysisAfterCoreReloadTest, CoreAdminHandlerTest, 
TestStressLucene, SpatialFilterTest, SuggesterWFSTTest, PolyFieldTest, 
SolrCoreCheckLockOnStartupTest, TestUpdate, DirectUpdateHandlerOptimizeTest, 
SortByFunctionTest, TestRemoteStreaming, DocValuesMultiTest, 
DistanceFunctionTest, DebugComponentTest, TestSurroundQueryParser, 
TestWriterPerf, PrimitiveFieldTypeTest, FileBasedSpellCheckerTest, 
DocumentBuilderTest, FastVectorHighlighterTest, SolrPluginUtilsTest, 
TestJmxIntegration, MBeansHandlerTest, TestSolrCoreProperties, 
NotRequiredUniqueKeyTest, TestXIncludeConfig, EchoParamsTest, 
TestDFRSimilarityFactory, TestIBSimilarityFactory, TestFastWriter, 
ResourceLoaderTest, ScriptEngineTest, TestFastLRUCache, PreAnalyzedFieldTest, 
ClusterStateTest, TestLRUCache, TestDocumentBuilder, SliceStateTest, 
SystemInfoHandlerTest, CircularListTest, CursorPagingTest, TestCrossCoreJoin, 
TestDistributedMissingSort, TestHighlightDedupGrouping, TestTolerantSearch, 
TestLuceneIndexBackCompat, ActionThrottleTest, 
AsyncCallRequestStatusResponseTest, ChaosMonkeySafeLeaderWithPullReplicasTest, 
ConfigSetsAPITest, CreateRoutedAliasTest, DeleteReplicaTest, 
DistribJoinFromCollectionTest, DistributedQueueTest, 
DistributedVersionInfoTest, HttpPartitionTest, 
LeaderFailureAfterFreshStartTest, MoveReplicaHDFSFailoverTest, NodeMutatorTest, 
OutOfBoxZkACLAndCredentialsProvidersTest, 
OverriddenZkACLAndCredentialsProvidersTest, OverseerRolesTest, 
OverseerTaskQueueTest, RecoveryAfterSoftCommitTest, ReplaceNodeTest, 
RollingRestartTest, SSLMigrationTest, SaslZkACLProviderTest, 
SolrCLIZkUtilsTest, SolrCloudExampleTest, TestCloudDeleteByQuery, 
TestCloudInspectUtil, TestCloudJSONFacetJoinDomain, TestConfigSetsAPI, 
TestConfigSetsAPIZkFailure, TestLeaderElectionZkExpiry, TestRandomFlRTGCloud, 
TestSolrCloudWithDelegationTokens, TestSolrCloudWithKerberosAlt, 
TestTolerantUpdateProcessorRandomCloud, TlogReplayBufferedWhileIndexingTest, 
AssignTest, CollectionsAPIDistributedZkTest, TestCollectionAPI, 
TestReplicaProperties, TestRequestStatusCollectionAPI, 
AutoAddReplicasIntegrationTest, HdfsAutoAddReplicasIntegrationTest, 
TestClusterStateProvider, TestComputePlanAction, TestDistribStateManager, 
TestExecutePlanAction, TestLargeCluster, TestTriggerIntegration, 
CdcrBidirectionalTest, CdcrBootstrapTest, CdcrVersionReplicationTest, 
HDFSCollectionsAPITest, HdfsChaosMonkeySafeLeaderTest]
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 
'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  
-Dtestcase=HdfsChaosMonkeySafeLeaderTest -Dtests.seed=77E9F6DCC7BA209B 
-Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true 
-Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt
 -Dtests.locale=id-ID -Dtests.timezone=Iceland -Dtests.asserts=true 
-Dtests.file.encoding=US-ASCII
   [junit4] ERROR   0.00s J0 | HdfsChaosMonkeySafeLeaderTest (suite) <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: ObjectTracker found 1 
object(s) that were not released!!! [NRTCachingDirectory]
   [junit4]    > 
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: 
org.apache.lucene.store.NRTCachingDirectory
   [junit4]    >        at 
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
   [junit4]    >        at 
org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)
   [junit4]    >        at 
org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:351)
   [junit4]    >        at 
org.apache.solr.core.SolrCore.cleanupOldIndexDirectories(SolrCore.java:3031)
   [junit4]    >        at 
org.apache.solr.core.SolrCore.close(SolrCore.java:1604)
   [junit4]    >        at 
org.apache.solr.core.SolrCore.<init>(SolrCore.java:1000)
   [junit4]    >        at 
org.apache.solr.core.SolrCore.<init>(SolrCore.java:863)
   [junit4]    >        at 
org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1043)
   [junit4]    >        at 
org.apache.solr.core.CoreContainer.lambda$load$13(CoreContainer.java:644)
   [junit4]    >        at 
com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
   [junit4]    >        at 
java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]    >        at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
   [junit4]    >        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4]    >        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]    >        at java.lang.Thread.run(Thread.java:748)
   [junit4]    >        at 
__randomizedtesting.SeedInfo.seed([77E9F6DCC7BA209B]:0)
   [junit4]    >        at 
org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:295)
   [junit4]    >        at java.lang.Thread.run(Thread.java:748)
   [junit4] Completed [608/775 (4!)] on J0 in 146.63s, 1 test, 1 failure <<< 
FAILURES!

[...truncated 603 lines...]
   [junit4] JVM J1: stdout was not empty, see: 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/temp/junit4-J1-20180204_143151_903340260775363412571.sysout
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: Java heap space
   [junit4] Dumping heap to 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/heapdumps/java_pid16269.hprof
 ...
   [junit4] Heap dump file created [450015838 bytes in 1.907 secs]
   [junit4] <<< JVM J1: EOF ----

[...truncated 8608 lines...]
BUILD FAILED
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/build.xml:844:
 The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/build.xml:788:
 Some of the tests produced a heap dump, but did not fail. Maybe a suppressed 
OutOfMemoryError? Dumps created:
* java_pid16269.hprof

Total time: 582 minutes 17 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Reply via email to