[JENKINS] Lucene-Solr-BadApples-7.x-Linux (64bit/jdk1.8.0_162) - Build # 7 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-BadApples-7.x-Linux/7/
Java: 64bit/jdk1.8.0_162 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC

33 tests failed.
FAILED:  
org.apache.solr.cloud.autoscaling.AutoAddReplicasIntegrationTest.testSimple

Error Message:
Waiting for collection testSimple1 null Live Nodes: [127.0.0.1:34077_solr, 
127.0.0.1:41817_solr] Last available state: 
DocCollection(testSimple1//collections/testSimple1/state.json/8)={   
"pullReplicas":"0",   "replicationFactor":"2",   "shards":{     "shard1":{      
 "range":"80000000-ffffffff",       "state":"active",       "replicas":{        
 "core_node3":{           "core":"testSimple1_shard1_replica_n1",           
"base_url":"https://127.0.0.1:34077/solr";,           
"node_name":"127.0.0.1:34077_solr",           "state":"active",           
"type":"NRT",           "leader":"true"},         "core_node5":{           
"core":"testSimple1_shard1_replica_n2",           
"base_url":"https://127.0.0.1:43077/solr";,           
"node_name":"127.0.0.1:43077_solr",           "state":"down",           
"type":"NRT"}}},     "shard2":{       "range":"0-7fffffff",       
"state":"active",       "replicas":{         "core_node7":{           
"core":"testSimple1_shard2_replica_n4",           
"base_url":"https://127.0.0.1:34077/solr";,           
"node_name":"127.0.0.1:34077_solr",           "state":"active",           
"type":"NRT",           "leader":"true"},         "core_node8":{           
"core":"testSimple1_shard2_replica_n6",           
"base_url":"https://127.0.0.1:43077/solr";,           
"node_name":"127.0.0.1:43077_solr",           "state":"down",           
"type":"NRT"}}}},   "router":{"name":"compositeId"},   "maxShardsPerNode":"2",  
 "autoAddReplicas":"true",   "nrtReplicas":"2",   "tlogReplicas":"0"}

Stack Trace:
java.lang.AssertionError: Waiting for collection testSimple1
null
Live Nodes: [127.0.0.1:34077_solr, 127.0.0.1:41817_solr]
Last available state: 
DocCollection(testSimple1//collections/testSimple1/state.json/8)={
  "pullReplicas":"0",
  "replicationFactor":"2",
  "shards":{
    "shard1":{
      "range":"80000000-ffffffff",
      "state":"active",
      "replicas":{
        "core_node3":{
          "core":"testSimple1_shard1_replica_n1",
          "base_url":"https://127.0.0.1:34077/solr";,
          "node_name":"127.0.0.1:34077_solr",
          "state":"active",
          "type":"NRT",
          "leader":"true"},
        "core_node5":{
          "core":"testSimple1_shard1_replica_n2",
          "base_url":"https://127.0.0.1:43077/solr";,
          "node_name":"127.0.0.1:43077_solr",
          "state":"down",
          "type":"NRT"}}},
    "shard2":{
      "range":"0-7fffffff",
      "state":"active",
      "replicas":{
        "core_node7":{
          "core":"testSimple1_shard2_replica_n4",
          "base_url":"https://127.0.0.1:34077/solr";,
          "node_name":"127.0.0.1:34077_solr",
          "state":"active",
          "type":"NRT",
          "leader":"true"},
        "core_node8":{
          "core":"testSimple1_shard2_replica_n6",
          "base_url":"https://127.0.0.1:43077/solr";,
          "node_name":"127.0.0.1:43077_solr",
          "state":"down",
          "type":"NRT"}}}},
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"2",
  "autoAddReplicas":"true",
  "nrtReplicas":"2",
  "tlogReplicas":"0"}
        at 
__randomizedtesting.SeedInfo.seed([33D4E2619A79FBD3:B67C69FBD8A2F02]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at 
org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:269)
        at 
org.apache.solr.cloud.autoscaling.AutoAddReplicasIntegrationTest.testSimple(AutoAddReplicasIntegrationTest.java:94)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  
org.apache.solr.cloud.autoscaling.AutoAddReplicasIntegrationTest.testSimple

Error Message:
Waiting for collection testSimple1 null Live Nodes: [127.0.0.1:38305_solr, 
127.0.0.1:41189_solr] Last available state: 
DocCollection(testSimple1//collections/testSimple1/state.json/6)={   
"pullReplicas":"0",   "replicationFactor":"2",   "shards":{     "shard1":{      
 "range":"80000000-ffffffff",       "state":"active",       "replicas":{        
 "core_node3":{           "core":"testSimple1_shard1_replica_n1",           
"base_url":"https://127.0.0.1:41189/solr";,           
"node_name":"127.0.0.1:41189_solr",           "state":"active",           
"type":"NRT",           "leader":"true"},         "core_node5":{           
"core":"testSimple1_shard1_replica_n2",           
"base_url":"https://127.0.0.1:39087/solr";,           
"node_name":"127.0.0.1:39087_solr",           "state":"down",           
"type":"NRT"}}},     "shard2":{       "range":"0-7fffffff",       
"state":"active",       "replicas":{         "core_node7":{           
"core":"testSimple1_shard2_replica_n4",           
"base_url":"https://127.0.0.1:41189/solr";,           
"node_name":"127.0.0.1:41189_solr",           "state":"active",           
"type":"NRT",           "leader":"true"},         "core_node8":{           
"core":"testSimple1_shard2_replica_n6",           
"base_url":"https://127.0.0.1:39087/solr";,           
"node_name":"127.0.0.1:39087_solr",           "state":"down",           
"type":"NRT"}}}},   "router":{"name":"compositeId"},   "maxShardsPerNode":"2",  
 "autoAddReplicas":"true",   "nrtReplicas":"2",   "tlogReplicas":"0"}

Stack Trace:
java.lang.AssertionError: Waiting for collection testSimple1
null
Live Nodes: [127.0.0.1:38305_solr, 127.0.0.1:41189_solr]
Last available state: 
DocCollection(testSimple1//collections/testSimple1/state.json/6)={
  "pullReplicas":"0",
  "replicationFactor":"2",
  "shards":{
    "shard1":{
      "range":"80000000-ffffffff",
      "state":"active",
      "replicas":{
        "core_node3":{
          "core":"testSimple1_shard1_replica_n1",
          "base_url":"https://127.0.0.1:41189/solr";,
          "node_name":"127.0.0.1:41189_solr",
          "state":"active",
          "type":"NRT",
          "leader":"true"},
        "core_node5":{
          "core":"testSimple1_shard1_replica_n2",
          "base_url":"https://127.0.0.1:39087/solr";,
          "node_name":"127.0.0.1:39087_solr",
          "state":"down",
          "type":"NRT"}}},
    "shard2":{
      "range":"0-7fffffff",
      "state":"active",
      "replicas":{
        "core_node7":{
          "core":"testSimple1_shard2_replica_n4",
          "base_url":"https://127.0.0.1:41189/solr";,
          "node_name":"127.0.0.1:41189_solr",
          "state":"active",
          "type":"NRT",
          "leader":"true"},
        "core_node8":{
          "core":"testSimple1_shard2_replica_n6",
          "base_url":"https://127.0.0.1:39087/solr";,
          "node_name":"127.0.0.1:39087_solr",
          "state":"down",
          "type":"NRT"}}}},
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"2",
  "autoAddReplicas":"true",
  "nrtReplicas":"2",
  "tlogReplicas":"0"}
        at 
__randomizedtesting.SeedInfo.seed([33D4E2619A79FBD3:B67C69FBD8A2F02]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at 
org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:269)
        at 
org.apache.solr.cloud.autoscaling.AutoAddReplicasIntegrationTest.testSimple(AutoAddReplicasIntegrationTest.java:94)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  
org.apache.solr.cloud.autoscaling.AutoAddReplicasIntegrationTest.testSimple

Error Message:
Waiting for collection testSimple1 null Live Nodes: [127.0.0.1:33101_solr, 
127.0.0.1:34167_solr] Last available state: 
DocCollection(testSimple1//collections/testSimple1/state.json/10)={   
"pullReplicas":"0",   "replicationFactor":"2",   "shards":{     "shard1":{      
 "range":"80000000-ffffffff",       "state":"active",       "replicas":{        
 "core_node3":{           "core":"testSimple1_shard1_replica_n1",           
"base_url":"https://127.0.0.1:33101/solr";,           
"node_name":"127.0.0.1:33101_solr",           "state":"active",           
"type":"NRT",           "leader":"true"},         "core_node5":{           
"core":"testSimple1_shard1_replica_n2",           
"base_url":"https://127.0.0.1:36967/solr";,           
"node_name":"127.0.0.1:36967_solr",           "state":"down",           
"type":"NRT"}}},     "shard2":{       "range":"0-7fffffff",       
"state":"active",       "replicas":{         "core_node7":{           
"core":"testSimple1_shard2_replica_n4",           
"base_url":"https://127.0.0.1:33101/solr";,           
"node_name":"127.0.0.1:33101_solr",           "state":"active",           
"type":"NRT",           "leader":"true"},         "core_node8":{           
"core":"testSimple1_shard2_replica_n6",           
"base_url":"https://127.0.0.1:36967/solr";,           
"node_name":"127.0.0.1:36967_solr",           "state":"down",           
"type":"NRT"}}}},   "router":{"name":"compositeId"},   "maxShardsPerNode":"2",  
 "autoAddReplicas":"true",   "nrtReplicas":"2",   "tlogReplicas":"0"}

Stack Trace:
java.lang.AssertionError: Waiting for collection testSimple1
null
Live Nodes: [127.0.0.1:33101_solr, 127.0.0.1:34167_solr]
Last available state: 
DocCollection(testSimple1//collections/testSimple1/state.json/10)={
  "pullReplicas":"0",
  "replicationFactor":"2",
  "shards":{
    "shard1":{
      "range":"80000000-ffffffff",
      "state":"active",
      "replicas":{
        "core_node3":{
          "core":"testSimple1_shard1_replica_n1",
          "base_url":"https://127.0.0.1:33101/solr";,
          "node_name":"127.0.0.1:33101_solr",
          "state":"active",
          "type":"NRT",
          "leader":"true"},
        "core_node5":{
          "core":"testSimple1_shard1_replica_n2",
          "base_url":"https://127.0.0.1:36967/solr";,
          "node_name":"127.0.0.1:36967_solr",
          "state":"down",
          "type":"NRT"}}},
    "shard2":{
      "range":"0-7fffffff",
      "state":"active",
      "replicas":{
        "core_node7":{
          "core":"testSimple1_shard2_replica_n4",
          "base_url":"https://127.0.0.1:33101/solr";,
          "node_name":"127.0.0.1:33101_solr",
          "state":"active",
          "type":"NRT",
          "leader":"true"},
        "core_node8":{
          "core":"testSimple1_shard2_replica_n6",
          "base_url":"https://127.0.0.1:36967/solr";,
          "node_name":"127.0.0.1:36967_solr",
          "state":"down",
          "type":"NRT"}}}},
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"2",
  "autoAddReplicas":"true",
  "nrtReplicas":"2",
  "tlogReplicas":"0"}
        at 
__randomizedtesting.SeedInfo.seed([33D4E2619A79FBD3:B67C69FBD8A2F02]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at 
org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:269)
        at 
org.apache.solr.cloud.autoscaling.AutoAddReplicasIntegrationTest.testSimple(AutoAddReplicasIntegrationTest.java:94)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  
org.apache.solr.cloud.autoscaling.AutoAddReplicasIntegrationTest.testSimple

Error Message:
Waiting for collection testSimple1 null Live Nodes: [127.0.0.1:35515_solr, 
127.0.0.1:42103_solr] Last available state: 
DocCollection(testSimple1//collections/testSimple1/state.json/9)={   
"pullReplicas":"0",   "replicationFactor":"2",   "shards":{     "shard1":{      
 "range":"80000000-ffffffff",       "state":"active",       "replicas":{        
 "core_node3":{           "core":"testSimple1_shard1_replica_n1",           
"base_url":"https://127.0.0.1:36997/solr";,           
"node_name":"127.0.0.1:36997_solr",           "state":"down",           
"type":"NRT"},         "core_node5":{           
"core":"testSimple1_shard1_replica_n2",           
"base_url":"https://127.0.0.1:35515/solr";,           
"node_name":"127.0.0.1:35515_solr",           "state":"active",           
"type":"NRT",           "leader":"true"}}},     "shard2":{       
"range":"0-7fffffff",       "state":"active",       "replicas":{         
"core_node7":{           "core":"testSimple1_shard2_replica_n4",           
"base_url":"https://127.0.0.1:36997/solr";,           
"node_name":"127.0.0.1:36997_solr",           "state":"down",           
"type":"NRT"},         "core_node8":{           
"core":"testSimple1_shard2_replica_n6",           
"base_url":"https://127.0.0.1:35515/solr";,           
"node_name":"127.0.0.1:35515_solr",           "state":"active",           
"type":"NRT",           "leader":"true"}}}},   "router":{"name":"compositeId"}, 
  "maxShardsPerNode":"2",   "autoAddReplicas":"true",   "nrtReplicas":"2",   
"tlogReplicas":"0"}

Stack Trace:
java.lang.AssertionError: Waiting for collection testSimple1
null
Live Nodes: [127.0.0.1:35515_solr, 127.0.0.1:42103_solr]
Last available state: 
DocCollection(testSimple1//collections/testSimple1/state.json/9)={
  "pullReplicas":"0",
  "replicationFactor":"2",
  "shards":{
    "shard1":{
      "range":"80000000-ffffffff",
      "state":"active",
      "replicas":{
        "core_node3":{
          "core":"testSimple1_shard1_replica_n1",
          "base_url":"https://127.0.0.1:36997/solr";,
          "node_name":"127.0.0.1:36997_solr",
          "state":"down",
          "type":"NRT"},
        "core_node5":{
          "core":"testSimple1_shard1_replica_n2",
          "base_url":"https://127.0.0.1:35515/solr";,
          "node_name":"127.0.0.1:35515_solr",
          "state":"active",
          "type":"NRT",
          "leader":"true"}}},
    "shard2":{
      "range":"0-7fffffff",
      "state":"active",
      "replicas":{
        "core_node7":{
          "core":"testSimple1_shard2_replica_n4",
          "base_url":"https://127.0.0.1:36997/solr";,
          "node_name":"127.0.0.1:36997_solr",
          "state":"down",
          "type":"NRT"},
        "core_node8":{
          "core":"testSimple1_shard2_replica_n6",
          "base_url":"https://127.0.0.1:35515/solr";,
          "node_name":"127.0.0.1:35515_solr",
          "state":"active",
          "type":"NRT",
          "leader":"true"}}}},
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"2",
  "autoAddReplicas":"true",
  "nrtReplicas":"2",
  "tlogReplicas":"0"}
        at 
__randomizedtesting.SeedInfo.seed([33D4E2619A79FBD3:B67C69FBD8A2F02]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at 
org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:269)
        at 
org.apache.solr.cloud.autoscaling.AutoAddReplicasIntegrationTest.testSimple(AutoAddReplicasIntegrationTest.java:94)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  
org.apache.solr.cloud.autoscaling.AutoAddReplicasIntegrationTest.testSimple

Error Message:
Waiting for collection testSimple1 null Live Nodes: [127.0.0.1:39909_solr, 
127.0.0.1:43131_solr] Last available state: 
DocCollection(testSimple1//collections/testSimple1/state.json/9)={   
"pullReplicas":"0",   "replicationFactor":"2",   "shards":{     "shard1":{      
 "range":"80000000-ffffffff",       "state":"active",       "replicas":{        
 "core_node3":{           "core":"testSimple1_shard1_replica_n1",           
"base_url":"https://127.0.0.1:43131/solr";,           
"node_name":"127.0.0.1:43131_solr",           "state":"active",           
"type":"NRT",           "leader":"true"},         "core_node5":{           
"core":"testSimple1_shard1_replica_n2",           
"base_url":"https://127.0.0.1:46441/solr";,           
"node_name":"127.0.0.1:46441_solr",           "state":"down",           
"type":"NRT"}}},     "shard2":{       "range":"0-7fffffff",       
"state":"active",       "replicas":{         "core_node7":{           
"core":"testSimple1_shard2_replica_n4",           
"base_url":"https://127.0.0.1:43131/solr";,           
"node_name":"127.0.0.1:43131_solr",           "state":"active",           
"type":"NRT",           "leader":"true"},         "core_node8":{           
"core":"testSimple1_shard2_replica_n6",           
"base_url":"https://127.0.0.1:46441/solr";,           
"node_name":"127.0.0.1:46441_solr",           "state":"down",           
"type":"NRT"}}}},   "router":{"name":"compositeId"},   "maxShardsPerNode":"2",  
 "autoAddReplicas":"true",   "nrtReplicas":"2",   "tlogReplicas":"0"}

Stack Trace:
java.lang.AssertionError: Waiting for collection testSimple1
null
Live Nodes: [127.0.0.1:39909_solr, 127.0.0.1:43131_solr]
Last available state: 
DocCollection(testSimple1//collections/testSimple1/state.json/9)={
  "pullReplicas":"0",
  "replicationFactor":"2",
  "shards":{
    "shard1":{
      "range":"80000000-ffffffff",
      "state":"active",
      "replicas":{
        "core_node3":{
          "core":"testSimple1_shard1_replica_n1",
          "base_url":"https://127.0.0.1:43131/solr";,
          "node_name":"127.0.0.1:43131_solr",
          "state":"active",
          "type":"NRT",
          "leader":"true"},
        "core_node5":{
          "core":"testSimple1_shard1_replica_n2",
          "base_url":"https://127.0.0.1:46441/solr";,
          "node_name":"127.0.0.1:46441_solr",
          "state":"down",
          "type":"NRT"}}},
    "shard2":{
      "range":"0-7fffffff",
      "state":"active",
      "replicas":{
        "core_node7":{
          "core":"testSimple1_shard2_replica_n4",
          "base_url":"https://127.0.0.1:43131/solr";,
          "node_name":"127.0.0.1:43131_solr",
          "state":"active",
          "type":"NRT",
          "leader":"true"},
        "core_node8":{
          "core":"testSimple1_shard2_replica_n6",
          "base_url":"https://127.0.0.1:46441/solr";,
          "node_name":"127.0.0.1:46441_solr",
          "state":"down",
          "type":"NRT"}}}},
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"2",
  "autoAddReplicas":"true",
  "nrtReplicas":"2",
  "tlogReplicas":"0"}
        at 
__randomizedtesting.SeedInfo.seed([33D4E2619A79FBD3:B67C69FBD8A2F02]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at 
org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:269)
        at 
org.apache.solr.cloud.autoscaling.AutoAddReplicasIntegrationTest.testSimple(AutoAddReplicasIntegrationTest.java:94)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  
org.apache.solr.cloud.autoscaling.AutoAddReplicasIntegrationTest.testSimple

Error Message:
Waiting for collection testSimple1 null Live Nodes: [127.0.0.1:34705_solr, 
127.0.0.1:41827_solr] Last available state: 
DocCollection(testSimple1//collections/testSimple1/state.json/6)={   
"pullReplicas":"0",   "replicationFactor":"2",   "shards":{     "shard1":{      
 "range":"80000000-ffffffff",       "state":"active",       "replicas":{        
 "core_node3":{           "core":"testSimple1_shard1_replica_n1",           
"base_url":"https://127.0.0.1:41827/solr";,           
"node_name":"127.0.0.1:41827_solr",           "state":"active",           
"type":"NRT",           "leader":"true"},         "core_node5":{           
"core":"testSimple1_shard1_replica_n2",           
"base_url":"https://127.0.0.1:43739/solr";,           
"node_name":"127.0.0.1:43739_solr",           "state":"down",           
"type":"NRT"}}},     "shard2":{       "range":"0-7fffffff",       
"state":"active",       "replicas":{         "core_node7":{           
"core":"testSimple1_shard2_replica_n4",           
"base_url":"https://127.0.0.1:41827/solr";,           
"node_name":"127.0.0.1:41827_solr",           "state":"active",           
"type":"NRT",           "leader":"true"},         "core_node8":{           
"core":"testSimple1_shard2_replica_n6",           
"base_url":"https://127.0.0.1:43739/solr";,           
"node_name":"127.0.0.1:43739_solr",           "state":"down",           
"type":"NRT"}}}},   "router":{"name":"compositeId"},   "maxShardsPerNode":"2",  
 "autoAddReplicas":"true",   "nrtReplicas":"2",   "tlogReplicas":"0"}

Stack Trace:
java.lang.AssertionError: Waiting for collection testSimple1
null
Live Nodes: [127.0.0.1:34705_solr, 127.0.0.1:41827_solr]
Last available state: 
DocCollection(testSimple1//collections/testSimple1/state.json/6)={
  "pullReplicas":"0",
  "replicationFactor":"2",
  "shards":{
    "shard1":{
      "range":"80000000-ffffffff",
      "state":"active",
      "replicas":{
        "core_node3":{
          "core":"testSimple1_shard1_replica_n1",
          "base_url":"https://127.0.0.1:41827/solr";,
          "node_name":"127.0.0.1:41827_solr",
          "state":"active",
          "type":"NRT",
          "leader":"true"},
        "core_node5":{
          "core":"testSimple1_shard1_replica_n2",
          "base_url":"https://127.0.0.1:43739/solr";,
          "node_name":"127.0.0.1:43739_solr",
          "state":"down",
          "type":"NRT"}}},
    "shard2":{
      "range":"0-7fffffff",
      "state":"active",
      "replicas":{
        "core_node7":{
          "core":"testSimple1_shard2_replica_n4",
          "base_url":"https://127.0.0.1:41827/solr";,
          "node_name":"127.0.0.1:41827_solr",
          "state":"active",
          "type":"NRT",
          "leader":"true"},
        "core_node8":{
          "core":"testSimple1_shard2_replica_n6",
          "base_url":"https://127.0.0.1:43739/solr";,
          "node_name":"127.0.0.1:43739_solr",
          "state":"down",
          "type":"NRT"}}}},
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"2",
  "autoAddReplicas":"true",
  "nrtReplicas":"2",
  "tlogReplicas":"0"}
        at 
__randomizedtesting.SeedInfo.seed([33D4E2619A79FBD3:B67C69FBD8A2F02]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at 
org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:269)
        at 
org.apache.solr.cloud.autoscaling.AutoAddReplicasIntegrationTest.testSimple(AutoAddReplicasIntegrationTest.java:94)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  
org.apache.solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest.testSimple

Error Message:
Waiting for collection testSimple1 null Live Nodes: [127.0.0.1:36685_solr, 
127.0.0.1:40243_solr] Last available state: 
DocCollection(testSimple1//collections/testSimple1/state.json/7)={   
"pullReplicas":"0",   "replicationFactor":"2",   "shards":{     "shard1":{      
 "range":"80000000-ffffffff",       "state":"active",       "replicas":{        
 "core_node3":{           
"dataDir":"hdfs://localhost.localdomain:42111/data/testSimple1/core_node3/data/",
           "base_url":"https://127.0.0.1:40959/solr";,           
"node_name":"127.0.0.1:40959_solr",           "type":"NRT",           
"ulogDir":"hdfs://localhost.localdomain:42111/data/testSimple1/core_node3/data/tlog",
           "core":"testSimple1_shard1_replica_n1",           
"shared_storage":"true",           "state":"down"},         "core_node5":{      
     
"dataDir":"hdfs://localhost.localdomain:42111/data/testSimple1/core_node5/data/",
           "base_url":"https://127.0.0.1:40243/solr";,           
"node_name":"127.0.0.1:40243_solr",           "type":"NRT",           
"ulogDir":"hdfs://localhost.localdomain:42111/data/testSimple1/core_node5/data/tlog",
           "core":"testSimple1_shard1_replica_n2",           
"shared_storage":"true",           "state":"active",           
"leader":"true"}}},     "shard2":{       "range":"0-7fffffff",       
"state":"active",       "replicas":{         "core_node7":{           
"dataDir":"hdfs://localhost.localdomain:42111/data/testSimple1/core_node7/data/",
           "base_url":"https://127.0.0.1:40959/solr";,           
"node_name":"127.0.0.1:40959_solr",           "type":"NRT",           
"ulogDir":"hdfs://localhost.localdomain:42111/data/testSimple1/core_node7/data/tlog",
           "core":"testSimple1_shard2_replica_n4",           
"shared_storage":"true",           "state":"down"},         "core_node8":{      
     
"dataDir":"hdfs://localhost.localdomain:42111/data/testSimple1/core_node8/data/",
           "base_url":"https://127.0.0.1:40243/solr";,           
"node_name":"127.0.0.1:40243_solr",           "type":"NRT",           
"ulogDir":"hdfs://localhost.localdomain:42111/data/testSimple1/core_node8/data/tlog",
           "core":"testSimple1_shard2_replica_n6",           
"shared_storage":"true",           "state":"active",           
"leader":"true"}}}},   "router":{"name":"compositeId"},   
"maxShardsPerNode":"2",   "autoAddReplicas":"true",   "nrtReplicas":"2",   
"tlogReplicas":"0"}

Stack Trace:
java.lang.AssertionError: Waiting for collection testSimple1
null
Live Nodes: [127.0.0.1:36685_solr, 127.0.0.1:40243_solr]
Last available state: 
DocCollection(testSimple1//collections/testSimple1/state.json/7)={
  "pullReplicas":"0",
  "replicationFactor":"2",
  "shards":{
    "shard1":{
      "range":"80000000-ffffffff",
      "state":"active",
      "replicas":{
        "core_node3":{
          
"dataDir":"hdfs://localhost.localdomain:42111/data/testSimple1/core_node3/data/",
          "base_url":"https://127.0.0.1:40959/solr";,
          "node_name":"127.0.0.1:40959_solr",
          "type":"NRT",
          
"ulogDir":"hdfs://localhost.localdomain:42111/data/testSimple1/core_node3/data/tlog",
          "core":"testSimple1_shard1_replica_n1",
          "shared_storage":"true",
          "state":"down"},
        "core_node5":{
          
"dataDir":"hdfs://localhost.localdomain:42111/data/testSimple1/core_node5/data/",
          "base_url":"https://127.0.0.1:40243/solr";,
          "node_name":"127.0.0.1:40243_solr",
          "type":"NRT",
          
"ulogDir":"hdfs://localhost.localdomain:42111/data/testSimple1/core_node5/data/tlog",
          "core":"testSimple1_shard1_replica_n2",
          "shared_storage":"true",
          "state":"active",
          "leader":"true"}}},
    "shard2":{
      "range":"0-7fffffff",
      "state":"active",
      "replicas":{
        "core_node7":{
          
"dataDir":"hdfs://localhost.localdomain:42111/data/testSimple1/core_node7/data/",
          "base_url":"https://127.0.0.1:40959/solr";,
          "node_name":"127.0.0.1:40959_solr",
          "type":"NRT",
          
"ulogDir":"hdfs://localhost.localdomain:42111/data/testSimple1/core_node7/data/tlog",
          "core":"testSimple1_shard2_replica_n4",
          "shared_storage":"true",
          "state":"down"},
        "core_node8":{
          
"dataDir":"hdfs://localhost.localdomain:42111/data/testSimple1/core_node8/data/",
          "base_url":"https://127.0.0.1:40243/solr";,
          "node_name":"127.0.0.1:40243_solr",
          "type":"NRT",
          
"ulogDir":"hdfs://localhost.localdomain:42111/data/testSimple1/core_node8/data/tlog",
          "core":"testSimple1_shard2_replica_n6",
          "shared_storage":"true",
          "state":"active",
          "leader":"true"}}}},
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"2",
  "autoAddReplicas":"true",
  "nrtReplicas":"2",
  "tlogReplicas":"0"}
        at 
__randomizedtesting.SeedInfo.seed([33D4E2619A79FBD3:B67C69FBD8A2F02]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at 
org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:269)
        at 
org.apache.solr.cloud.autoscaling.AutoAddReplicasIntegrationTest.testSimple(AutoAddReplicasIntegrationTest.java:94)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  
org.apache.solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest.testSimple

Error Message:
Waiting for collection testSimple1 null Live Nodes: [127.0.0.1:36071_solr, 
127.0.0.1:46629_solr] Last available state: 
DocCollection(testSimple1//collections/testSimple1/state.json/7)={   
"pullReplicas":"0",   "replicationFactor":"2",   "shards":{     "shard1":{      
 "range":"80000000-ffffffff",       "state":"active",       "replicas":{        
 "core_node3":{           
"dataDir":"hdfs://localhost.localdomain:38395/data/testSimple1/core_node3/data/",
           "base_url":"https://127.0.0.1:39477/solr";,           
"node_name":"127.0.0.1:39477_solr",           "type":"NRT",           
"ulogDir":"hdfs://localhost.localdomain:38395/data/testSimple1/core_node3/data/tlog",
           "core":"testSimple1_shard1_replica_n1",           
"shared_storage":"true",           "state":"active"},         "core_node5":{    
       
"dataDir":"hdfs://localhost.localdomain:38395/data/testSimple1/core_node5/data/",
           "base_url":"https://127.0.0.1:36071/solr";,           
"node_name":"127.0.0.1:36071_solr",           "type":"NRT",           
"ulogDir":"hdfs://localhost.localdomain:38395/data/testSimple1/core_node5/data/tlog",
           "core":"testSimple1_shard1_replica_n2",           
"shared_storage":"true",           "state":"active",           
"leader":"true"}}},     "shard2":{       "range":"0-7fffffff",       
"state":"active",       "replicas":{         "core_node7":{           
"dataDir":"hdfs://localhost.localdomain:38395/data/testSimple1/core_node7/data/",
           "base_url":"https://127.0.0.1:39477/solr";,           
"node_name":"127.0.0.1:39477_solr",           "type":"NRT",           
"ulogDir":"hdfs://localhost.localdomain:38395/data/testSimple1/core_node7/data/tlog",
           "core":"testSimple1_shard2_replica_n4",           
"shared_storage":"true",           "state":"active"},         "core_node8":{    
       
"dataDir":"hdfs://localhost.localdomain:38395/data/testSimple1/core_node8/data/",
           "base_url":"https://127.0.0.1:36071/solr";,           
"node_name":"127.0.0.1:36071_solr",           "type":"NRT",           
"ulogDir":"hdfs://localhost.localdomain:38395/data/testSimple1/core_node8/data/tlog",
           "core":"testSimple1_shard2_replica_n6",           
"shared_storage":"true",           "state":"active",           
"leader":"true"}}}},   "router":{"name":"compositeId"},   
"maxShardsPerNode":"2",   "autoAddReplicas":"true",   "nrtReplicas":"2",   
"tlogReplicas":"0"}

Stack Trace:
java.lang.AssertionError: Waiting for collection testSimple1
null
Live Nodes: [127.0.0.1:36071_solr, 127.0.0.1:46629_solr]
Last available state: 
DocCollection(testSimple1//collections/testSimple1/state.json/7)={
  "pullReplicas":"0",
  "replicationFactor":"2",
  "shards":{
    "shard1":{
      "range":"80000000-ffffffff",
      "state":"active",
      "replicas":{
        "core_node3":{
          
"dataDir":"hdfs://localhost.localdomain:38395/data/testSimple1/core_node3/data/",
          "base_url":"https://127.0.0.1:39477/solr";,
          "node_name":"127.0.0.1:39477_solr",
          "type":"NRT",
          
"ulogDir":"hdfs://localhost.localdomain:38395/data/testSimple1/core_node3/data/tlog",
          "core":"testSimple1_shard1_replica_n1",
          "shared_storage":"true",
          "state":"active"},
        "core_node5":{
          
"dataDir":"hdfs://localhost.localdomain:38395/data/testSimple1/core_node5/data/",
          "base_url":"https://127.0.0.1:36071/solr";,
          "node_name":"127.0.0.1:36071_solr",
          "type":"NRT",
          
"ulogDir":"hdfs://localhost.localdomain:38395/data/testSimple1/core_node5/data/tlog",
          "core":"testSimple1_shard1_replica_n2",
          "shared_storage":"true",
          "state":"active",
          "leader":"true"}}},
    "shard2":{
      "range":"0-7fffffff",
      "state":"active",
      "replicas":{
        "core_node7":{
          
"dataDir":"hdfs://localhost.localdomain:38395/data/testSimple1/core_node7/data/",
          "base_url":"https://127.0.0.1:39477/solr";,
          "node_name":"127.0.0.1:39477_solr",
          "type":"NRT",
          
"ulogDir":"hdfs://localhost.localdomain:38395/data/testSimple1/core_node7/data/tlog",
          "core":"testSimple1_shard2_replica_n4",
          "shared_storage":"true",
          "state":"active"},
        "core_node8":{
          
"dataDir":"hdfs://localhost.localdomain:38395/data/testSimple1/core_node8/data/",
          "base_url":"https://127.0.0.1:36071/solr";,
          "node_name":"127.0.0.1:36071_solr",
          "type":"NRT",
          
"ulogDir":"hdfs://localhost.localdomain:38395/data/testSimple1/core_node8/data/tlog",
          "core":"testSimple1_shard2_replica_n6",
          "shared_storage":"true",
          "state":"active",
          "leader":"true"}}}},
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"2",
  "autoAddReplicas":"true",
  "nrtReplicas":"2",
  "tlogReplicas":"0"}
        at 
__randomizedtesting.SeedInfo.seed([33D4E2619A79FBD3:B67C69FBD8A2F02]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at 
org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:269)
        at 
org.apache.solr.cloud.autoscaling.AutoAddReplicasIntegrationTest.testSimple(AutoAddReplicasIntegrationTest.java:94)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  
org.apache.solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest.testSimple

Error Message:
Waiting for collection testSimple1 null Live Nodes: [127.0.0.1:32845_solr, 
127.0.0.1:33239_solr] Last available state: 
DocCollection(testSimple1//collections/testSimple1/state.json/8)={   
"pullReplicas":"0",   "replicationFactor":"2",   "shards":{     "shard1":{      
 "range":"80000000-ffffffff",       "state":"active",       "replicas":{        
 "core_node3":{           
"dataDir":"hdfs://localhost.localdomain:42599/data/testSimple1/core_node3/data/",
           "base_url":"https://127.0.0.1:35507/solr";,           
"node_name":"127.0.0.1:35507_solr",           "type":"NRT",           
"ulogDir":"hdfs://localhost.localdomain:42599/data/testSimple1/core_node3/data/tlog",
           "core":"testSimple1_shard1_replica_n1",           
"shared_storage":"true",           "state":"down"},         "core_node5":{      
     
"dataDir":"hdfs://localhost.localdomain:42599/data/testSimple1/core_node5/data/",
           "base_url":"https://127.0.0.1:33239/solr";,           
"node_name":"127.0.0.1:33239_solr",           "type":"NRT",           
"ulogDir":"hdfs://localhost.localdomain:42599/data/testSimple1/core_node5/data/tlog",
           "core":"testSimple1_shard1_replica_n2",           
"shared_storage":"true",           "state":"active",           
"leader":"true"}}},     "shard2":{       "range":"0-7fffffff",       
"state":"active",       "replicas":{         "core_node7":{           
"dataDir":"hdfs://localhost.localdomain:42599/data/testSimple1/core_node7/data/",
           "base_url":"https://127.0.0.1:35507/solr";,           
"node_name":"127.0.0.1:35507_solr",           "type":"NRT",           
"ulogDir":"hdfs://localhost.localdomain:42599/data/testSimple1/core_node7/data/tlog",
           "core":"testSimple1_shard2_replica_n4",           
"shared_storage":"true",           "state":"down"},         "core_node8":{      
     
"dataDir":"hdfs://localhost.localdomain:42599/data/testSimple1/core_node8/data/",
           "base_url":"https://127.0.0.1:33239/solr";,           
"node_name":"127.0.0.1:33239_solr",           "type":"NRT",           
"ulogDir":"hdfs://localhost.localdomain:42599/data/testSimple1/core_node8/data/tlog",
           "core":"testSimple1_shard2_replica_n6",           
"shared_storage":"true",           "state":"active",           
"leader":"true"}}}},   "router":{"name":"compositeId"},   
"maxShardsPerNode":"2",   "autoAddReplicas":"true",   "nrtReplicas":"2",   
"tlogReplicas":"0"}

Stack Trace:
java.lang.AssertionError: Waiting for collection testSimple1
null
Live Nodes: [127.0.0.1:32845_solr, 127.0.0.1:33239_solr]
Last available state: 
DocCollection(testSimple1//collections/testSimple1/state.json/8)={
  "pullReplicas":"0",
  "replicationFactor":"2",
  "shards":{
    "shard1":{
      "range":"80000000-ffffffff",
      "state":"active",
      "replicas":{
        "core_node3":{
          
"dataDir":"hdfs://localhost.localdomain:42599/data/testSimple1/core_node3/data/",
          "base_url":"https://127.0.0.1:35507/solr";,
          "node_name":"127.0.0.1:35507_solr",
          "type":"NRT",
          
"ulogDir":"hdfs://localhost.localdomain:42599/data/testSimple1/core_node3/data/tlog",
          "core":"testSimple1_shard1_replica_n1",
          "shared_storage":"true",
          "state":"down"},
        "core_node5":{
          
"dataDir":"hdfs://localhost.localdomain:42599/data/testSimple1/core_node5/data/",
          "base_url":"https://127.0.0.1:33239/solr";,
          "node_name":"127.0.0.1:33239_solr",
          "type":"NRT",
          
"ulogDir":"hdfs://localhost.localdomain:42599/data/testSimple1/core_node5/data/tlog",
          "core":"testSimple1_shard1_replica_n2",
          "shared_storage":"true",
          "state":"active",
          "leader":"true"}}},
    "shard2":{
      "range":"0-7fffffff",
      "state":"active",
      "replicas":{
        "core_node7":{
          
"dataDir":"hdfs://localhost.localdomain:42599/data/testSimple1/core_node7/data/",
          "base_url":"https://127.0.0.1:35507/solr";,
          "node_name":"127.0.0.1:35507_solr",
          "type":"NRT",
          
"ulogDir":"hdfs://localhost.localdomain:42599/data/testSimple1/core_node7/data/tlog",
          "core":"testSimple1_shard2_replica_n4",
          "shared_storage":"true",
          "state":"down"},
        "core_node8":{
          
"dataDir":"hdfs://localhost.localdomain:42599/data/testSimple1/core_node8/data/",
          "base_url":"https://127.0.0.1:33239/solr";,
          "node_name":"127.0.0.1:33239_solr",
          "type":"NRT",
          
"ulogDir":"hdfs://localhost.localdomain:42599/data/testSimple1/core_node8/data/tlog",
          "core":"testSimple1_shard2_replica_n6",
          "shared_storage":"true",
          "state":"active",
          "leader":"true"}}}},
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"2",
  "autoAddReplicas":"true",
  "nrtReplicas":"2",
  "tlogReplicas":"0"}
        at 
__randomizedtesting.SeedInfo.seed([33D4E2619A79FBD3:B67C69FBD8A2F02]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at 
org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:269)
        at 
org.apache.solr.cloud.autoscaling.AutoAddReplicasIntegrationTest.testSimple(AutoAddReplicasIntegrationTest.java:94)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  
org.apache.solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest.testSimple

Error Message:
Waiting for collection testSimple1 null Live Nodes: [127.0.0.1:32889_solr, 
127.0.0.1:37983_solr] Last available state: 
DocCollection(testSimple1//collections/testSimple1/state.json/10)={   
"pullReplicas":"0",   "replicationFactor":"2",   "shards":{     "shard1":{      
 "range":"80000000-ffffffff",       "state":"active",       "replicas":{        
 "core_node3":{           
"dataDir":"hdfs://localhost.localdomain:39667/data/testSimple1/core_node3/data/",
           "base_url":"https://127.0.0.1:35131/solr";,           
"node_name":"127.0.0.1:35131_solr",           "type":"NRT",           
"ulogDir":"hdfs://localhost.localdomain:39667/data/testSimple1/core_node3/data/tlog",
           "core":"testSimple1_shard1_replica_n1",           
"shared_storage":"true",           "state":"down"},         "core_node5":{      
     
"dataDir":"hdfs://localhost.localdomain:39667/data/testSimple1/core_node5/data/",
           "base_url":"https://127.0.0.1:37983/solr";,           
"node_name":"127.0.0.1:37983_solr",           "type":"NRT",           
"ulogDir":"hdfs://localhost.localdomain:39667/data/testSimple1/core_node5/data/tlog",
           "core":"testSimple1_shard1_replica_n2",           
"shared_storage":"true",           "state":"active",           
"leader":"true"}}},     "shard2":{       "range":"0-7fffffff",       
"state":"active",       "replicas":{         "core_node7":{           
"dataDir":"hdfs://localhost.localdomain:39667/data/testSimple1/core_node7/data/",
           "base_url":"https://127.0.0.1:35131/solr";,           
"node_name":"127.0.0.1:35131_solr",           "type":"NRT",           
"ulogDir":"hdfs://localhost.localdomain:39667/data/testSimple1/core_node7/data/tlog",
           "core":"testSimple1_shard2_replica_n4",           
"shared_storage":"true",           "state":"down"},         "core_node8":{      
     
"dataDir":"hdfs://localhost.localdomain:39667/data/testSimple1/core_node8/data/",
           "base_url":"https://127.0.0.1:37983/solr";,           
"node_name":"127.0.0.1:37983_solr",           "type":"NRT",           
"ulogDir":"hdfs://localhost.localdomain:39667/data/testSimple1/core_node8/data/tlog",
           "core":"testSimple1_shard2_replica_n6",           
"shared_storage":"true",           "state":"active",           
"leader":"true"}}}},   "router":{"name":"compositeId"},   
"maxShardsPerNode":"2",   "autoAddReplicas":"true",   "nrtReplicas":"2",   
"tlogReplicas":"0"}

Stack Trace:
java.lang.AssertionError: Waiting for collection testSimple1
null
Live Nodes: [127.0.0.1:32889_solr, 127.0.0.1:37983_solr]
Last available state: 
DocCollection(testSimple1//collections/testSimple1/state.json/10)={
  "pullReplicas":"0",
  "replicationFactor":"2",
  "shards":{
    "shard1":{
      "range":"80000000-ffffffff",
      "state":"active",
      "replicas":{
        "core_node3":{
          
"dataDir":"hdfs://localhost.localdomain:39667/data/testSimple1/core_node3/data/",
          "base_url":"https://127.0.0.1:35131/solr";,
          "node_name":"127.0.0.1:35131_solr",
          "type":"NRT",
          
"ulogDir":"hdfs://localhost.localdomain:39667/data/testSimple1/core_node3/data/tlog",
          "core":"testSimple1_shard1_replica_n1",
          "shared_storage":"true",
          "state":"down"},
        "core_node5":{
          
"dataDir":"hdfs://localhost.localdomain:39667/data/testSimple1/core_node5/data/",
          "base_url":"https://127.0.0.1:37983/solr";,
          "node_name":"127.0.0.1:37983_solr",
          "type":"NRT",
          
"ulogDir":"hdfs://localhost.localdomain:39667/data/testSimple1/core_node5/data/tlog",
          "core":"testSimple1_shard1_replica_n2",
          "shared_storage":"true",
          "state":"active",
          "leader":"true"}}},
    "shard2":{
      "range":"0-7fffffff",
      "state":"active",
      "replicas":{
        "core_node7":{
          
"dataDir":"hdfs://localhost.localdomain:39667/data/testSimple1/core_node7/data/",
          "base_url":"https://127.0.0.1:35131/solr";,
          "node_name":"127.0.0.1:35131_solr",
          "type":"NRT",
          
"ulogDir":"hdfs://localhost.localdomain:39667/data/testSimple1/core_node7/data/tlog",
          "core":"testSimple1_shard2_replica_n4",
          "shared_storage":"true",
          "state":"down"},
        "core_node8":{
          
"dataDir":"hdfs://localhost.localdomain:39667/data/testSimple1/core_node8/data/",
          "base_url":"https://127.0.0.1:37983/solr";,
          "node_name":"127.0.0.1:37983_solr",
          "type":"NRT",
          
"ulogDir":"hdfs://localhost.localdomain:39667/data/testSimple1/core_node8/data/tlog",
          "core":"testSimple1_shard2_replica_n6",
          "shared_storage":"true",
          "state":"active",
          "leader":"true"}}}},
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"2",
  "autoAddReplicas":"true",
  "nrtReplicas":"2",
  "tlogReplicas":"0"}
        at 
__randomizedtesting.SeedInfo.seed([33D4E2619A79FBD3:B67C69FBD8A2F02]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at 
org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:269)
        at 
org.apache.solr.cloud.autoscaling.AutoAddReplicasIntegrationTest.testSimple(AutoAddReplicasIntegrationTest.java:94)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  
org.apache.solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest.testSimple

Error Message:
Waiting for collection testSimple1 null Live Nodes: [127.0.0.1:32955_solr, 
127.0.0.1:41077_solr] Last available state: 
DocCollection(testSimple1//collections/testSimple1/state.json/10)={   
"pullReplicas":"0",   "replicationFactor":"2",   "shards":{     "shard1":{      
 "range":"80000000-ffffffff",       "state":"active",       "replicas":{        
 "core_node3":{           
"dataDir":"hdfs://localhost.localdomain:42757/data/testSimple1/core_node3/data/",
           "base_url":"https://127.0.0.1:46725/solr";,           
"node_name":"127.0.0.1:46725_solr",           "type":"NRT",           
"ulogDir":"hdfs://localhost.localdomain:42757/data/testSimple1/core_node3/data/tlog",
           "core":"testSimple1_shard1_replica_n1",           
"shared_storage":"true",           "state":"down"},         "core_node5":{      
     
"dataDir":"hdfs://localhost.localdomain:42757/data/testSimple1/core_node5/data/",
           "base_url":"https://127.0.0.1:41077/solr";,           
"node_name":"127.0.0.1:41077_solr",           "type":"NRT",           
"ulogDir":"hdfs://localhost.localdomain:42757/data/testSimple1/core_node5/data/tlog",
           "core":"testSimple1_shard1_replica_n2",           
"shared_storage":"true",           "state":"active",           
"leader":"true"}}},     "shard2":{       "range":"0-7fffffff",       
"state":"active",       "replicas":{         "core_node7":{           
"dataDir":"hdfs://localhost.localdomain:42757/data/testSimple1/core_node7/data/",
           "base_url":"https://127.0.0.1:46725/solr";,           
"node_name":"127.0.0.1:46725_solr",           "type":"NRT",           
"ulogDir":"hdfs://localhost.localdomain:42757/data/testSimple1/core_node7/data/tlog",
           "core":"testSimple1_shard2_replica_n4",           
"shared_storage":"true",           "state":"down"},         "core_node8":{      
     
"dataDir":"hdfs://localhost.localdomain:42757/data/testSimple1/core_node8/data/",
           "base_url":"https://127.0.0.1:41077/solr";,           
"node_name":"127.0.0.1:41077_solr",           "type":"NRT",           
"ulogDir":"hdfs://localhost.localdomain:42757/data/testSimple1/core_node8/data/tlog",
           "core":"testSimple1_shard2_replica_n6",           
"shared_storage":"true",           "state":"active",           
"leader":"true"}}}},   "router":{"name":"compositeId"},   
"maxShardsPerNode":"2",   "autoAddReplicas":"true",   "nrtReplicas":"2",   
"tlogReplicas":"0"}

Stack Trace:
java.lang.AssertionError: Waiting for collection testSimple1
null
Live Nodes: [127.0.0.1:32955_solr, 127.0.0.1:41077_solr]
Last available state: 
DocCollection(testSimple1//collections/testSimple1/state.json/10)={
  "pullReplicas":"0",
  "replicationFactor":"2",
  "shards":{
    "shard1":{
      "range":"80000000-ffffffff",
      "state":"active",
      "replicas":{
        "core_node3":{
          
"dataDir":"hdfs://localhost.localdomain:42757/data/testSimple1/core_node3/data/",
          "base_url":"https://127.0.0.1:46725/solr";,
          "node_name":"127.0.0.1:46725_solr",
          "type":"NRT",
          
"ulogDir":"hdfs://localhost.localdomain:42757/data/testSimple1/core_node3/data/tlog",
          "core":"testSimple1_shard1_replica_n1",
          "shared_storage":"true",
          "state":"down"},
        "core_node5":{
          
"dataDir":"hdfs://localhost.localdomain:42757/data/testSimple1/core_node5/data/",
          "base_url":"https://127.0.0.1:41077/solr";,
          "node_name":"127.0.0.1:41077_solr",
          "type":"NRT",
          
"ulogDir":"hdfs://localhost.localdomain:42757/data/testSimple1/core_node5/data/tlog",
          "core":"testSimple1_shard1_replica_n2",
          "shared_storage":"true",
          "state":"active",
          "leader":"true"}}},
    "shard2":{
      "range":"0-7fffffff",
      "state":"active",
      "replicas":{
        "core_node7":{
          
"dataDir":"hdfs://localhost.localdomain:42757/data/testSimple1/core_node7/data/",
          "base_url":"https://127.0.0.1:46725/solr";,
          "node_name":"127.0.0.1:46725_solr",
          "type":"NRT",
          
"ulogDir":"hdfs://localhost.localdomain:42757/data/testSimple1/core_node7/data/tlog",
          "core":"testSimple1_shard2_replica_n4",
          "shared_storage":"true",
          "state":"down"},
        "core_node8":{
          
"dataDir":"hdfs://localhost.localdomain:42757/data/testSimple1/core_node8/data/",
          "base_url":"https://127.0.0.1:41077/solr";,
          "node_name":"127.0.0.1:41077_solr",
          "type":"NRT",
          
"ulogDir":"hdfs://localhost.localdomain:42757/data/testSimple1/core_node8/data/tlog",
          "core":"testSimple1_shard2_replica_n6",
          "shared_storage":"true",
          "state":"active",
          "leader":"true"}}}},
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"2",
  "autoAddReplicas":"true",
  "nrtReplicas":"2",
  "tlogReplicas":"0"}
        at 
__randomizedtesting.SeedInfo.seed([33D4E2619A79FBD3:B67C69FBD8A2F02]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at 
org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:269)
        at 
org.apache.solr.cloud.autoscaling.AutoAddReplicasIntegrationTest.testSimple(AutoAddReplicasIntegrationTest.java:94)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  
org.apache.solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest.testSimple

Error Message:
Waiting for collection testSimple1 null Live Nodes: [127.0.0.1:33295_solr, 
127.0.0.1:36237_solr] Last available state: 
DocCollection(testSimple1//collections/testSimple1/state.json/12)={   
"pullReplicas":"0",   "replicationFactor":"2",   "shards":{     "shard1":{      
 "range":"80000000-ffffffff",       "state":"active",       "replicas":{        
 "core_node3":{           
"dataDir":"hdfs://localhost.localdomain:33331/data/testSimple1/core_node3/data/",
           "base_url":"https://127.0.0.1:35139/solr";,           
"node_name":"127.0.0.1:35139_solr",           "type":"NRT",           
"ulogDir":"hdfs://localhost.localdomain:33331/data/testSimple1/core_node3/data/tlog",
           "core":"testSimple1_shard1_replica_n1",           
"shared_storage":"true",           "state":"down"},         "core_node5":{      
     
"dataDir":"hdfs://localhost.localdomain:33331/data/testSimple1/core_node5/data/",
           "

[...truncated too long message...]

ag=null
   [junit4]   2> 1063661 INFO  
(TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[33D4E2619A79FBD3])
 [    ] o.a.s.m.r.SolrJmxReporter Closing reporter 
[org.apache.solr.metrics.reporters.SolrJmxReporter@773d9e0: rootName = null, 
domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / 
com.codahale.metrics.MetricRegistry@67c8c4ce
   [junit4]   2> 1063662 INFO  
(TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[33D4E2619A79FBD3])
 [    ] o.a.s.m.SolrMetricManager Closing metric reporters for 
registry=solr.jetty, tag=null
   [junit4]   2> 1063662 INFO  
(TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[33D4E2619A79FBD3])
 [    ] o.a.s.m.r.SolrJmxReporter Closing reporter 
[org.apache.solr.metrics.reporters.SolrJmxReporter@6e644e1b: rootName = null, 
domain = solr.jetty, service url = null, agent id = null] for registry 
solr.jetty / com.codahale.metrics.MetricRegistry@34f6300e
   [junit4]   2> 1063663 INFO  (coreCloseExecutor-1120-thread-1) [    
x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore 
org.apache.solr.core.SolrCore@4aeb5abd
   [junit4]   2> 1063663 INFO  (coreCloseExecutor-1120-thread-1) [    
x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for 
registry=solr.core.collection1, tag=1256938173
   [junit4]   2> 1063663 INFO  (coreCloseExecutor-1120-thread-1) [    
x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter 
[org.apache.solr.metrics.reporters.SolrJmxReporter@4200aeaa: rootName = null, 
domain = solr.core.collection1, service url = null, agent id = null] for 
registry solr.core.collection1 / com.codahale.metrics.MetricRegistry@3d06b9cf
   [junit4]   2> 1063667 INFO  
(TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[33D4E2619A79FBD3])
 [    ] o.e.j.s.h.ContextHandler Stopped 
o.e.j.s.ServletContextHandler@7d6ea73{/solr,null,UNAVAILABLE}
   [junit4]   2> 1063667 INFO  
(TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[33D4E2619A79FBD3])
 [    ] o.e.j.s.session Stopped scavenging
   [junit4]   2> 1063668 INFO  
(TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[33D4E2619A79FBD3])
 [    ] o.e.j.s.AbstractConnector Stopped 
ServerConnector@35c46e5d{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 1063668 INFO  
(TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[33D4E2619A79FBD3])
 [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=181115651
   [junit4]   2> 1063668 INFO  
(TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[33D4E2619A79FBD3])
 [    ] o.a.s.m.SolrMetricManager Closing metric reporters for 
registry=solr.node, tag=null
   [junit4]   2> 1063668 INFO  
(TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[33D4E2619A79FBD3])
 [    ] o.a.s.m.r.SolrJmxReporter Closing reporter 
[org.apache.solr.metrics.reporters.SolrJmxReporter@5f7e3e05: rootName = null, 
domain = solr.node, service url = null, agent id = null] for registry solr.node 
/ com.codahale.metrics.MetricRegistry@8356f7e
   [junit4]   2> 1063672 INFO  
(TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[33D4E2619A79FBD3])
 [    ] o.a.s.m.SolrMetricManager Closing metric reporters for 
registry=solr.jvm, tag=null
   [junit4]   2> 1063672 INFO  
(TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[33D4E2619A79FBD3])
 [    ] o.a.s.m.r.SolrJmxReporter Closing reporter 
[org.apache.solr.metrics.reporters.SolrJmxReporter@36822f38: rootName = null, 
domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / 
com.codahale.metrics.MetricRegistry@67c8c4ce
   [junit4]   2> 1063675 INFO  
(TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[33D4E2619A79FBD3])
 [    ] o.a.s.m.SolrMetricManager Closing metric reporters for 
registry=solr.jetty, tag=null
   [junit4]   2> 1063675 INFO  
(TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[33D4E2619A79FBD3])
 [    ] o.a.s.m.r.SolrJmxReporter Closing reporter 
[org.apache.solr.metrics.reporters.SolrJmxReporter@6c5a0271: rootName = null, 
domain = solr.jetty, service url = null, agent id = null] for registry 
solr.jetty / com.codahale.metrics.MetricRegistry@34f6300e
   [junit4]   2> 1063675 INFO  (coreCloseExecutor-1122-thread-1) [    
x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore 
org.apache.solr.core.SolrCore@32dd7ba0
   [junit4]   2> 1063675 INFO  (coreCloseExecutor-1122-thread-1) [    
x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for 
registry=solr.core.collection1, tag=853375904
   [junit4]   2> 1063675 INFO  (coreCloseExecutor-1122-thread-1) [    
x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter 
[org.apache.solr.metrics.reporters.SolrJmxReporter@73b9305c: rootName = null, 
domain = solr.core.collection1, service url = null, agent id = null] for 
registry solr.core.collection1 / com.codahale.metrics.MetricRegistry@553e717c
   [junit4]   2> 1063685 INFO  
(TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[33D4E2619A79FBD3])
 [    ] o.e.j.s.h.ContextHandler Stopped 
o.e.j.s.ServletContextHandler@57545748{/solr,null,UNAVAILABLE}
   [junit4]   2> 1063685 INFO  
(TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[33D4E2619A79FBD3])
 [    ] o.e.j.s.session Stopped scavenging
   [junit4] OK      0.66s J1 | 
TestReplicationHandler.doTestReplicateAfterStartup
   [junit4]   2> NOTE: leaving temporary files on disk at: 
/home/jenkins/workspace/Lucene-Solr-BadApples-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_33D4E2619A79FBD3-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): 
{name=PostingsFormat(name=MockRandom), id=PostingsFormat(name=MockRandom), 
newname=PostingsFormat(name=MockRandom)}, docValues:{}, 
maxPointsInLeafNode=1861, maxMBSortInHeap=6.192642069065338, 
sim=RandomSimilarity(queryNorm=true): {}, locale=fr-CA, timezone=Africa/Khartoum
   [junit4]   2> NOTE: Linux 4.13.0-36-generic amd64/Oracle Corporation 
1.8.0_162 (64-bit)/cpus=8,threads=1,free=252727192,total=528482304
   [junit4]   2> NOTE: All tests run in this JVM: [TestJmxIntegration, 
HdfsAutoAddReplicasIntegrationTest, AutoscalingHistoryHandlerTest, 
AutoscalingHistoryHandlerTest, AutoAddReplicasIntegrationTest, 
AutoAddReplicasIntegrationTest, TestReplicationHandler]
   [junit4] Completed [25/25 (25!)] on J1 in 72.52s, 16 tests, 1 error <<< 
FAILURES!
   [junit4] 
   [junit4] 
   [junit4] Tests with failures [seed: 33D4E2619A79FBD3] (first 10 out of 25):
   [junit4]   - org.apache.solr.core.TestJmxIntegration.testJmxOnCoreReload
   [junit4]   - org.apache.solr.core.TestJmxIntegration.testJmxOnCoreReload
   [junit4]   - org.apache.solr.core.TestJmxIntegration.testJmxOnCoreReload
   [junit4]   - 
org.apache.solr.handler.TestReplicationHandler.doTestIndexFetchOnMasterRestart
   [junit4]   - 
org.apache.solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest.testSimple
   [junit4]   - 
org.apache.solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest.testSimple
   [junit4]   - 
org.apache.solr.cloud.autoscaling.AutoAddReplicasIntegrationTest.testSimple
   [junit4]   - 
org.apache.solr.handler.TestReplicationHandler.doTestIndexFetchOnMasterRestart
   [junit4]   - 
org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest.testHistory
   [junit4]   - 
org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest.testHistory
   [junit4] 
   [junit4] 
   [junit4] JVM J0:     0.40 ..  1046.86 =  1046.46s
   [junit4] JVM J1:     0.39 ..  1065.84 =  1065.45s
   [junit4] JVM J2:     0.39 ..  1040.29 =  1039.90s
   [junit4] Execution time total: 17 minutes 45 seconds
   [junit4] Tests summary: 25 suites, 110 tests, 5 errors, 20 failures

BUILD FAILED
/home/jenkins/workspace/Lucene-Solr-BadApples-7.x-Linux/lucene/common-build.xml:1561:
 The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-BadApples-7.x-Linux/lucene/common-build.xml:1085:
 There were test failures: 25 suites, 110 tests, 5 errors, 20 failures [seed: 
33D4E2619A79FBD3]

Total time: 17 minutes 47 seconds

[repro] Setting last failure code to 256

[repro] Failures:
[repro]   5/5 failed: 
org.apache.solr.cloud.autoscaling.AutoAddReplicasIntegrationTest
[repro]   5/5 failed: 
org.apache.solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest
[repro]   5/5 failed: org.apache.solr.core.TestJmxIntegration
[repro]   5/5 failed: org.apache.solr.handler.TestReplicationHandler
[repro]   5/5 failed: 
org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest
[repro]   5/5 failed: org.apache.solr.ltr.TestLTRReRankingPipeline
[repro] Exiting with code 256
+ mv lucene/build lucene/build.repro
+ mv solr/build solr/build.repro
+ mv lucene/build.orig lucene/build
+ mv solr/build.orig solr/build
Archiving artifacts
Setting 
ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
[WARNINGS] Parsing warnings in console log with parser Java Compiler (javac)
Setting 
ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting 
ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
<Git Blamer> Using GitBlamer to create author and commit information for all 
warnings.
<Git Blamer> GIT_COMMIT=d77f618a672d719b971684c55b9fa07d0ba59aaf, 
workspace=/var/lib/jenkins/workspace/Lucene-Solr-BadApples-7.x-Linux
[WARNINGS] Computing warning deltas based on reference build #6
Recording test results
Setting 
ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
Setting 
ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting 
ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting 
ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting 
ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org