Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/26986/
Java: 64bit/jdk-13.0.2 -XX:-UseCompressedOops -XX:+UseParallelGC

1 tests failed.
FAILED:  org.apache.solr.cloud.autoscaling.sim.TestSimScenario.testSplitShard

Error Message:
java.util.concurrent.TimeoutException: OverseerTriggerThread never caught up to 
the latest znodeVersion

Stack Trace:
java.lang.AssertionError: java.util.concurrent.TimeoutException: 
OverseerTriggerThread never caught up to the latest znodeVersion
        at 
__randomizedtesting.SeedInfo.seed([AD9AE3BE82781274:76904ED29C8D2ECB]:0)
        at org.junit.Assert.fail(Assert.java:88)
        at org.junit.Assert.assertTrue(Assert.java:41)
        at 
org.apache.solr.cloud.autoscaling.sim.TestSimScenario.testSplitShard(TestSimScenario.java:160)
        at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:567)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1754)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:942)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:978)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:992)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:819)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:470)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:951)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:836)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:887)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:898)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:826)
        at java.base/java.lang.Thread.run(Thread.java:830)




Build Log:
[...truncated 14188 lines...]
   [junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestSimScenario
   [junit4]   2> 846718 INFO  
(SUITE-TestSimScenario-seed#[AD9AE3BE82781274]-worker) [     ] 
o.a.s.SolrTestCase Setting 'solr.default.confdir' system property to 
test-framework derived value of 
'/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/server/solr/configsets/_default/conf'
   [junit4]   2> 846718 INFO  
(SUITE-TestSimScenario-seed#[AD9AE3BE82781274]-worker) [     ] 
o.a.s.SolrTestCaseJ4 Created dataDir: 
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.sim.TestSimScenario_AD9AE3BE82781274-001/data-dir-109-001
   [junit4]   2> 846719 INFO  
(SUITE-TestSimScenario-seed#[AD9AE3BE82781274]-worker) [     ] 
o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) 
w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 846719 INFO  
(SUITE-TestSimScenario-seed#[AD9AE3BE82781274]-worker) [     ] 
o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: 
@org.apache.solr.util.RandomizeSSL(reason="", ssl=0.0/0.0, value=0.0/0.0, 
clientAuth=0.0/0.0)
   [junit4]   2> 846719 INFO  
(SUITE-TestSimScenario-seed#[AD9AE3BE82781274]-worker) [     ] 
o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: 
test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 846721 INFO  
(TEST-TestSimScenario.testAutoAddReplicas-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.SolrTestCaseJ4 ###Starting testAutoAddReplicas
   [junit4]   2> 846725 INFO  
(TEST-TestSimScenario.testAutoAddReplicas-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario 1.     CreateCluster   numNodes=2
   [junit4]   2> 846726 INFO  
(TEST-TestSimScenario.testAutoAddReplicas-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario                CreateCluster   numNodes=2
   [junit4]   2> 846727 DEBUG 
(TEST-TestSimScenario.testAutoAddReplicas-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 
127.0.0.1:10011_solr
   [junit4]   2> 846727 INFO  
(TEST-TestSimScenario.testAutoAddReplicas-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history 
in memory.
   [junit4]   2> 846727 INFO  
(TEST-TestSimScenario.testAutoAddReplicas-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario 2.     LoadAutoscaling 
json={'cluster-policy'+:+[{'replica'+:+'<3',+'shard'+:+'#EACH',+'collection'+:+'testCollection','node':'#ANY'}]}&defaultWaitFor=10
   [junit4]   2> 846727 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
 ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and 
.scheduled_maintenance triggers
   [junit4]   2> 846727 INFO  
(TEST-TestSimScenario.testAutoAddReplicas-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario                LoadAutoscaling 
json={'cluster-policy'+:+[{'replica'+:+'<3',+'shard'+:+'#EACH',+'collection'+:+'testCollection','node':'#ANY'}]}&defaultWaitFor=10
   [junit4]   2> 846727 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
 ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode 
version 1
   [junit4]   2> 846727 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
 ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 0
   [junit4]   2> 846727 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
 ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
   [junit4]   2> 846727 DEBUG 
(TEST-TestSimScenario.testAutoAddReplicas-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 
2
   [junit4]   2> 846728 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
 ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial 
livenodes: [127.0.0.1:10012_solr, 127.0.0.1:10011_solr]
   [junit4]   2> 846728 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
 ] o.a.s.c.a.OverseerTriggerThread -- deactivating old nodeLost / nodeAdded 
markers
   [junit4]   2> 846728 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
 ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 1
   [junit4]   2> 846728 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
 ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
   [junit4]   2> 846728 DEBUG (ScheduledTrigger-14104-thread-2) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 846728 INFO  
(TEST-TestSimScenario.testAutoAddReplicas-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario 3.     RunSolrRequest  
path=/admin/collections&action=CREATE&autoAddReplicas=true&name=testCollection&numShards=2&replicationFactor=2&maxShardsPerNode=2
   [junit4]   2> 846728 INFO  
(TEST-TestSimScenario.testAutoAddReplicas-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario                RunSolrRequest  
path=/admin/collections&action=CREATE&autoAddReplicas=true&name=testCollection&numShards=2&replicationFactor=2&maxShardsPerNode=2
   [junit4]   2> 846728 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
 ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial 
livenodes: [127.0.0.1:10012_solr, 127.0.0.1:10011_solr]
   [junit4]   2> 846728 DEBUG (ScheduledTrigger-14104-thread-1) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 846728 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
 ] o.a.s.c.a.OverseerTriggerThread -- deactivating old nodeLost / nodeAdded 
markers
   [junit4]   2> 846728 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
 ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
   [junit4]   2> 846729 DEBUG (simCloudManagerPool-14103-thread-1) [     ] 
o.a.s.c.a.s.SimClusterStateProvider -- simCreateCollection testCollection
   [junit4]   2> 846748 DEBUG (ScheduledTrigger-14104-thread-3) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 846757 DEBUG (simCloudManagerPool-14103-thread-2) [     ] 
o.a.s.c.a.s.SimClusterStateProvider Running leader election (testCollection / 
shard1)
   [junit4]   2> 846758 DEBUG (simCloudManagerPool-14103-thread-2) [     ] 
o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testCollection / 
shard1 (currentVersion=2): {"core_node1":{
   [junit4]   2>     "core":"testCollection_shard1_replica_n1",
   [junit4]   2>     "shard":"shard1",
   [junit4]   2>     "collection":"testCollection",
   [junit4]   2>     "node_name":"127.0.0.1:10011_solr",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "state":"active",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>     "SEARCHER.searcher.numDocs":0}}
   [junit4]   2> 846768 DEBUG (ScheduledTrigger-14104-thread-1) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 846769 DEBUG (simCloudManagerPool-14103-thread-4) [     ] 
o.a.s.c.a.s.SimClusterStateProvider Running leader election (testCollection / 
shard2)
   [junit4]   2> 846769 DEBUG (simCloudManagerPool-14103-thread-4) [     ] 
o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testCollection / 
shard2 (currentVersion=3): {"core_node3":{
   [junit4]   2>     "core":"testCollection_shard2_replica_n3",
   [junit4]   2>     "shard":"shard2",
   [junit4]   2>     "collection":"testCollection",
   [junit4]   2>     "node_name":"127.0.0.1:10012_solr",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "state":"active",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>     "SEARCHER.searcher.numDocs":0}}
   [junit4]   2> 846769 DEBUG (simCloudManagerPool-14103-thread-1) [     ] 
o.a.s.c.a.s.SimClusterStateProvider -- finished createCollection testCollection
   [junit4]   2> 846769 INFO  
(TEST-TestSimScenario.testAutoAddReplicas-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario 4.     WaitCollection  
collection=testCollection&shards=2&replicas=2
   [junit4]   2> 846769 INFO  
(TEST-TestSimScenario.testAutoAddReplicas-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario                WaitCollection  
collection=testCollection&shards=2&replicas=2
   [junit4]   2> 846789 DEBUG (ScheduledTrigger-14104-thread-1) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 846790 INFO  
(TEST-TestSimScenario.testAutoAddReplicas-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario 5.     SetEventListener        
trigger=.auto_add_replicas&stage=SUCCEEDED
   [junit4]   2> 846790 INFO  
(TEST-TestSimScenario.testAutoAddReplicas-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario                SetEventListener        
trigger=.auto_add_replicas&stage=SUCCEEDED
   [junit4]   2> 846791 INFO  
(TEST-TestSimScenario.testAutoAddReplicas-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario 6.     KillNodes       node=${_random_node_}
   [junit4]   2> 846791 INFO  
(TEST-TestSimScenario.testAutoAddReplicas-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario                KillNodes       node=127.0.0.1:10011_solr
   [junit4]   2> 846791 DEBUG 
(TEST-TestSimScenario.testAutoAddReplicas-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimClusterStateProvider  -- created marker: 
/autoscaling/nodeLost/127.0.0.1:10011_solr
   [junit4]   2> 846791 DEBUG 
(TEST-TestSimScenario.testAutoAddReplicas-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 
127.0.0.1:10012_solr
   [junit4]   2> 846792 INFO  
(TEST-TestSimScenario.testAutoAddReplicas-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario 7.     WaitEvent       
trigger=.auto_add_replicas&wait=60
   [junit4]   2> 846792 INFO  
(TEST-TestSimScenario.testAutoAddReplicas-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario                WaitEvent       
trigger=.auto_add_replicas&wait=60
   [junit4]   2> 846792 DEBUG (simCloudManagerPool-14103-thread-7) [     ] 
o.a.s.c.a.s.SimClusterStateProvider Running leader election (testCollection / 
shard1)
   [junit4]   2> 846792 INFO  (simCloudManagerPool-14103-thread-7) [     ] 
o.a.s.c.ActionThrottle Throttling leader attempts - waiting for 3274ms
   [junit4]   2> 846809 DEBUG (ScheduledTrigger-14104-thread-4) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 1 and last live nodes: 2
   [junit4]   2> 846809 DEBUG (ScheduledTrigger-14104-thread-4) [     ] 
o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10011_solr
   [junit4]   2> 846829 DEBUG (ScheduledTrigger-14104-thread-1) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 1 and last live nodes: 1
   [junit4]   2> 846849 DEBUG (ScheduledTrigger-14104-thread-4) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 1 and last live nodes: 1
   [junit4]   2> 846857 DEBUG (simCloudManagerPool-14103-thread-7) [     ] 
o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testCollection / 
shard1 (currentVersion=8): {"core_node2":{
   [junit4]   2>     "core":"testCollection_shard1_replica_n2",
   [junit4]   2>     "shard":"shard1",
   [junit4]   2>     "collection":"testCollection",
   [junit4]   2>     "node_name":"127.0.0.1:10012_solr",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "state":"active",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>     "SEARCHER.searcher.numDocs":0}}
   [junit4]   2> 846869 DEBUG (ScheduledTrigger-14104-thread-1) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 1 and last live nodes: 1
   [junit4]   2> 846889 DEBUG (ScheduledTrigger-14104-thread-4) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 1 and last live nodes: 1
   [junit4]   2> 846909 DEBUG (ScheduledTrigger-14104-thread-1) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 1 and last live nodes: 1
   [junit4]   2> 846929 DEBUG (ScheduledTrigger-14104-thread-4) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 1 and last live nodes: 1
   [junit4]   2> 846949 DEBUG (ScheduledTrigger-14104-thread-1) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 1 and last live nodes: 1
   [junit4]   2> 846969 DEBUG (ScheduledTrigger-14104-thread-4) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 1 and last live nodes: 1
   [junit4]   2> 846990 DEBUG (ScheduledTrigger-14104-thread-1) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 1 and last live nodes: 1
   [junit4]   2> 847010 DEBUG (ScheduledTrigger-14104-thread-4) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 1 and last live nodes: 1
   [junit4]   2> 847010 DEBUG (ScheduledTrigger-14104-thread-4) [     ] 
o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost 
nodes: [127.0.0.1:10011_solr]
   [junit4]   2> 847010 DEBUG (ScheduledTrigger-14104-thread-4) [     ] 
o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"f30d951ae1aaTqroj7gwhrtovdh3jejvw9hb9",
   [junit4]   2>   "source":".auto_add_replicas",
   [junit4]   2>   "eventTime":267239661691306,
   [junit4]   2>   "eventType":"NODELOST",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[267239661691306],
   [junit4]   2>     "preferredOperation":"movereplica",
   [junit4]   2>     "nodeNames":["127.0.0.1:10011_solr"]}}
   [junit4]   2> 847010 DEBUG (ScheduledTrigger-14104-thread-4) [     ] 
o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, 
.scheduled_maintenance]
   [junit4]   2> 847011 DEBUG (ScheduledTrigger-14104-thread-4) [     ] 
o.a.s.c.a.SystemLogListener Collection .system missing, skip sending event {
   [junit4]   2>   "id":"f30d951ae1aaTqroj7gwhrtovdh3jejvw9hb9",
   [junit4]   2>   "source":".auto_add_replicas",
   [junit4]   2>   "eventTime":267239661691306,
   [junit4]   2>   "eventType":"NODELOST",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[267239661691306],
   [junit4]   2>     "preferredOperation":"movereplica",
   [junit4]   2>     "_enqueue_time_":267249744441006,
   [junit4]   2>     "nodeNames":["127.0.0.1:10011_solr"]}}
   [junit4]   2> 847011 DEBUG (AutoscalingActionExecutor-14105-thread-1) [     
] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"f30d951ae1aaTqroj7gwhrtovdh3jejvw9hb9",
   [junit4]   2>   "source":".auto_add_replicas",
   [junit4]   2>   "eventTime":267239661691306,
   [junit4]   2>   "eventType":"NODELOST",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[267239661691306],
   [junit4]   2>     "preferredOperation":"movereplica",
   [junit4]   2>     "_enqueue_time_":267249744441006,
   [junit4]   2>     "nodeNames":["127.0.0.1:10011_solr"]}}
   [junit4]   2> 847011 DEBUG (AutoscalingActionExecutor-14105-thread-1) [     
] o.a.s.c.a.ComputePlanAction -- processing event: {
   [junit4]   2>   "id":"f30d951ae1aaTqroj7gwhrtovdh3jejvw9hb9",
   [junit4]   2>   "source":".auto_add_replicas",
   [junit4]   2>   "eventTime":267239661691306,
   [junit4]   2>   "eventType":"NODELOST",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[267239661691306],
   [junit4]   2>     "preferredOperation":"movereplica",
   [junit4]   2>     "_enqueue_time_":267249744441006,
   [junit4]   2>     "nodeNames":["127.0.0.1:10011_solr"]}} with context 
properties: {BEFORE_ACTION=[auto_add_replicas_plan]}
   [junit4]   2> 847012 DEBUG (AutoscalingActionExecutor-14105-thread-1) [     
] o.a.s.c.a.ComputePlanAction Computed Plan: 
action=MOVEREPLICA&collection=testCollection&targetNode=127.0.0.1:10012_solr&inPlaceMove=true&replica=core_node1
   [junit4]   2> 847012 DEBUG (AutoscalingActionExecutor-14105-thread-1) [     
] o.a.s.c.a.ComputePlanAction Computed Plan: 
action=MOVEREPLICA&collection=testCollection&targetNode=127.0.0.1:10012_solr&inPlaceMove=true&replica=core_node4
   [junit4]   2> 847012 DEBUG (AutoscalingActionExecutor-14105-thread-1) [     
] o.a.s.c.a.ExecutePlanAction -- processing event: {
   [junit4]   2>   "id":"f30d951ae1aaTqroj7gwhrtovdh3jejvw9hb9",
   [junit4]   2>   "source":".auto_add_replicas",
   [junit4]   2>   "eventTime":267239661691306,
   [junit4]   2>   "eventType":"NODELOST",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[267239661691306],
   [junit4]   2>     "preferredOperation":"movereplica",
   [junit4]   2>     "_enqueue_time_":267249744441006,
   [junit4]   2>     "nodeNames":["127.0.0.1:10011_solr"]}} with context 
properties: {operations=[{
   [junit4]   2>   
"class":"org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica",
   [junit4]   2>   "method":"GET",
   [junit4]   2>   "params.action":"MOVEREPLICA",
   [junit4]   2>   "params.collection":"testCollection",
   [junit4]   2>   "params.targetNode":"127.0.0.1:10012_solr",
   [junit4]   2>   "params.inPlaceMove":"true",
   [junit4]   2>   "params.replica":"core_node1"}, {
   [junit4]   2>   
"class":"org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica",
   [junit4]   2>   "method":"GET",
   [junit4]   2>   "params.action":"MOVEREPLICA",
   [junit4]   2>   "params.collection":"testCollection",
   [junit4]   2>   "params.targetNode":"127.0.0.1:10012_solr",
   [junit4]   2>   "params.inPlaceMove":"true",
   [junit4]   2>   "params.replica":"core_node4"}], 
AFTER_ACTION=[auto_add_replicas_plan], BEFORE_ACTION=[auto_add_replicas_plan, 
execute_plan]}
   [junit4]   2> 847012 DEBUG (AutoscalingActionExecutor-14105-thread-1) [     
] o.a.s.c.a.ExecutePlanAction Executing operation: 
action=MOVEREPLICA&collection=testCollection&targetNode=127.0.0.1:10012_solr&inPlaceMove=true&replica=core_node1
   [junit4]   2> 847113 DEBUG (simCloudManagerPool-14103-thread-8) [     ] 
o.a.s.c.a.s.SimClusterStateProvider -- new replica: {"core_node6":{
   [junit4]   2>     "core":"testCollection_shard1_replica_n5",
   [junit4]   2>     "shard":"shard1",
   [junit4]   2>     "collection":"testCollection",
   [junit4]   2>     "node_name":"127.0.0.1:10012_solr",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "state":"down",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>     "SEARCHER.searcher.numDocs":0}}
   [junit4]   2> 847225 DEBUG (AutoscalingActionExecutor-14105-thread-1) [     
] o.a.s.c.a.ExecutePlanAction Executing operation: 
action=MOVEREPLICA&collection=testCollection&targetNode=127.0.0.1:10012_solr&inPlaceMove=true&replica=core_node4
   [junit4]   2> 847326 DEBUG (simCloudManagerPool-14103-thread-11) [     ] 
o.a.s.c.a.s.SimClusterStateProvider -- new replica: {"core_node8":{
   [junit4]   2>     "core":"testCollection_shard2_replica_n7",
   [junit4]   2>     "shard":"shard2",
   [junit4]   2>     "collection":"testCollection",
   [junit4]   2>     "node_name":"127.0.0.1:10012_solr",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "state":"down",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>     "SEARCHER.searcher.numDocs":0}}
   [junit4]   2> 847437 DEBUG (AutoscalingActionExecutor-14105-thread-1) [     
] o.a.s.c.a.SystemLogListener Collection .system missing, skip sending event {
   [junit4]   2>   "id":"f30d951ae1aaTqroj7gwhrtovdh3jejvw9hb9",
   [junit4]   2>   "source":".auto_add_replicas",
   [junit4]   2>   "eventTime":267239661691306,
   [junit4]   2>   "eventType":"NODELOST",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[267239661691306],
   [junit4]   2>     "preferredOperation":"movereplica",
   [junit4]   2>     "_enqueue_time_":267249744441006,
   [junit4]   2>     "nodeNames":["127.0.0.1:10011_solr"]}}
   [junit4]   2> 847437 DEBUG (AutoscalingActionExecutor-14105-thread-1) [     
] o.a.s.c.a.ScheduledTriggers Resuming trigger: .scheduled_maintenance after 
100ms
   [junit4]   2> 847437 DEBUG (AutoscalingActionExecutor-14105-thread-1) [     
] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
   [junit4]   2> 847437 DEBUG (AutoscalingActionExecutor-14105-thread-1) [     
] o.a.s.c.a.ScheduledTriggers -- processing took 21305 ms for event 
id=f30d951ae1aaTqroj7gwhrtovdh3jejvw9hb9
   [junit4]   2> 847437 INFO  
(TEST-TestSimScenario.testAutoAddReplicas-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario 8.     WaitCollection  
collection=testCollection&shards=2&replicas=2
   [junit4]   2> 847437 INFO  
(TEST-TestSimScenario.testAutoAddReplicas-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario                WaitCollection  
collection=testCollection&shards=2&replicas=2
   [junit4]   2> 847437 INFO  
(TEST-TestSimScenario.testAutoAddReplicas-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario 9.     SaveSnapshot    path=${snapshotPath}
   [junit4]   2> 847437 INFO  
(TEST-TestSimScenario.testAutoAddReplicas-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario                SaveSnapshot    
path=/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.sim.TestSimScenario_AD9AE3BE82781274-001/tempDir-001/snapshot
   [junit4]   2> 847440 WARN  
(TEST-TestSimScenario.testAutoAddReplicas-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimUtils V2 action SET_CLUSTER_PROPERTY_OBJ has no equivalent V1 
action
   [junit4]   2> 847440 WARN  
(TEST-TestSimScenario.testAutoAddReplicas-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimUtils V2 action GET_NODES has no equivalent V1 action
   [junit4]   2> 847442 DEBUG 
(TEST-TestSimScenario.testAutoAddReplicas-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SnapshotDistribStateManager - copying 23 resources from 
SimDistribStateManager
   [junit4]   2> 847452 INFO  (Simulated OverseerAutoScalingTriggerThread) [    
 ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are 
closed, exiting.
   [junit4]   2> 847452 DEBUG 
(TEST-TestSimScenario.testAutoAddReplicas-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 847452 DEBUG 
(TEST-TestSimScenario.testAutoAddReplicas-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 847452 DEBUG 
(TEST-TestSimScenario.testAutoAddReplicas-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 847452 DEBUG 
(TEST-TestSimScenario.testAutoAddReplicas-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool 
executor
   [junit4]   2> 847452 DEBUG 
(TEST-TestSimScenario.testAutoAddReplicas-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 847452 DEBUG 
(TEST-TestSimScenario.testAutoAddReplicas-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 847454 DEBUG 
(TEST-TestSimScenario.testAutoAddReplicas-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SnapshotDistribStateManager - loaded snapshot of 23 resources
   [junit4]   2> 847455 INFO  
(TEST-TestSimScenario.testAutoAddReplicas-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.SolrTestCaseJ4 ###Ending testAutoAddReplicas
   [junit4]   2> 847456 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.SolrTestCaseJ4 ###Starting testSuggestions
   [junit4]   2> 847457 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario 1. CreateCluster   numNodes=2
   [junit4]   2> 847457 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario            CreateCluster   numNodes=2
   [junit4]   2> 847458 DEBUG 
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 
127.0.0.1:10013_solr
   [junit4]   2> 847459 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
 ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and 
.scheduled_maintenance triggers
   [junit4]   2> 847459 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history 
in memory.
   [junit4]   2> 847459 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario 2. LoadAutoscaling json={'cluster-policy':[]}
   [junit4]   2> 847459 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario            LoadAutoscaling json={'cluster-policy':[]}
   [junit4]   2> 847459 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
 ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode 
version 1
   [junit4]   2> 847459 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
 ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 0
   [junit4]   2> 847459 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
 ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
   [junit4]   2> 847459 DEBUG 
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 
2
   [junit4]   2> 847460 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
 ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial 
livenodes: [127.0.0.1:10013_solr, 127.0.0.1:10014_solr]
   [junit4]   2> 847460 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
 ] o.a.s.c.a.OverseerTriggerThread -- deactivating old nodeLost / nodeAdded 
markers
   [junit4]   2> 847460 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
 ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 1
   [junit4]   2> 847460 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
 ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
   [junit4]   2> 847460 DEBUG (ScheduledTrigger-14110-thread-2) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 847460 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
 ] o.a.s.c.a.OverseerTriggerThread -- deactivating old nodeLost / nodeAdded 
markers
   [junit4]   2> 847460 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
 ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
   [junit4]   2> 847465 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario 3. RunSolrRequest  
path=/admin/collections&action=CREATE&autoAddReplicas=true&name=testCollection&numShards=2&replicationFactor=2&maxShardsPerNode=2
   [junit4]   2> 847465 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario            RunSolrRequest  
path=/admin/collections&action=CREATE&autoAddReplicas=true&name=testCollection&numShards=2&replicationFactor=2&maxShardsPerNode=2
   [junit4]   2> 847465 DEBUG (simCloudManagerPool-14109-thread-1) [     ] 
o.a.s.c.a.s.SimClusterStateProvider -- simCreateCollection testCollection
   [junit4]   2> 847480 DEBUG (ScheduledTrigger-14110-thread-1) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 847486 DEBUG (simCloudManagerPool-14109-thread-2) [     ] 
o.a.s.c.a.s.SimClusterStateProvider Running leader election (testCollection / 
shard1)
   [junit4]   2> 847486 DEBUG (simCloudManagerPool-14109-thread-2) [     ] 
o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testCollection / 
shard1 (currentVersion=2): {"core_node1":{
   [junit4]   2>     "core":"testCollection_shard1_replica_n1",
   [junit4]   2>     "shard":"shard1",
   [junit4]   2>     "collection":"testCollection",
   [junit4]   2>     "node_name":"127.0.0.1:10014_solr",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "state":"active",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>     "SEARCHER.searcher.numDocs":0}}
   [junit4]   2> 847500 DEBUG (ScheduledTrigger-14110-thread-2) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 847507 DEBUG (simCloudManagerPool-14109-thread-4) [     ] 
o.a.s.c.a.s.SimClusterStateProvider Running leader election (testCollection / 
shard2)
   [junit4]   2> 847507 DEBUG (simCloudManagerPool-14109-thread-4) [     ] 
o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testCollection / 
shard2 (currentVersion=4): {"core_node3":{
   [junit4]   2>     "core":"testCollection_shard2_replica_n3",
   [junit4]   2>     "shard":"shard2",
   [junit4]   2>     "collection":"testCollection",
   [junit4]   2>     "node_name":"127.0.0.1:10014_solr",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "state":"active",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>     "SEARCHER.searcher.numDocs":0}}
   [junit4]   2> 847507 DEBUG (simCloudManagerPool-14109-thread-1) [     ] 
o.a.s.c.a.s.SimClusterStateProvider -- finished createCollection testCollection
   [junit4]   2> 847507 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario 4. WaitCollection  
collection=testCollection&shards=2&replicas=2
   [junit4]   2> 847507 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario            WaitCollection  
collection=testCollection&shards=2&replicas=2
   [junit4]   2> 847518 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario 5. CtxSet  key=myNode&value=${_random_node_}
   [junit4]   2> 847518 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario            CtxSet  key=myNode&value=127.0.0.1:10014_solr
   [junit4]   2> 847518 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario 6. RunSolrRequest  
path=/admin/collections&action=ADDREPLICA&collection=testCollection&shard=shard1&node=${myNode}
   [junit4]   2> 847518 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario            RunSolrRequest  
path=/admin/collections&action=ADDREPLICA&collection=testCollection&shard=shard1&node=127.0.0.1:10014_solr
   [junit4]   2> 847518 INFO  (simCloudManagerPool-14109-thread-6) [     ] 
o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10014_solr for creating new 
replica of shard shard1 for collection testCollection
   [junit4]   2> 847518 INFO  (simCloudManagerPool-14109-thread-6) [     ] 
o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command.
   [junit4]   2> 847520 DEBUG (ScheduledTrigger-14110-thread-1) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 847529 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario 7. RunSolrRequest  
path=/admin/collections&action=ADDREPLICA&collection=testCollection&shard=shard1&node=${myNode}
   [junit4]   2> 847529 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario            RunSolrRequest  
path=/admin/collections&action=ADDREPLICA&collection=testCollection&shard=shard1&node=127.0.0.1:10014_solr
   [junit4]   2> 847529 INFO  (simCloudManagerPool-14109-thread-7) [     ] 
o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10014_solr for creating new 
replica of shard shard1 for collection testCollection
   [junit4]   2> 847529 INFO  (simCloudManagerPool-14109-thread-7) [     ] 
o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command.
   [junit4]   2> 847540 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario 8. LoopOp  iterations=${iterative}
   [junit4]   2> 847540 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario            LoopOp  iterations=0
   [junit4]   2> 847540 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario 9. LoopOp  iterations=${justCalc}
   [junit4]   2> 847540 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario            LoopOp  iterations=1
   [junit4]   2> 847540 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario    * iter 1 :
   [junit4]   2> 847540 DEBUG (ScheduledTrigger-14110-thread-1) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 847540 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario      - CalculateSuggestions        )
   [junit4]   2> 847541 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario         - 1 suggestions
   [junit4]   2> 847541 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario      - SaveSnapshot        
path=/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.sim.TestSimScenario_AD9AE3BE82781274-001/tempDir-002/snapshot/0)
   [junit4]   2> 847542 DEBUG 
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SnapshotDistribStateManager - copying 22 resources from 
SimDistribStateManager
   [junit4]   2> 847544 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario 10.        Dump    redact=true
   [junit4]   2> 847544 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario            Dump    redact=true
   [junit4]   2> 847544 DEBUG 
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SnapshotDistribStateManager - copying 22 resources from 
SimDistribStateManager
   [junit4]   2> 847547 INFO  (Simulated OverseerAutoScalingTriggerThread) [    
 ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are 
closed, exiting.
   [junit4]   2> 847547 DEBUG 
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 847547 DEBUG 
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 847547 DEBUG 
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 847547 DEBUG 
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool 
executor
   [junit4]   2> 847547 DEBUG 
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 847547 DEBUG 
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 847547 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario 1. CreateCluster   numNodes=2
   [junit4]   2> 847547 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario            CreateCluster   numNodes=2
   [junit4]   2> 847549 DEBUG 
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 
127.0.0.1:10015_solr
   [junit4]   2> 847549 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history 
in memory.
   [junit4]   2> 847549 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
 ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and 
.scheduled_maintenance triggers
   [junit4]   2> 847549 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario 2. LoadAutoscaling json={'cluster-policy':[]}
   [junit4]   2> 847549 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario            LoadAutoscaling json={'cluster-policy':[]}
   [junit4]   2> 847549 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
 ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode 
version 1
   [junit4]   2> 847549 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
 ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 0
   [junit4]   2> 847549 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
 ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
   [junit4]   2> 847550 DEBUG 
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 
2
   [junit4]   2> 847550 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
 ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial 
livenodes: [127.0.0.1:10016_solr, 127.0.0.1:10015_solr]
   [junit4]   2> 847550 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
 ] o.a.s.c.a.OverseerTriggerThread -- deactivating old nodeLost / nodeAdded 
markers
   [junit4]   2> 847550 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
 ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 1
   [junit4]   2> 847550 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
 ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
   [junit4]   2> 847550 DEBUG (ScheduledTrigger-14116-thread-2) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 847550 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
 ] o.a.s.c.a.OverseerTriggerThread -- deactivating old nodeLost / nodeAdded 
markers
   [junit4]   2> 847550 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
 ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
   [junit4]   2> 847555 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario 3. RunSolrRequest  
path=/admin/collections&action=CREATE&autoAddReplicas=true&name=testCollection&numShards=2&replicationFactor=2&maxShardsPerNode=2
   [junit4]   2> 847555 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario            RunSolrRequest  
path=/admin/collections&action=CREATE&autoAddReplicas=true&name=testCollection&numShards=2&replicationFactor=2&maxShardsPerNode=2
   [junit4]   2> 847555 DEBUG (simCloudManagerPool-14115-thread-1) [     ] 
o.a.s.c.a.s.SimClusterStateProvider -- simCreateCollection testCollection
   [junit4]   2> 847570 DEBUG (ScheduledTrigger-14116-thread-3) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 847576 DEBUG (simCloudManagerPool-14115-thread-2) [     ] 
o.a.s.c.a.s.SimClusterStateProvider Running leader election (testCollection / 
shard1)
   [junit4]   2> 847576 DEBUG (simCloudManagerPool-14115-thread-2) [     ] 
o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testCollection / 
shard1 (currentVersion=2): {"core_node1":{
   [junit4]   2>     "core":"testCollection_shard1_replica_n1",
   [junit4]   2>     "shard":"shard1",
   [junit4]   2>     "collection":"testCollection",
   [junit4]   2>     "node_name":"127.0.0.1:10015_solr",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "state":"active",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>     "SEARCHER.searcher.numDocs":0}}
   [junit4]   2> 847587 DEBUG (simCloudManagerPool-14115-thread-1) [     ] 
o.a.s.c.a.s.SimClusterStateProvider -- finished createCollection testCollection
   [junit4]   2> 847587 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario 4. WaitCollection  
collection=testCollection&shards=2&replicas=2
   [junit4]   2> 847587 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario            WaitCollection  
collection=testCollection&shards=2&replicas=2
   [junit4]   2> 847590 DEBUG (ScheduledTrigger-14116-thread-4) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 847597 DEBUG (simCloudManagerPool-14115-thread-5) [     ] 
o.a.s.c.a.s.SimClusterStateProvider Running leader election (testCollection / 
shard2)
   [junit4]   2> 847597 DEBUG (simCloudManagerPool-14115-thread-5) [     ] 
o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testCollection / 
shard2 (currentVersion=4): {"core_node4":{
   [junit4]   2>     "core":"testCollection_shard2_replica_n4",
   [junit4]   2>     "shard":"shard2",
   [junit4]   2>     "collection":"testCollection",
   [junit4]   2>     "node_name":"127.0.0.1:10016_solr",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "state":"active",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>     "SEARCHER.searcher.numDocs":0}}
   [junit4]   2> 847608 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario 5. CtxSet  key=myNode&value=${_random_node_}
   [junit4]   2> 847608 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario            CtxSet  key=myNode&value=127.0.0.1:10015_solr
   [junit4]   2> 847608 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario 6. RunSolrRequest  
path=/admin/collections&action=ADDREPLICA&collection=testCollection&shard=shard1&node=${myNode}
   [junit4]   2> 847608 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario            RunSolrRequest  
path=/admin/collections&action=ADDREPLICA&collection=testCollection&shard=shard1&node=127.0.0.1:10015_solr
   [junit4]   2> 847608 INFO  (simCloudManagerPool-14115-thread-6) [     ] 
o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10015_solr for creating new 
replica of shard shard1 for collection testCollection
   [junit4]   2> 847608 INFO  (simCloudManagerPool-14115-thread-6) [     ] 
o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command.
   [junit4]   2> 847610 DEBUG (ScheduledTrigger-14116-thread-3) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 847618 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario 7. RunSolrRequest  
path=/admin/collections&action=ADDREPLICA&collection=testCollection&shard=shard1&node=${myNode}
   [junit4]   2> 847619 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario            RunSolrRequest  
path=/admin/collections&action=ADDREPLICA&collection=testCollection&shard=shard1&node=127.0.0.1:10015_solr
   [junit4]   2> 847619 INFO  (simCloudManagerPool-14115-thread-7) [     ] 
o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10015_solr for creating new 
replica of shard shard1 for collection testCollection
   [junit4]   2> 847619 INFO  (simCloudManagerPool-14115-thread-7) [     ] 
o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command.
   [junit4]   2> 847629 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario 8. LoopOp  iterations=${iterative}
   [junit4]   2> 847629 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario            LoopOp  iterations=10
   [junit4]   2> 847629 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario    * iter 1 :
   [junit4]   2> 847629 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario      - CalculateSuggestions        )
   [junit4]   2> 847630 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario         - 1 suggestions
   [junit4]   2> 847630 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario      - ApplySuggestions    )
   [junit4]   2> 847630 DEBUG (ScheduledTrigger-14116-thread-3) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 847651 DEBUG (ScheduledTrigger-14116-thread-3) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 847671 DEBUG (ScheduledTrigger-14116-thread-3) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 847691 DEBUG (ScheduledTrigger-14116-thread-3) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 847711 DEBUG (ScheduledTrigger-14116-thread-3) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 847731 DEBUG (ScheduledTrigger-14116-thread-2) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 847731 DEBUG (simCloudManagerPool-14115-thread-8) [     ] 
o.a.s.c.a.s.SimClusterStateProvider -- new replica: {"core_node10":{
   [junit4]   2>     "core":"testCollection_shard2_replica_n9",
   [junit4]   2>     "shard":"shard2",
   [junit4]   2>     "collection":"testCollection",
   [junit4]   2>     "node_name":"127.0.0.1:10016_solr",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "state":"active",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>     "SEARCHER.searcher.numDocs":0}}
   [junit4]   2> 847751 DEBUG (ScheduledTrigger-14116-thread-3) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 847771 DEBUG (ScheduledTrigger-14116-thread-2) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 847792 DEBUG (ScheduledTrigger-14116-thread-3) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 847812 DEBUG (ScheduledTrigger-14116-thread-4) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 847832 DEBUG (ScheduledTrigger-14116-thread-2) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 847842 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario      - RunSolrRequest      
path=/admin/collections&action=ADDREPLICA&collection=testCollection&shard=shard1&node=127.0.0.1:10015_solr)
   [junit4]   2> 847843 INFO  (simCloudManagerPool-14115-thread-9) [     ] 
o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10015_solr for creating new 
replica of shard shard1 for collection testCollection
   [junit4]   2> 847843 INFO  (simCloudManagerPool-14115-thread-9) [     ] 
o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command.
   [junit4]   2> 847852 DEBUG (ScheduledTrigger-14116-thread-4) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 847853 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario      - RunSolrRequest      
path=/admin/collections&action=ADDREPLICA&collection=testCollection&shard=shard1&node=127.0.0.1:10015_solr)
   [junit4]   2> 847854 INFO  (simCloudManagerPool-14115-thread-10) [     ] 
o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10015_solr for creating new 
replica of shard shard1 for collection testCollection
   [junit4]   2> 847854 INFO  (simCloudManagerPool-14115-thread-10) [     ] 
o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command.
   [junit4]   2> 847864 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario    * iter 2 :
   [junit4]   2> 847864 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario      - CalculateSuggestions        )
   [junit4]   2> 847865 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario         - 1 suggestions
   [junit4]   2> 847865 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario      - ApplySuggestions    )
   [junit4]   2> 847872 DEBUG (ScheduledTrigger-14116-thread-2) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 847892 DEBUG (ScheduledTrigger-14116-thread-2) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 847912 DEBUG (ScheduledTrigger-14116-thread-4) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 847932 DEBUG (ScheduledTrigger-14116-thread-2) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 847952 DEBUG (ScheduledTrigger-14116-thread-1) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 847965 DEBUG (simCloudManagerPool-14115-thread-11) [     ] 
o.a.s.c.a.s.SimClusterStateProvider -- new replica: {"core_node16":{
   [junit4]   2>     "core":"testCollection_shard1_replica_n15",
   [junit4]   2>     "shard":"shard1",
   [junit4]   2>     "collection":"testCollection",
   [junit4]   2>     "node_name":"127.0.0.1:10016_solr",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "state":"active",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>     "SEARCHER.searcher.numDocs":0}}
   [junit4]   2> 847973 DEBUG (ScheduledTrigger-14116-thread-4) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 847993 DEBUG (ScheduledTrigger-14116-thread-1) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 848013 DEBUG (ScheduledTrigger-14116-thread-4) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 848033 DEBUG (ScheduledTrigger-14116-thread-1) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 848053 DEBUG (ScheduledTrigger-14116-thread-1) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 848073 DEBUG (ScheduledTrigger-14116-thread-1) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 848076 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario      - RunSolrRequest      
path=/admin/collections&action=ADDREPLICA&collection=testCollection&shard=shard1&node=127.0.0.1:10015_solr)
   [junit4]   2> 848076 INFO  (simCloudManagerPool-14115-thread-12) [     ] 
o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10015_solr for creating new 
replica of shard shard1 for collection testCollection
   [junit4]   2> 848076 INFO  (simCloudManagerPool-14115-thread-12) [     ] 
o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command.
   [junit4]   2> 848087 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario      - RunSolrRequest      
path=/admin/collections&action=ADDREPLICA&collection=testCollection&shard=shard1&node=127.0.0.1:10015_solr)
   [junit4]   2> 848087 INFO  (simCloudManagerPool-14115-thread-13) [     ] 
o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10015_solr for creating new 
replica of shard shard1 for collection testCollection
   [junit4]   2> 848088 INFO  (simCloudManagerPool-14115-thread-13) [     ] 
o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command.
   [junit4]   2> 848093 DEBUG (ScheduledTrigger-14116-thread-4) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 848098 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario    * iter 3 :
   [junit4]   2> 848098 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario      - CalculateSuggestions        )
   [junit4]   2> 848098 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario         - 1 suggestions
   [junit4]   2> 848098 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario      - ApplySuggestions    )
   [junit4]   2> 848113 DEBUG (ScheduledTrigger-14116-thread-2) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 848133 DEBUG (ScheduledTrigger-14116-thread-3) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 848153 DEBUG (ScheduledTrigger-14116-thread-4) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 848174 DEBUG (ScheduledTrigger-14116-thread-1) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 848194 DEBUG (ScheduledTrigger-14116-thread-2) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 848199 DEBUG (simCloudManagerPool-14115-thread-14) [     ] 
o.a.s.c.a.s.SimClusterStateProvider -- new replica: {"core_node22":{
   [junit4]   2>     "core":"testCollection_shard1_replica_n21",
   [junit4]   2>     "shard":"shard1",
   [junit4]   2>     "collection":"testCollection",
   [junit4]   2>     "node_name":"127.0.0.1:10016_solr",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "path":"/admin/collections",
   [junit4]   2>     "node":"127.0.0.1:10015_solr",
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "action":"ADDREPLICA",
   [junit4]   2>     "createNodeSet":"127.0.0.1:10015_solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6}}
   [junit4]   2> 848214 DEBUG (ScheduledTrigger-14116-thread-1) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 848234 DEBUG (ScheduledTrigger-14116-thread-4) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 848254 DEBUG (ScheduledTrigger-14116-thread-3) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 848274 DEBUG (ScheduledTrigger-14116-thread-4) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 848294 DEBUG (ScheduledTrigger-14116-thread-3) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 848310 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario      - RunSolrRequest      
path=/admin/collections&action=ADDREPLICA&collection=testCollection&shard=shard1&node=127.0.0.1:10015_solr)
   [junit4]   2> 848310 INFO  (simCloudManagerPool-14115-thread-15) [     ] 
o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10015_solr for creating new 
replica of shard shard1 for collection testCollection
   [junit4]   2> 848310 INFO  (simCloudManagerPool-14115-thread-15) [     ] 
o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command.
   [junit4]   2> 848314 DEBUG (ScheduledTrigger-14116-thread-4) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 848321 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario      - RunSolrRequest      
path=/admin/collections&action=ADDREPLICA&collection=testCollection&shard=shard1&node=127.0.0.1:10015_solr)
   [junit4]   2> 848321 INFO  (simCloudManagerPool-14115-thread-16) [     ] 
o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10015_solr for creating new 
replica of shard shard1 for collection testCollection
   [junit4]   2> 848321 INFO  (simCloudManagerPool-14115-thread-16) [     ] 
o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command.
   [junit4]   2> 848331 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario    * iter 4 :
   [junit4]   2> 848332 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario      - CalculateSuggestions        )
   [junit4]   2> 848332 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario         - 1 suggestions
   [junit4]   2> 848332 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario      - ApplySuggestions    )
   [junit4]   2> 848334 DEBUG (ScheduledTrigger-14116-thread-2) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 848355 DEBUG (ScheduledTrigger-14116-thread-2) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 848375 DEBUG (ScheduledTrigger-14116-thread-1) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 848395 DEBUG (ScheduledTrigger-14116-thread-3) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 848415 DEBUG (ScheduledTrigger-14116-thread-4) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 848433 DEBUG (simCloudManagerPool-14115-thread-17) [     ] 
o.a.s.c.a.s.SimClusterStateProvider -- new replica: {"core_node28":{
   [junit4]   2>     "core":"testCollection_shard1_replica_n27",
   [junit4]   2>     "shard":"shard1",
   [junit4]   2>     "collection":"testCollection",
   [junit4]   2>     "node_name":"127.0.0.1:10016_solr",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "path":"/admin/collections",
   [junit4]   2>     "node":"127.0.0.1:10015_solr",
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "action":"ADDREPLICA",
   [junit4]   2>     "createNodeSet":"127.0.0.1:10015_solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6}}
   [junit4]   2> 848435 DEBUG (ScheduledTrigger-14116-thread-3) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 848455 DEBUG (ScheduledTrigger-14116-thread-4) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 848475 DEBUG (ScheduledTrigger-14116-thread-1) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 848495 DEBUG (ScheduledTrigger-14116-thread-4) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 848515 DEBUG (ScheduledTrigger-14116-thread-1) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 848536 DEBUG (ScheduledTrigger-14116-thread-4) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 848543 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario      - RunSolrRequest      
path=/admin/collections&action=ADDREPLICA&collection=testCollection&shard=shard1&node=127.0.0.1:10015_solr)
   [junit4]   2> 848544 INFO  (simCloudManagerPool-14115-thread-18) [     ] 
o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10015_solr for creating new 
replica of shard shard1 for collection testCollection
   [junit4]   2> 848544 INFO  (simCloudManagerPool-14115-thread-18) [     ] 
o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command.
   [junit4]   2> 848554 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario      - RunSolrRequest      
path=/admin/collections&action=ADDREPLICA&collection=testCollection&shard=shard1&node=127.0.0.1:10015_solr)
   [junit4]   2> 848555 INFO  (simCloudManagerPool-14115-thread-19) [     ] 
o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10015_solr for creating new 
replica of shard shard1 for collection testCollection
   [junit4]   2> 848555 INFO  (simCloudManagerPool-14115-thread-19) [     ] 
o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command.
   [junit4]   2> 848556 DEBUG (ScheduledTrigger-14116-thread-1) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 848566 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario    * iter 5 :
   [junit4]   2> 848566 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario      - CalculateSuggestions        )
   [junit4]   2> 848566 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario         - 1 suggestions
   [junit4]   2> 848566 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario      - ApplySuggestions    )
   [junit4]   2> 848576 DEBUG (ScheduledTrigger-14116-thread-3) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 848596 DEBUG (ScheduledTrigger-14116-thread-2) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 848616 DEBUG (ScheduledTrigger-14116-thread-1) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 848636 DEBUG (ScheduledTrigger-14116-thread-2) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 848656 DEBUG (ScheduledTrigger-14116-thread-4) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 848667 DEBUG (simCloudManagerPool-14115-thread-20) [     ] 
o.a.s.c.a.s.SimClusterStateProvider -- new replica: {"core_node34":{
   [junit4]   2>     "core":"testCollection_shard1_replica_n33",
   [junit4]   2>     "shard":"shard1",
   [junit4]   2>     "collection":"testCollection",
   [junit4]   2>     "node_name":"127.0.0.1:10016_solr",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "path":"/admin/collections",
   [junit4]   2>     "node":"127.0.0.1:10015_solr",
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "action":"ADDREPLICA",
   [junit4]   2>     "createNodeSet":"127.0.0.1:10015_solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6}}
   [junit4]   2> 848676 DEBUG (ScheduledTrigger-14116-thread-3) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 848697 DEBUG (ScheduledTrigger-14116-thread-2) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 848717 DEBUG (ScheduledTrigger-14116-thread-4) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 848737 DEBUG (ScheduledTrigger-14116-thread-3) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 848757 DEBUG (ScheduledTrigger-14116-thread-2) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 848777 DEBUG (ScheduledTrigger-14116-thread-4) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 848778 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario      - RunSolrRequest      
path=/admin/collections&action=ADDREPLICA&collection=testCollection&shard=shard1&node=127.0.0.1:10015_solr)
   [junit4]   2> 848778 INFO  (simCloudManagerPool-14115-thread-21) [     ] 
o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10015_solr for creating new 
replica of shard shard1 for collection testCollection
   [junit4]   2> 848778 INFO  (simCloudManagerPool-14115-thread-21) [     ] 
o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command.
   [junit4]   2> 848789 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario      - RunSolrRequest      
path=/admin/collections&action=ADDREPLICA&collection=testCollection&shard=shard1&node=127.0.0.1:10015_solr)
   [junit4]   2> 848790 INFO  (simCloudManagerPool-14115-thread-22) [     ] 
o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10015_solr for creating new 
replica of shard shard1 for collection testCollection
   [junit4]   2> 848790 INFO  (simCloudManagerPool-14115-thread-22) [     ] 
o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command.
   [junit4]   2> 848797 DEBUG (ScheduledTrigger-14116-thread-2) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 848800 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario    * iter 6 :
   [junit4]   2> 848800 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario      - CalculateSuggestions        )
   [junit4]   2> 848801 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario         - 1 suggestions
   [junit4]   2> 848801 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario      - ApplySuggestions    )
   [junit4]   2> 848817 DEBUG (ScheduledTrigger-14116-thread-3) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 848837 DEBUG (ScheduledTrigger-14116-thread-2) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 848858 DEBUG (ScheduledTrigger-14116-thread-3) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 848878 DEBUG (ScheduledTrigger-14116-thread-4) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 848898 DEBUG (ScheduledTrigger-14116-thread-3) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 848901 DEBUG (simCloudManagerPool-14115-thread-23) [     ] 
o.a.s.c.a.s.SimClusterStateProvider -- new replica: {"core_node40":{
   [junit4]   2>     "core":"testCollection_shard1_replica_n39",
   [junit4]   2>     "shard":"shard1",
   [junit4]   2>     "collection":"testCollection",
   [junit4]   2>     "node_name":"127.0.0.1:10016_solr",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "path":"/admin/collections",
   [junit4]   2>     "node":"127.0.0.1:10015_solr",
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "action":"ADDREPLICA",
   [junit4]   2>     "createNodeSet":"127.0.0.1:10015_solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6}}
   [junit4]   2> 848918 DEBUG (ScheduledTrigger-14116-thread-4) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 848938 DEBUG (ScheduledTrigger-14116-thread-1) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 848958 DEBUG (ScheduledTrigger-14116-thread-4) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 848978 DEBUG (ScheduledTrigger-14116-thread-1) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 848998 DEBUG (ScheduledTrigger-14116-thread-4) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 849012 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario      - RunSolrRequest      
path=/admin/collections&action=ADDREPLICA&collection=testCollection&shard=shard1&node=127.0.0.1:10015_solr)
   [junit4]   2> 849013 INFO  (simCloudManagerPool-14115-thread-24) [     ] 
o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10015_solr for creating new 
replica of shard shard1 for collection testCollection
   [junit4]   2> 849013 INFO  (simCloudManagerPool-14115-thread-24) [     ] 
o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command.
   [junit4]   2> 849019 DEBUG (ScheduledTrigger-14116-thread-1) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 849023 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario      - RunSolrRequest      
path=/admin/collections&action=ADDREPLICA&collection=testCollection&shard=shard1&node=127.0.0.1:10015_solr)
   [junit4]   2> 849024 INFO  (simCloudManagerPool-14115-thread-25) [     ] 
o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10015_solr for creating new 
replica of shard shard1 for collection testCollection
   [junit4]   2> 849024 INFO  (simCloudManagerPool-14115-thread-25) [     ] 
o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command.
   [junit4]   2> 849034 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario    * iter 7 :
   [junit4]   2> 849034 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario      - CalculateSuggestions        )
   [junit4]   2> 849035 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario         - 1 suggestions
   [junit4]   2> 849035 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario      - ApplySuggestions    )
   [junit4]   2> 849039 DEBUG (ScheduledTrigger-14116-thread-4) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 849059 DEBUG (ScheduledTrigger-14116-thread-1) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 849079 DEBUG (ScheduledTrigger-14116-thread-3) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 849099 DEBUG (ScheduledTrigger-14116-thread-1) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 849119 DEBUG (ScheduledTrigger-14116-thread-4) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 849136 DEBUG (simCloudManagerPool-14115-thread-26) [     ] 
o.a.s.c.a.s.SimClusterStateProvider -- new replica: {"core_node46":{
   [junit4]   2>     "core":"testCollection_shard1_replica_n45",
   [junit4]   2>     "shard":"shard1",
   [junit4]   2>     "collection":"testCollection",
   [junit4]   2>     "node_name":"127.0.0.1:10016_solr",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "path":"/admin/collections",
   [junit4]   2>     "node":"127.0.0.1:10015_solr",
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "action":"ADDREPLICA",
   [junit4]   2>     "createNodeSet":"127.0.0.1:10015_solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6}}
   [junit4]   2> 849139 DEBUG (ScheduledTrigger-14116-thread-1) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 849159 DEBUG (ScheduledTrigger-14116-thread-4) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 849180 DEBUG (ScheduledTrigger-14116-thread-1) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 849200 DEBUG (ScheduledTrigger-14116-thread-4) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 849220 DEBUG (ScheduledTrigger-14116-thread-1) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 849240 DEBUG (ScheduledTrigger-14116-thread-3) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 849247 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario      - RunSolrRequest      
path=/admin/collections&action=ADDREPLICA&collection=testCollection&shard=shard1&node=127.0.0.1:10015_solr)
   [junit4]   2> 849247 INFO  (simCloudManagerPool-14115-thread-27) [     ] 
o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10015_solr for creating new 
replica of shard shard1 for collection testCollection
   [junit4]   2> 849247 INFO  (simCloudManagerPool-14115-thread-27) [     ] 
o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command.
   [junit4]   2> 849258 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario      - RunSolrRequest      
path=/admin/collections&action=ADDREPLICA&collection=testCollection&shard=shard1&node=127.0.0.1:10015_solr)
   [junit4]   2> 849259 INFO  (simCloudManagerPool-14115-thread-28) [     ] 
o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10015_solr for creating new 
replica of shard shard1 for collection testCollection
   [junit4]   2> 849259 INFO  (simCloudManagerPool-14115-thread-28) [     ] 
o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command.
   [junit4]   2> 849260 DEBUG (ScheduledTrigger-14116-thread-1) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 849269 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario    * iter 8 :
   [junit4]   2> 849269 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario      - CalculateSuggestions        )
   [junit4]   2> 849270 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario         - 1 suggestions
   [junit4]   2> 849270 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario      - ApplySuggestions    )
   [junit4]   2> 849280 DEBUG (ScheduledTrigger-14116-thread-4) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 849300 DEBUG (ScheduledTrigger-14116-thread-3) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 849320 DEBUG (ScheduledTrigger-14116-thread-4) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 849341 DEBUG (ScheduledTrigger-14116-thread-3) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 849361 DEBUG (ScheduledTrigger-14116-thread-2) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 849371 DEBUG (simCloudManagerPool-14115-thread-29) [     ] 
o.a.s.c.a.s.SimClusterStateProvider -- new replica: {"core_node52":{
   [junit4]   2>     "core":"testCollection_shard1_replica_n51",
   [junit4]   2>     "shard":"shard1",
   [junit4]   2>     "collection":"testCollection",
   [junit4]   2>     "node_name":"127.0.0.1:10016_solr",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "path":"/admin/collections",
   [junit4]   2>     "node":"127.0.0.1:10015_solr",
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "action":"ADDREPLICA",
   [junit4]   2>     "createNodeSet":"127.0.0.1:10015_solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6}}
   [junit4]   2> 849381 DEBUG (ScheduledTrigger-14116-thread-1) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 849401 DEBUG (ScheduledTrigger-14116-thread-3) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 849421 DEBUG (ScheduledTrigger-14116-thread-2) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 849441 DEBUG (ScheduledTrigger-14116-thread-4) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 849461 DEBUG (ScheduledTrigger-14116-thread-2) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 849481 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario      - RunSolrRequest      
path=/admin/collections&action=ADDREPLICA&collection=testCollection&shard=shard1&node=127.0.0.1:10015_solr)
   [junit4]   2> 849481 DEBUG (ScheduledTrigger-14116-thread-1) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 849482 INFO  (simCloudManagerPool-14115-thread-30) [     ] 
o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10015_solr for creating new 
replica of shard shard1 for collection testCollection
   [junit4]   2> 849482 INFO  (simCloudManagerPool-14115-thread-30) [     ] 
o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command.
   [junit4]   2> 849492 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario      - RunSolrRequest      
path=/admin/collections&action=ADDREPLICA&collection=testCollection&shard=shard1&node=127.0.0.1:10015_solr)
   [junit4]   2> 849493 INFO  (simCloudManagerPool-14115-thread-31) [     ] 
o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10015_solr for creating new 
replica of shard shard1 for collection testCollection
   [junit4]   2> 849493 INFO  (simCloudManagerPool-14115-thread-31) [     ] 
o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command.
   [junit4]   2> 849501 DEBUG (ScheduledTrigger-14116-thread-4) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 849503 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario    * iter 9 :
   [junit4]   2> 849503 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario      - CalculateSuggestions        )
   [junit4]   2> 849504 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario         - 1 suggestions
   [junit4]   2> 849504 INFO  
(TEST-TestSimScenario.testSuggestions-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.s.SimScenario      - ApplySuggestions    )
   [junit4]   2> 849522 DEBUG (ScheduledTrigger-14116-thread-1) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2

[...truncated too long message...]

tTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 851563 DEBUG (ScheduledTrigger-14128-thread-4) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 851583 DEBUG (ScheduledTrigger-14128-thread-3) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 851603 DEBUG (ScheduledTrigger-14128-thread-4) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 851623 DEBUG (ScheduledTrigger-14128-thread-3) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 851643 DEBUG (ScheduledTrigger-14128-thread-4) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 851664 DEBUG (ScheduledTrigger-14128-thread-2) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 851684 DEBUG (ScheduledTrigger-14128-thread-4) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 851704 DEBUG (ScheduledTrigger-14128-thread-2) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 851724 DEBUG (ScheduledTrigger-14128-thread-4) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 851744 DEBUG (ScheduledTrigger-14128-thread-2) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 851764 DEBUG (ScheduledTrigger-14128-thread-4) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 851784 DEBUG (ScheduledTrigger-14128-thread-2) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 851804 DEBUG (ScheduledTrigger-14128-thread-3) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 851824 DEBUG (ScheduledTrigger-14128-thread-2) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 851844 DEBUG (ScheduledTrigger-14128-thread-3) [     ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 851861 INFO  (Simulated OverseerAutoScalingTriggerThread) [    
 ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are 
closed, exiting.
   [junit4]   2> 851861 DEBUG 
(TEST-TestSimScenario.testSplitShard-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 851861 DEBUG 
(TEST-TestSimScenario.testSplitShard-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 851862 DEBUG 
(TEST-TestSimScenario.testSplitShard-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 851862 DEBUG 
(TEST-TestSimScenario.testSplitShard-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool 
executor
   [junit4]   2> 851862 DEBUG 
(TEST-TestSimScenario.testSplitShard-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 851862 DEBUG 
(TEST-TestSimScenario.testSplitShard-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 851862 INFO  
(TEST-TestSimScenario.testSplitShard-seed#[AD9AE3BE82781274]) [     ] 
o.a.s.SolrTestCaseJ4 ###Ending testSplitShard
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestSimScenario 
-Dtests.method=testSplitShard -Dtests.seed=AD9AE3BE82781274 
-Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=kn 
-Dtests.timezone=America/Creston -Dtests.asserts=true 
-Dtests.file.encoding=UTF-8
   [junit4] FAILURE 0.61s J2 | TestSimScenario.testSplitShard <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: 
java.util.concurrent.TimeoutException: OverseerTriggerThread never caught up to 
the latest znodeVersion
   [junit4]    >        at 
__randomizedtesting.SeedInfo.seed([AD9AE3BE82781274:76904ED29C8D2ECB]:0)
   [junit4]    >        at 
org.apache.solr.cloud.autoscaling.sim.TestSimScenario.testSplitShard(TestSimScenario.java:160)
   [junit4]    >        at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]    >        at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]    >        at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]    >        at 
java.base/java.lang.reflect.Method.invoke(Method.java:567)
   [junit4]    >        at java.base/java.lang.Thread.run(Thread.java:830)
   [junit4]   2> NOTE: leaving temporary files on disk at: 
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.sim.TestSimScenario_AD9AE3BE82781274-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene86): {}, 
docValues:{}, maxPointsInLeafNode=1774, maxMBSortInHeap=6.946108428845359, 
sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@e3337d0),
 locale=kn, timezone=America/Creston
   [junit4]   2> NOTE: Linux 5.3.0-59-generic amd64/AdoptOpenJDK 13.0.2 
(64-bit)/cpus=16,threads=1,free=214540528,total=498073600
   [junit4]   2> NOTE: All tests run in this JVM: [TestRTimerTree, 
BufferStoreTest, NotRequiredUniqueKeyTest, TestNumericTerms32, TestQueryUtils, 
TestChildDocTransformer, TestTextField, 
TestUnifiedSolrHighlighterWithoutStoredId, TestFreeTextSuggestions, 
RandomizedTaggerTest, TestInPlaceUpdateWithRouteField, TestSimpleTextCodec, 
DeleteNodeTest, TestComponentsName, HdfsRestartWhileUpdatingTest, 
RequestLoggingTest, TestNumericTokenStream, ScheduledMaintenanceTriggerTest, 
TestDynamicFieldResource, TestCloudNestedDocsSort, AuthToolTest, 
MetricsHistoryWithAuthIntegrationTest, OverseerTaskQueueTest, 
TestCollationField, BasicAuthOnSingleNodeTest, TestConfig, TestWithCollection, 
RollingRestartTest, TestXIncludeConfig, TolerantUpdateProcessorTest, 
SuggestComponentTest, DistribJoinFromCollectionTest, TestCloudInspectUtil, 
SpatialHeatmapFacetsTest, TestDistributedGrouping, 
SharedFSAutoReplicaFailoverTest, TestDynamicURP, TestJoin, TestSQLHandler, 
TestMergePolicyConfig, SuggesterTest, TestOmitPositions, 
TestDeleteCollectionOnDownNodes, TestPullReplicaErrorHandling, 
SolrCloudReportersTest, PrimitiveFieldTypeTest, 
PeerSyncWithIndexFingerprintCachingTest, TestNoOpRegenerator, TestUtils, 
StressHdfsTest, TestEmbeddedSolrServerConstructors, TestExactSharedStatsCache, 
SpellPossibilityIteratorTest, TestManagedSynonymFilterFactory, 
PreAnalyzedUpdateProcessorTest, TestJsonFacets, 
ConjunctionSolrSpellCheckerTest, SOLR749Test, TestLuceneMatchVersion, 
TestFilteredDocIdSet, CopyFieldTest, TestRecovery, 
ConcurrentDeleteAndCreateCollectionTest, LeaderElectionTest, ConfigSetsAPITest, 
TestDistributedTracing, FullHLLTest, TestCloudSearcherWarming, 
CoreAdminCreateDiscoverTest, SolrLogPostToolTest, TestPKIAuthenticationPlugin, 
ChaosMonkeySafeLeaderTest, SubstringBytesRefFilterTest, 
FullSolrCloudDistribCmdsTest, OutOfBoxZkACLAndCredentialsProvidersTest, 
OverriddenZkACLAndCredentialsProvidersTest, RecoveryAfterSoftCommitTest, 
SolrCloudExampleTest, TestCloudPseudoReturnFields, 
TestExactSharedStatsCacheCloud, TestExactStatsCacheCloud, 
TestExclusionRuleCollectionAccess, TestLocalStatsCacheCloud, 
TestRandomRequestDistribution, TestRequestForwarding, TestSSLRandomization, 
TestSizeLimitedDistributedMap, TestSkipOverseerOperations, 
TestTlogReplayVsRecovery, TestTlogReplica, TestCollectionAPI, 
TestHdfsCloudBackupRestore, IndexSizeTriggerMixedBoundsTest, 
NodeAddedTriggerTest, TriggerCooldownIntegrationTest, TestSimNodeAddedTrigger, 
TestSimScenario]
   [junit4] Completed [591/910 (1!)] on J2 in 5.16s, 4 tests, 1 failure <<< 
FAILURES!

[...truncated 53064 lines...]
[repro] Jenkins log URL: 
https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/26986/consoleText

[repro] Revision: 6801d4c13982b42007ec6d1ea1f443902e2fb438

[repro] Ant options: "-Dargs=-XX:-UseCompressedOops -XX:+UseParallelGC"
[repro] Repro line:  ant test  -Dtestcase=TestSimScenario 
-Dtests.method=testSplitShard -Dtests.seed=AD9AE3BE82781274 
-Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=kn 
-Dtests.timezone=America/Creston -Dtests.asserts=true 
-Dtests.file.encoding=UTF-8

[repro] JUnit rest result XML files will be moved to: ./repro-reports
[repro] ant clean

[...truncated 6 lines...]
[repro] Test suites by module:
[repro]    solr/core
[repro]       TestSimScenario
[repro] ant compile-test

[...truncated 2469 lines...]
[repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=5 
-Dtests.class="*.TestSimScenario" -Dtests.showOutput=onerror 
"-Dargs=-XX:-UseCompressedOops -XX:+UseParallelGC" 
-Dtests.seed=AD9AE3BE82781274 -Dtests.multiplier=3 -Dtests.slow=true 
-Dtests.locale=kn -Dtests.timezone=America/Creston -Dtests.asserts=true 
-Dtests.file.encoding=UTF-8

[...truncated 96 lines...]
[repro] Failures w/original seeds:
[repro]   0/5 failed: org.apache.solr.cloud.autoscaling.sim.TestSimScenario
[repro] Exiting with code 0

[...truncated 43 lines...]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to