Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2792/

1 tests failed.
FAILED:  
org.apache.solr.cloud.autoscaling.sim.TestSimComputePlanAction.testNodeAdded

Error Message:
ComputePlanAction should have computed exactly 1 operation, but was: 
[org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@5ccf28a8,
 
org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@75ac8258]
 expected:<1> but was:<2>

Stack Trace:
java.lang.AssertionError: ComputePlanAction should have computed exactly 1 
operation, but was: 
[org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@5ccf28a8,
 
org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@75ac8258]
 expected:<1> but was:<2>
        at 
__randomizedtesting.SeedInfo.seed([BD1BA4CE4A8EFFE7:D8D8F2B9E82D57E4]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.failNotEquals(Assert.java:647)
        at org.junit.Assert.assertEquals(Assert.java:128)
        at org.junit.Assert.assertEquals(Assert.java:472)
        at 
org.apache.solr.cloud.autoscaling.sim.TestSimComputePlanAction.testNodeAdded(TestSimComputePlanAction.java:314)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 12659 lines...]
   [junit4] Suite: 
org.apache.solr.cloud.autoscaling.sim.TestSimComputePlanAction
   [junit4]   2> 301207 INFO  
(SUITE-TestSimComputePlanAction-seed#[BD1BA4CE4A8EFFE7]-worker) [    ] 
o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: 
test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.sim.TestSimComputePlanAction_BD1BA4CE4A8EFFE7-001/init-core-data-001
   [junit4]   2> 301208 INFO  
(SUITE-TestSimComputePlanAction-seed#[BD1BA4CE4A8EFFE7]-worker) [    ] 
o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) 
w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 301210 INFO  
(SUITE-TestSimComputePlanAction-seed#[BD1BA4CE4A8EFFE7]-worker) [    ] 
o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: 
@org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 301216 DEBUG 
(SUITE-TestSimComputePlanAction-seed#[BD1BA4CE4A8EFFE7]-worker) [    ] 
o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 
127.0.0.1:10007_solr
   [junit4]   2> 301216 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and 
.scheduled_maintenance triggers
   [junit4]   2> 301216 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode 
version 0
   [junit4]   2> 301216 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion -1
   [junit4]   2> 301216 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 0
   [junit4]   2> 301217 INFO  
(SUITE-TestSimComputePlanAction-seed#[BD1BA4CE4A8EFFE7]-worker) [    ] 
o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history 
in memory.
   [junit4]   2> 301218 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial 
livenodes: [127.0.0.1:10007_solr]
   [junit4]   2> 301218 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 301218 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion 0
   [junit4]   2> 301220 INFO  
(TEST-TestSimComputePlanAction.testNodeLost-seed#[BD1BA4CE4A8EFFE7]) [    ] 
o.a.s.SolrTestCaseJ4 ###Starting testNodeLost
   [junit4]   2> 301220 DEBUG 
(TEST-TestSimComputePlanAction.testNodeLost-seed#[BD1BA4CE4A8EFFE7]) [    ] 
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 
1
   [junit4]   2> 301220 INFO  
(TEST-TestSimComputePlanAction.testNodeLost-seed#[BD1BA4CE4A8EFFE7]) [    ] 
o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing 
object cache...
   [junit4]   2> 301220 WARN  (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are 
closed, exiting.
   [junit4]   2> 301220 DEBUG 
(TEST-TestSimComputePlanAction.testNodeLost-seed#[BD1BA4CE4A8EFFE7]) [    ] 
o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 301220 DEBUG 
(TEST-TestSimComputePlanAction.testNodeLost-seed#[BD1BA4CE4A8EFFE7]) [    ] 
o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 301220 DEBUG 
(TEST-TestSimComputePlanAction.testNodeLost-seed#[BD1BA4CE4A8EFFE7]) [    ] 
o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 301220 DEBUG 
(TEST-TestSimComputePlanAction.testNodeLost-seed#[BD1BA4CE4A8EFFE7]) [    ] 
o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool 
executor
   [junit4]   2> 301221 WARN  (ScheduledTrigger-1518-thread-2) [    ] 
o.a.s.c.a.TriggerEventQueue Exception peeking queue of trigger 
.auto_add_replicas
   [junit4]   2> java.lang.InterruptedException: null
   [junit4]   2>        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1220)
 ~[?:1.8.0_172]
   [junit4]   2>        at 
java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:335)
 ~[?:1.8.0_172]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.sim.GenericDistributedQueue.firstChild(GenericDistributedQueue.java:380)
 ~[test/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.sim.GenericDistributedQueue.firstElement(GenericDistributedQueue.java:515)
 ~[test/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.sim.GenericDistributedQueue.peek(GenericDistributedQueue.java:152)
 ~[test/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.TriggerEventQueue.peekEvent(TriggerEventQueue.java:68)
 [java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerWrapper.run(ScheduledTriggers.java:573)
 [java/:?]
   [junit4]   2>        at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
[?:1.8.0_172]
   [junit4]   2>        at 
java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [?:1.8.0_172]
   [junit4]   2>        at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
 [?:1.8.0_172]
   [junit4]   2>        at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
 [?:1.8.0_172]
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 
[?:1.8.0_172]
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
[?:1.8.0_172]
   [junit4]   2>        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
   [junit4]   2> 301221 WARN  (ScheduledTrigger-1518-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger NodeLostTrigger ran but was already closed
   [junit4]   2> 301221 DEBUG 
(TEST-TestSimComputePlanAction.testNodeLost-seed#[BD1BA4CE4A8EFFE7]) [    ] 
o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 301221 DEBUG 
(TEST-TestSimComputePlanAction.testNodeLost-seed#[BD1BA4CE4A8EFFE7]) [    ] 
o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 301222 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and 
.scheduled_maintenance triggers
   [junit4]   2> 301222 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode 
version 2
   [junit4]   2> 301222 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion -1
   [junit4]   2> 301222 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
   [junit4]   2> 301222 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial 
livenodes: [127.0.0.1:10007_solr]
   [junit4]   2> 301222 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 301223 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
   [junit4]   2> 301223 DEBUG (ScheduledTrigger-1523-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 1
   [junit4]   2> 301243 DEBUG (ScheduledTrigger-1523-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 1
   [junit4]   2> 301264 DEBUG (ScheduledTrigger-1523-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 1
   [junit4]   2> 301284 DEBUG (ScheduledTrigger-1523-thread-4) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 1
   [junit4]   2> 301304 DEBUG (ScheduledTrigger-1523-thread-4) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 1
   [junit4]   2> 301324 DEBUG (simCloudManagerPool-1522-thread-1) [    ] 
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 301324 DEBUG (simCloudManagerPool-1522-thread-1) [    ] 
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 
3
   [junit4]   2> 301325 DEBUG (ScheduledTrigger-1523-thread-4) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 1
   [junit4]   2> 301327 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
   [junit4]   2> 301327 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 301327 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
   [junit4]   2> 301338 DEBUG (simCloudManagerPool-1522-thread-2) [    ] 
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 301338 DEBUG (simCloudManagerPool-1522-thread-2) [    ] 
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 
4
   [junit4]   2> 301338 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
   [junit4]   2> 301339 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 301339 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
   [junit4]   2> 301345 DEBUG (ScheduledTrigger-1523-thread-4) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 1
   [junit4]   2> 301365 DEBUG (ScheduledTrigger-1523-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 1
   [junit4]   2> 301385 DEBUG (ScheduledTrigger-1523-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 1
   [junit4]   2> 301406 DEBUG (ScheduledTrigger-1523-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 1
   [junit4]   2> 301426 DEBUG (ScheduledTrigger-1523-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 1
   [junit4]   2> 301446 DEBUG (simCloudManagerPool-1522-thread-3) [    ] 
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 301446 DEBUG (simCloudManagerPool-1522-thread-3) [    ] 
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 
5
   [junit4]   2> 301446 DEBUG (ScheduledTrigger-1523-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2
   [junit4]   2> 301447 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
   [junit4]   2> 301447 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.NodeLostTrigger NodeLostTrigger node_lost_trigger - Initial 
livenodes: [127.0.0.1:10008_solr, 127.0.0.1:10007_solr]
   [junit4]   2> 301447 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 301447 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
   [junit4]   2> 301447 DEBUG (ScheduledTrigger-1523-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with 
currently live nodes: 2
   [junit4]   2> 301447 DEBUG (simCloudManagerPool-1522-thread-4) [    ] 
o.a.s.c.o.ClusterStateMutator building a new cName: testNodeLost
   [junit4]   2> 301466 INFO  (simCloudManagerPool-1522-thread-4) [    ] 
o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 3772411043 
sessionWrapper.createTime 3772410928182109, this.sessionWrapper.createTime 
3772410928182109 
   [junit4]   2> 301467 DEBUG (ScheduledTrigger-1523-thread-4) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2
   [junit4]   2> 301471 DEBUG (ScheduledTrigger-1523-thread-4) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with 
currently live nodes: 2
   [junit4]   2> 301477 DEBUG (simCloudManagerPool-1522-thread-5) [    ] 
o.a.s.c.a.s.SimClusterStateProvider -- submit leader election for testNodeLost 
/ shard1
   [junit4]   2> 301478 DEBUG (simCloudManagerPool-1522-thread-7) [    ] 
o.a.s.c.a.s.SimClusterStateProvider Running leader election for testNodeLost / 
shard1
   [junit4]   2> 301478 DEBUG (simCloudManagerPool-1522-thread-7) [    ] 
o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeLost / 
shard1: core_node1
   [junit4]   2> 301487 DEBUG (ScheduledTrigger-1523-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2
   [junit4]   2> 301491 DEBUG (ScheduledTrigger-1523-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with 
currently live nodes: 2
   [junit4]   2> 301497 DEBUG (simCloudManagerPool-1522-thread-6) [    ] 
o.a.s.c.a.s.SimClusterStateProvider -- already has leader testNodeLost / shard1
   [junit4]   2> 301497 DEBUG 
(TEST-TestSimComputePlanAction.testNodeLost-seed#[BD1BA4CE4A8EFFE7]) [    ] 
o.a.s.c.a.s.TestSimComputePlanAction -- cluster state: znodeVersion: 7
   [junit4]   2> live nodes:[127.0.0.1:10008_solr, 127.0.0.1:10007_solr]
   [junit4]   2> 
collections:{testNodeLost=DocCollection(testNodeLost//clusterstate.json/6)={
   [junit4]   2>   "replicationFactor":"1",
   [junit4]   2>   "pullReplicas":"0",
   [junit4]   2>   "router":{"name":"compositeId"},
   [junit4]   2>   "maxShardsPerNode":"1",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"2",
   [junit4]   2>   "tlogReplicas":"0",
   [junit4]   2>   "autoCreated":"true",
   [junit4]   2>   "shards":{"shard1":{
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node1":{
   [junit4]   2>           "core":"testNodeLost_shard1_replica_n1",
   [junit4]   2>           "leader":"true",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":10240,
   [junit4]   2>           "node_name":"127.0.0.1:10008_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>           "SEARCHER.searcher.numDocs":0},
   [junit4]   2>         "core_node2":{
   [junit4]   2>           "core":"testNodeLost_shard1_replica_n2",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":10240,
   [junit4]   2>           "node_name":"127.0.0.1:10007_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>           "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>       "range":"80000000-7fffffff",
   [junit4]   2>       "state":"active"}}}}
   [junit4]   2> 301498 DEBUG 
(TEST-TestSimComputePlanAction.testNodeLost-seed#[BD1BA4CE4A8EFFE7]) [    ] 
o.a.s.c.a.s.SimClusterStateProvider -- submit leader election for testNodeLost 
/ shard1
   [junit4]   2> 301503 INFO  
(TEST-TestSimComputePlanAction.testNodeLost-seed#[BD1BA4CE4A8EFFE7]) [    ] 
o.a.s.c.a.s.TestSimComputePlanAction Stopped_node : 127.0.0.1:10008_solr
   [junit4]   2> 301504 DEBUG (simCloudManagerPool-1522-thread-8) [    ] 
o.a.s.c.a.s.SimClusterStateProvider Running leader election for testNodeLost / 
shard1
   [junit4]   2> 301504 INFO  (simCloudManagerPool-1522-thread-8) [    ] 
o.a.s.c.ActionThrottle Throttling leader attempts - waiting for 3723ms
   [junit4]   2> 301507 DEBUG (ScheduledTrigger-1523-thread-4) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2
   [junit4]   2> 301507 DEBUG (ScheduledTrigger-1523-thread-4) [    ] 
o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10008_solr
   [junit4]   2> 301511 DEBUG (ScheduledTrigger-1523-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with 
currently live nodes: 2
   [junit4]   2> 301511 DEBUG (ScheduledTrigger-1523-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10008_solr
   [junit4]   2> 301527 DEBUG (ScheduledTrigger-1523-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2
   [junit4]   2> 301532 DEBUG (ScheduledTrigger-1523-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with 
currently live nodes: 2
   [junit4]   2> 301548 DEBUG (ScheduledTrigger-1523-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2
   [junit4]   2> 301552 DEBUG (ScheduledTrigger-1523-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with 
currently live nodes: 2
   [junit4]   2> 301568 DEBUG (ScheduledTrigger-1523-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2
   [junit4]   2> 301572 DEBUG (ScheduledTrigger-1523-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with 
currently live nodes: 2
   [junit4]   2> 301578 DEBUG (simCloudManagerPool-1522-thread-8) [    ] 
o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeLost / 
shard1: core_node2
   [junit4]   2> 301588 DEBUG (ScheduledTrigger-1523-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2
   [junit4]   2> 301592 DEBUG (ScheduledTrigger-1523-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with 
currently live nodes: 2
   [junit4]   2> 301608 DEBUG (ScheduledTrigger-1523-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2
   [junit4]   2> 301612 DEBUG (ScheduledTrigger-1523-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with 
currently live nodes: 2
   [junit4]   2> 301629 DEBUG (ScheduledTrigger-1523-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2
   [junit4]   2> 301633 DEBUG (ScheduledTrigger-1523-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with 
currently live nodes: 2
   [junit4]   2> 301649 DEBUG (ScheduledTrigger-1523-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2
   [junit4]   2> 301653 DEBUG (ScheduledTrigger-1523-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with 
currently live nodes: 2
   [junit4]   2> 301653 DEBUG (ScheduledTrigger-1523-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost 
nodes: [127.0.0.1:10008_solr]
   [junit4]   2> 301653 DEBUG (ScheduledTrigger-1523-thread-2) [    ] 
o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"d66fd6c94b81fTaio0mzda7rz53omopvisqsqzc",
   [junit4]   2>   "source":"node_lost_trigger",
   [junit4]   2>   "eventTime":3772413331683359,
   [junit4]   2>   "eventType":"NODELOST",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[3772413331683359],
   [junit4]   2>     "preferredOperation":"movereplica",
   [junit4]   2>     "nodeNames":["127.0.0.1:10008_solr"]}}
   [junit4]   2> 301654 DEBUG (ScheduledTrigger-1523-thread-2) [    ] 
o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, 
.scheduled_maintenance, node_lost_trigger]
   [junit4]   2> 301661 DEBUG (simCloudManagerPool-1522-thread-9) [    ] 
o.a.s.c.o.ClusterStateMutator building a new cName: .system
   [junit4]   2> 301674 INFO  (simCloudManagerPool-1522-thread-9) [    ] 
o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 3772421439 
sessionWrapper.createTime 3772421403462709, this.sessionWrapper.createTime 
3772421403462709 
   [junit4]   2> 301685 DEBUG (simCloudManagerPool-1522-thread-10) [    ] 
o.a.s.c.a.s.SimClusterStateProvider -- submit leader election for .system / 
shard1
   [junit4]   2> 301705 DEBUG (simCloudManagerPool-1522-thread-11) [    ] 
o.a.s.c.a.s.SimClusterStateProvider Running leader election for .system / shard1
   [junit4]   2> 301705 DEBUG (simCloudManagerPool-1522-thread-11) [    ] 
o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for .system / shard1: 
core_node1
   [junit4]   2> 301709 DEBUG (AutoscalingActionExecutor-1524-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"d66fd6c94b81fTaio0mzda7rz53omopvisqsqzc",
   [junit4]   2>   "source":"node_lost_trigger",
   [junit4]   2>   "eventTime":3772413331683359,
   [junit4]   2>   "eventType":"NODELOST",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[3772413331683359],
   [junit4]   2>     "preferredOperation":"movereplica",
   [junit4]   2>     "_enqueue_time_":3772420459938009,
   [junit4]   2>     "nodeNames":["127.0.0.1:10008_solr"]}}
   [junit4]   2> 301709 DEBUG (AutoscalingActionExecutor-1524-thread-1) [    ] 
o.a.s.c.a.ComputePlanAction -- processing event: {
   [junit4]   2>   "id":"d66fd6c94b81fTaio0mzda7rz53omopvisqsqzc",
   [junit4]   2>   "source":"node_lost_trigger",
   [junit4]   2>   "eventTime":3772413331683359,
   [junit4]   2>   "eventType":"NODELOST",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[3772413331683359],
   [junit4]   2>     "preferredOperation":"movereplica",
   [junit4]   2>     "_enqueue_time_":3772420459938009,
   [junit4]   2>     "nodeNames":["127.0.0.1:10008_solr"]}} with context 
properties: {BEFORE_ACTION=[compute_plan]}
   [junit4]   2> 301724 INFO  (AutoscalingActionExecutor-1524-thread-1) [    ] 
o.a.s.c.a.ComputePlanAction Computed Plan: 
action=MOVEREPLICA&collection=testNodeLost&targetNode=127.0.0.1:10009_solr&inPlaceMove=true&replica=core_node1
   [junit4]   2> 301724 INFO  (AutoscalingActionExecutor-1524-thread-1) [    ] 
o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 3772423965 
sessionWrapper.createTime 3772423294651709, this.sessionWrapper.createTime 
3772423294651709 
   [junit4]   2> 301731 DEBUG 
(TEST-TestSimComputePlanAction.testNodeLost-seed#[BD1BA4CE4A8EFFE7]) [    ] 
o.a.s.c.a.s.SimClusterStateProvider -- submit leader election for .system / 
shard1
   [junit4]   2> 301740 INFO  
(TEST-TestSimComputePlanAction.testNodeLost-seed#[BD1BA4CE4A8EFFE7]) [    ] 
o.a.s.c.a.s.TestSimComputePlanAction -------------_ FINAL STATE --------------
   [junit4]   2> 301740 INFO  
(TEST-TestSimComputePlanAction.testNodeLost-seed#[BD1BA4CE4A8EFFE7]) [    ] 
o.a.s.c.a.s.TestSimComputePlanAction * Node values: {
   [junit4]   2>   "127.0.0.1:10009_solr":{
   [junit4]   2>     "sysprop.java.vendor":"Oracle Corporation",
   [junit4]   2>     "cores":1,
   [junit4]   2>     "host":"127.0.0.1",
   [junit4]   2>     "freedisk":1023,
   [junit4]   2>     "sysprop.java.version":"1.8.0_172",
   [junit4]   2>     "ip_1":"127",
   [junit4]   2>     "ip_3":"0",
   [junit4]   2>     "ip_2":"0",
   [junit4]   2>     "ip_4":"1",
   [junit4]   2>     "node":"127.0.0.1:10009_solr",
   [junit4]   2>     "metrics:solr.jvm:buffers.direct.Count":0,
   [junit4]   2>     "port":10009,
   [junit4]   2>     "heapUsage":123450000,
   [junit4]   2>     "sysLoadAvg":1.0,
   [junit4]   2>     
"metrics:solr.node:ADMIN./admin/authorization.clientErrors:count":0},
   [junit4]   2>   "127.0.0.1:10008_solr":{
   [junit4]   2>     "sysprop.java.vendor":"Oracle Corporation",
   [junit4]   2>     "cores":1,
   [junit4]   2>     "host":"127.0.0.1",
   [junit4]   2>     "freedisk":1023,
   [junit4]   2>     "sysprop.java.version":"1.8.0_172",
   [junit4]   2>     "ip_1":"127",
   [junit4]   2>     "ip_3":"0",
   [junit4]   2>     "ip_2":"0",
   [junit4]   2>     "ip_4":"1",
   [junit4]   2>     "node":"127.0.0.1:10008_solr",
   [junit4]   2>     "metrics:solr.jvm:buffers.direct.Count":0,
   [junit4]   2>     "port":10008,
   [junit4]   2>     "heapUsage":123450000,
   [junit4]   2>     "sysLoadAvg":1.0,
   [junit4]   2>     
"metrics:solr.node:ADMIN./admin/authorization.clientErrors:count":0},
   [junit4]   2>   "127.0.0.1:10007_solr":{
   [junit4]   2>     "sysprop.java.vendor":"Oracle Corporation",
   [junit4]   2>     "cores":1,
   [junit4]   2>     "host":"127.0.0.1",
   [junit4]   2>     "freedisk":999,
   [junit4]   2>     "sysprop.java.version":"1.8.0_172",
   [junit4]   2>     "ip_1":"127",
   [junit4]   2>     "ip_3":"0",
   [junit4]   2>     "ip_2":"0",
   [junit4]   2>     "ip_4":"1",
   [junit4]   2>     "node":"127.0.0.1:10007_solr",
   [junit4]   2>     "metrics:solr.jvm:buffers.direct.Count":0,
   [junit4]   2>     "port":10007,
   [junit4]   2>     "heapUsage":123450000,
   [junit4]   2>     "sysLoadAvg":1.0,
   [junit4]   2>     
"metrics:solr.node:ADMIN./admin/authorization.clientErrors:count":0}}
   [junit4]   2> 301740 INFO  
(TEST-TestSimComputePlanAction.testNodeLost-seed#[BD1BA4CE4A8EFFE7]) [    ] 
o.a.s.c.a.s.TestSimComputePlanAction * Live nodes: [127.0.0.1:10007_solr]
   [junit4]   2> 301740 INFO  
(TEST-TestSimComputePlanAction.testNodeLost-seed#[BD1BA4CE4A8EFFE7]) [    ] 
o.a.s.c.a.s.TestSimComputePlanAction * Collection .system state: 
DocCollection(.system//clusterstate.json/14)={
   [junit4]   2>   "replicationFactor":"1",
   [junit4]   2>   "pullReplicas":"0",
   [junit4]   2>   "router":{"name":"compositeId"},
   [junit4]   2>   "maxShardsPerNode":"1",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"1",
   [junit4]   2>   "tlogReplicas":"0",
   [junit4]   2>   "autoCreated":"true",
   [junit4]   2>   "shards":{"shard1":{
   [junit4]   2>       "replicas":{"core_node1":{
   [junit4]   2>           "core":".system_shard1_replica_n1",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":2,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":11240,
   [junit4]   2>           "node_name":"127.0.0.1:10009_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "INDEX.sizeInGB":1.0468065738677979E-5,
   [junit4]   2>           "SEARCHER.searcher.numDocs":2}},
   [junit4]   2>       "range":"80000000-7fffffff",
   [junit4]   2>       "state":"active"}}}
   [junit4]   2> 301740 INFO  
(TEST-TestSimComputePlanAction.testNodeLost-seed#[BD1BA4CE4A8EFFE7]) [    ] 
o.a.s.c.a.s.TestSimComputePlanAction * Collection testNodeLost state: 
DocCollection(testNodeLost//clusterstate.json/14)={
   [junit4]   2>   "replicationFactor":"1",
   [junit4]   2>   "pullReplicas":"0",
   [junit4]   2>   "router":{"name":"compositeId"},
   [junit4]   2>   "maxShardsPerNode":"1",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"2",
   [junit4]   2>   "tlogReplicas":"0",
   [junit4]   2>   "autoCreated":"true",
   [junit4]   2>   "shards":{"shard1":{
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node1":{
   [junit4]   2>           "core":"testNodeLost_shard1_replica_n1",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":10240,
   [junit4]   2>           "node_name":"127.0.0.1:10008_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>           "SEARCHER.searcher.numDocs":0},
   [junit4]   2>         "core_node2":{
   [junit4]   2>           "core":"testNodeLost_shard1_replica_n2",
   [junit4]   2>           "leader":"true",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":10240,
   [junit4]   2>           "node_name":"127.0.0.1:10007_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>           "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>       "range":"80000000-7fffffff",
   [junit4]   2>       "state":"active"}}}
   [junit4]   2> 301740 INFO  
(TEST-TestSimComputePlanAction.testNodeLost-seed#[BD1BA4CE4A8EFFE7]) [    ] 
o.a.s.SolrTestCaseJ4 ###Ending testNodeLost
   [junit4]   2> 301741 INFO  
(TEST-TestSimComputePlanAction.testNodeLost-seed#[BD1BA4CE4A8EFFE7]) [    ] 
o.a.s.c.a.s.SimSolrCloudTestCase #######################################
   [junit4]   2> ############ CLUSTER STATE ############
   [junit4]   2> #######################################
   [junit4]   2> ## Live nodes:         1
   [junit4]   2> ## Empty nodes:        0
   [junit4]   2> ## Dead nodes:         2
   [junit4]   2> ##             127.0.0.1:10008_solr
   [junit4]   2> ##             127.0.0.1:10009_solr
   [junit4]   2> ## Collections:        [.system, testNodeLost]
   [junit4]   2> ## Max replicas per node:      1
   [junit4]   2> ## Min replicas per node:      1
   [junit4]   2> ## Total replicas:             1
   [junit4]   2> ## * testNodeLost              1
   [junit4]   2> ##             - active           1
   [junit4]   2> ######### Solr op counts ##########
   [junit4]   2> ##             - CREATE             1
   [junit4]   2> ##             - autoscaling        3
   [junit4]   2> ##             - update             2
   [junit4]   2> ######### Autoscaling event counts ###########
   [junit4]   2> ## * Trigger: node_lost_trigger
   [junit4]   2> ##             - BEFORE_ACTION     1
   [junit4]   2> ##             - STARTED         1
   [junit4]   2> 
   [junit4]   2> 301745 INFO  
(TEST-TestSimComputePlanAction.testNodeWithMultipleReplicasLost-seed#[BD1BA4CE4A8EFFE7])
 [    ] o.a.s.SolrTestCaseJ4 ###Starting testNodeWithMultipleReplicasLost
   [junit4]   2> 301745 DEBUG 
(TEST-TestSimComputePlanAction.testNodeWithMultipleReplicasLost-seed#[BD1BA4CE4A8EFFE7])
 [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode 
version 6
   [junit4]   2> 301745 DEBUG 
(TEST-TestSimComputePlanAction.testNodeWithMultipleReplicasLost-seed#[BD1BA4CE4A8EFFE7])
 [    ] o.a.s.c.a.s.SimNodeStateProvider - removing dead node values: 
127.0.0.1:10009_solr
   [junit4]   2> 301745 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
   [junit4]   2> 301745 DEBUG 
(TEST-TestSimComputePlanAction.testNodeWithMultipleReplicasLost-seed#[BD1BA4CE4A8EFFE7])
 [    ] o.a.s.c.a.s.SimNodeStateProvider - removing dead node values: 
127.0.0.1:10008_solr
   [junit4]   2> 301757 DEBUG (simCloudManagerPool-1522-thread-13) [    ] 
o.a.s.c.a.s.SimClusterStateProvider Running leader election for .system / shard1
   [junit4]   2> 301769 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 301769 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread   -- deleted 
/autoscaling/nodeLost/127.0.0.1:10009_solr
   [junit4]   2> 301769 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 301769 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion 6
   [junit4]   2> 301769 INFO  
(TEST-TestSimComputePlanAction.testNodeWithMultipleReplicasLost-seed#[BD1BA4CE4A8EFFE7])
 [    ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and 
clearing object cache...
   [junit4]   2> 301769 DEBUG 
(TEST-TestSimComputePlanAction.testNodeWithMultipleReplicasLost-seed#[BD1BA4CE4A8EFFE7])
 [    ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool 
executor now
   [junit4]   2> 301769 DEBUG 
(TEST-TestSimComputePlanAction.testNodeWithMultipleReplicasLost-seed#[BD1BA4CE4A8EFFE7])
 [    ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 301769 DEBUG 
(TEST-TestSimComputePlanAction.testNodeWithMultipleReplicasLost-seed#[BD1BA4CE4A8EFFE7])
 [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 301770 WARN  (AutoscalingActionExecutor-1524-thread-1) [    ] 
o.a.s.c.a.SystemLogListener Exception sending event to collection .system
   [junit4]   2> java.io.IOException: java.lang.InterruptedException
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.sim.SimCloudManager.request(SimCloudManager.java:632)
 ~[test/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.SystemLogListener.onEvent(SystemLogListener.java:118)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerListeners.fireListeners(ScheduledTriggers.java:792)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerListeners.fireListeners(ScheduledTriggers.java:768)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:315)
 ~[java/:?]
   [junit4]   2>        at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
~[?:1.8.0_172]
   [junit4]   2>        at 
java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_172]
   [junit4]   2>        at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
 ~[java/:?]
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 
[?:1.8.0_172]
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
[?:1.8.0_172]
   [junit4]   2>        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
   [junit4]   2> Caused by: java.lang.InterruptedException
   [junit4]   2>        at 
java.util.concurrent.FutureTask.awaitDone(FutureTask.java:404) ~[?:1.8.0_172]
   [junit4]   2>        at 
java.util.concurrent.FutureTask.get(FutureTask.java:191) ~[?:1.8.0_172]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.sim.SimCloudManager.request(SimCloudManager.java:630)
 ~[test/:?]
   [junit4]   2>        ... 10 more
   [junit4]   2> 301773 DEBUG (simCloudManagerPool-1522-thread-14) [    ] 
o.a.s.c.o.ClusterStateMutator building a new cName: .system
   [junit4]   2> 301775 WARN  (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are 
closed, exiting.
   [junit4]   2> 301785 DEBUG (simCloudManagerPool-1522-thread-15) [    ] 
o.a.s.c.o.ClusterStateMutator building a new cName: .system
   [junit4]   2> 301797 DEBUG (simCloudManagerPool-1522-thread-16) [    ] 
o.a.s.c.a.s.SimClusterStateProvider -- submit leader election for .system / 
shard1
   [junit4]   2> 301817 DEBUG (simCloudManagerPool-1522-thread-17) [    ] 
o.a.s.c.a.s.SimClusterStateProvider Running leader election for .system / shard1
   [junit4]   2> 301817 DEBUG (simCloudManagerPool-1522-thread-17) [    ] 
o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for .system / shard1: 
core_node2
   [junit4]   2> 303033 WARN  (AutoscalingActionExecutor-1524-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers Unhandled exception executing actions
   [junit4]   2> org.apache.lucene.store.AlreadyClosedException: 
ScheduledTrigger node_lost_trigger has been closed.
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerWrapper.dequeue(ScheduledTriggers.java:548)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:318)
 ~[java/:?]
   [junit4]   2>        at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
~[?:1.8.0_172]
   [junit4]   2>        at 
java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_172]
   [junit4]   2>        at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
 ~[java/:?]
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 
[?:1.8.0_172]
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
[?:1.8.0_172]
   [junit4]   2>        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
   [junit4]   2> 303034 DEBUG (AutoscalingActionExecutor-1524-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers -- processing took 66275 ms for event 
id=d66fd6c94b81fTaio0mzda7rz53omopvisqsqzc
   [junit4]   2> 303035 DEBUG 
(TEST-TestSimComputePlanAction.testNodeWithMultipleReplicasLost-seed#[BD1BA4CE4A8EFFE7])
 [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread 
pool executor
   [junit4]   2> 303035 DEBUG 
(TEST-TestSimComputePlanAction.testNodeWithMultipleReplicasLost-seed#[BD1BA4CE4A8EFFE7])
 [    ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 303035 DEBUG 
(TEST-TestSimComputePlanAction.testNodeWithMultipleReplicasLost-seed#[BD1BA4CE4A8EFFE7])
 [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed 
explicitly
   [junit4]   2> 303037 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and 
.scheduled_maintenance triggers
   [junit4]   2> 303037 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode 
version 7
   [junit4]   2> 303038 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion -1
   [junit4]   2> 303038 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 7
   [junit4]   2> 303039 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial 
livenodes: [127.0.0.1:10007_solr]
   [junit4]   2> 303048 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 303048 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion 7
   [junit4]   2> 303048 DEBUG (ScheduledTrigger-1526-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 1
   [junit4]   2> 303069 DEBUG (ScheduledTrigger-1526-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 1
   [junit4]   2> 303089 DEBUG (ScheduledTrigger-1526-thread-4) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 1
   [junit4]   2> 303109 DEBUG (ScheduledTrigger-1526-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 1
   [junit4]   2> 303129 DEBUG (ScheduledTrigger-1526-thread-4) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 1
   [junit4]   2> 303137 DEBUG (simCloudManagerPool-1525-thread-1) [    ] 
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 303137 DEBUG (simCloudManagerPool-1525-thread-1) [    ] 
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 
8
   [junit4]   2> 303141 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 8
   [junit4]   2> 303141 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 303141 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 8, lastZnodeVersion 8
   [junit4]   2> 303143 DEBUG (simCloudManagerPool-1525-thread-2) [    ] 
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 303143 DEBUG (simCloudManagerPool-1525-thread-2) [    ] 
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 
9
   [junit4]   2> 303143 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 9
   [junit4]   2> 303143 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 303143 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 9, lastZnodeVersion 9
   [junit4]   2> 303149 DEBUG (ScheduledTrigger-1526-thread-4) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 1
   [junit4]   2> 303170 DEBUG (ScheduledTrigger-1526-thread-4) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 1
   [junit4]   2> 303190 DEBUG (ScheduledTrigger-1526-thread-4) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 1
   [junit4]   2> 303210 DEBUG (ScheduledTrigger-1526-thread-4) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 1
   [junit4]   2> 303230 DEBUG (ScheduledTrigger-1526-thread-4) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 1
   [junit4]   2> 303247 DEBUG (simCloudManagerPool-1525-thread-3) [    ] 
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 303247 DEBUG (simCloudManagerPool-1525-thread-3) [    ] 
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 
10
   [junit4]   2> 303247 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 10
   [junit4]   2> 303247 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.NodeLostTrigger NodeLostTrigger node_lost_trigger - Initial 
livenodes: [127.0.0.1:10010_solr, 127.0.0.1:10012_solr, 127.0.0.1:10007_solr, 
127.0.0.1:10011_solr]
   [junit4]   2> 303248 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 303248 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 10, lastZnodeVersion 10
   [junit4]   2> 303248 DEBUG (ScheduledTrigger-1526-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with 
currently live nodes: 4
   [junit4]   2> 303249 DEBUG (simCloudManagerPool-1525-thread-4) [    ] 
o.a.s.c.o.ClusterStateMutator building a new cName: 
testNodeWithMultipleReplicasLost
   [junit4]   2> 303252 DEBUG (ScheduledTrigger-1526-thread-4) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 4
   [junit4]   2> 303274 DEBUG (ScheduledTrigger-1526-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 4
   [junit4]   2> 303281 DEBUG (ScheduledTrigger-1526-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with 
currently live nodes: 4
   [junit4]   2> 303281 INFO  (simCloudManagerPool-1525-thread-4) [    ] 
o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 3772501821 
sessionWrapper.createTime 3772500813332959, this.sessionWrapper.createTime 
3772500813332959 
   [junit4]   2> 303294 DEBUG (ScheduledTrigger-1526-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 4
   [junit4]   2> 303301 DEBUG (ScheduledTrigger-1526-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with 
currently live nodes: 4
   [junit4]   2> 303306 DEBUG (simCloudManagerPool-1525-thread-5) [    ] 
o.a.s.c.a.s.SimClusterStateProvider -- no replicas in 
testNodeWithMultipleReplicasLost / shard2
   [junit4]   2> 303306 DEBUG (simCloudManagerPool-1525-thread-5) [    ] 
o.a.s.c.a.s.SimClusterStateProvider -- submit leader election for 
testNodeWithMultipleReplicasLost / shard1
   [junit4]   2> 303313 DEBUG (simCloudManagerPool-1525-thread-11) [    ] 
o.a.s.c.a.s.SimClusterStateProvider Running leader election for 
testNodeWithMultipleReplicasLost / shard1
   [junit4]   2> 303316 DEBUG (simCloudManagerPool-1525-thread-6) [    ] 
o.a.s.c.a.s.SimClusterStateProvider -- no replicas in 
testNodeWithMultipleReplicasLost / shard2
   [junit4]   2> 303316 DEBUG (simCloudManagerPool-1525-thread-6) [    ] 
o.a.s.c.a.s.SimClusterStateProvider -- submit leader election for 
testNodeWithMultipleReplicasLost / shard1
   [junit4]   2> 303317 DEBUG (ScheduledTrigger-1526-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 4
   [junit4]   2> 303325 DEBUG (ScheduledTrigger-1526-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with 
currently live nodes: 4
   [junit4]   2> 303329 DEBUG (simCloudManagerPool-1525-thread-11) [    ] 
o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for 
testNodeWithMultipleReplicasLost / shard1: core_node1
   [junit4]   2> 303329 DEBUG (simCloudManagerPool-1525-thread-12) [    ] 
o.a.s.c.a.s.SimClusterStateProvider Running leader election for 
testNodeWithMultipleReplicasLost / shard1
   [junit4]   2> 303335 DEBUG (simCloudManagerPool-1525-thread-12) [    ] 
o.a.s.c.a.s.SimClusterStateProvider -- already has leader 
testNodeWithMultipleReplicasLost / shard1: shard1:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node1":{
   [junit4]   2>       
"core":"testNodeWithMultipleReplicasLost_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10011_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node2":{
   [junit4]   2>       
"core":"testNodeWithMultipleReplicasLost_shard1_replica_n2",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10012_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-ffffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 303337 DEBUG (ScheduledTrigger-1526-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 4
   [junit4]   2> 303339 DEBUG (simCloudManagerPool-1525-thread-7) [    ] 
o.a.s.c.a.s.SimClusterStateProvider -- no replicas in 
testNodeWithMultipleReplicasLost / shard2
   [junit4]   2> 303339 DEBUG (simCloudManagerPool-1525-thread-7) [    ] 
o.a.s.c.a.s.SimClusterStateProvider -- already has leader 
testNodeWithMultipleReplicasLost / shard1
   [junit4]   2> 303345 DEBUG (ScheduledTrigger-1526-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with 
currently live nodes: 4
   [junit4]   2> 303350 DEBUG (simCloudManagerPool-1525-thread-8) [    ] 
o.a.s.c.a.s.SimClusterStateProvider -- submit leader election for 
testNodeWithMultipleReplicasLost / shard2
   [junit4]   2> 303352 DEBUG (simCloudManagerPool-1525-thread-8) [    ] 
o.a.s.c.a.s.SimClusterStateProvider -- already has leader 
testNodeWithMultipleReplicasLost / shard1
   [junit4]   2> 303352 DEBUG (simCloudManagerPool-1525-thread-13) [    ] 
o.a.s.c.a.s.SimClusterStateProvider Running leader election for 
testNodeWithMultipleReplicasLost / shard2
   [junit4]   2> 303352 DEBUG (simCloudManagerPool-1525-thread-13) [    ] 
o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for 
testNodeWithMultipleReplicasLost / shard2: core_node4
   [junit4]   2> 303357 DEBUG (ScheduledTrigger-1526-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 4
   [junit4]   2> 303362 DEBUG (simCloudManagerPool-1525-thread-9) [    ] 
o.a.s.c.a.s.SimClusterStateProvider -- already has leader 
testNodeWithMultipleReplicasLost / shard2
   [junit4]   2> 303362 DEBUG (simCloudManagerPool-1525-thread-9) [    ] 
o.a.s.c.a.s.SimClusterStateProvider -- already has leader 
testNodeWithMultipleReplicasLost / shard1
   [junit4]   2> 303365 DEBUG (ScheduledTrigger-1526-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with 
currently live nodes: 4
   [junit4]   2> 303373 DEBUG (simCloudManagerPool-1525-thread-10) [    ] 
o.a.s.c.a.s.SimClusterStateProvider -- already has leader 
testNodeWithMultipleReplicasLost / shard2
   [junit4]   2> 303373 DEBUG (simCloudManagerPool-1525-thread-10) [    ] 
o.a.s.c.a.s.SimClusterStateProvider -- already has leader 
testNodeWithMultipleReplicasLost / shard1
   [junit4]   2> 303374 DEBUG 
(TEST-TestSimComputePlanAction.testNodeWithMultipleReplicasLost-seed#[BD1BA4CE4A8EFFE7])
 [    ] o.a.s.c.a.s.TestSimComputePlanAction -- cluster state: znodeVersion: 31
   [junit4]   2> live nodes:[127.0.0.1:10010_solr, 127.0.0.1:10012_solr, 
127.0.0.1:10007_solr, 127.0.0.1:10011_solr]
   [junit4]   2> 
collections:{.system=DocCollection(.system//clusterstate.json/30)={
   [junit4]   2>   "replicationFactor":"1",
   [junit4]   2>   "pullReplicas":"0",
   [junit4]   2>   "router":{"name":"compositeId"},
   [junit4]   2>   "maxShardsPerNode":"1",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"1",
   [junit4]   2>   "tlogReplicas":"0",
   [junit4]   2>   "autoCreated":"true",
   [junit4]   2>   "shards":{"shard1":{
   [junit4]   2>       "replicas":{"core_node2":{
   [junit4]   2>           "core":".system_shard1_replica_n1",
   [junit4]   2>           "leader":"true",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":2,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":11240,
   [junit4]   2>           "node_name":"127.0.0.1:10007_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "INDEX.sizeInGB":1.0468065738677979E-5,
   [junit4]   2>           "SEARCHER.searcher.numDocs":2}},
   [junit4]   2>       "range":"80000000-7fffffff",
   [junit4]   2>       "state":"active"}}}, 
testNodeWithMultipleReplicasLost=DocCollection(testNodeWithMultipleReplicasLost//clusterstate.json/30)={
   [junit4]   2>   "replicationFactor":"1",
   [junit4]   2>   "pullReplicas":"0",
   [junit4]   2>   "router":{"name":"compositeId"},
   [junit4]   2>   "maxShardsPerNode":"1",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"3",
   [junit4]   2>   "tlogReplicas":"0",
   [junit4]   2>   "autoCreated":"true",
   [junit4]   2>   "shards":{
   [junit4]   2>     "shard2":{
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node6":{
   [junit4]   2>           
"core":"testNodeWithMultipleReplicasLost_shard2_replica_n6",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":10240,
   [junit4]   2>           "node_name":"127.0.0.1:10011_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>           "SEARCHER.searcher.numDocs":0},
   [junit4]   2>         "core_node4":{
   [junit4]   2>           
"core":"testNodeWithMultipleReplicasLost_shard2_replica_n4",
   [junit4]   2>           "leader":"true",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":10240,
   [junit4]   2>           "node_name":"127.0.0.1:10010_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>           "SEARCHER.searcher.numDocs":0},
   [junit4]   2>         "core_node5":{
   [junit4]   2>           
"core":"testNodeWithMultipleReplicasLost_shard2_replica_n5",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":10240,
   [junit4]   2>           "node_name":"127.0.0.1:10012_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>           "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>       "range":"0-7fffffff",
   [junit4]   2>       "state":"active"},
   [junit4]   2>     "shard1":{
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node1":{
   [junit4]   2>           
"core":"testNodeWithMultipleReplicasLost_shard1_replica_n1",
   [junit4]   2>           "leader":"true",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":10240,
   [junit4]   2>           "node_name":"127.0.0.1:10011_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>           "SEARCHER.searcher.numDocs":0},
   [junit4]   2>         "core_node2":{
   [junit4]   2>           
"core":"testNodeWithMultipleReplicasLost_shard1_replica_n2",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":10240,
   [junit4]   2>           "node_name":"127.0.0.1:10012_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>           "SEARCHER.searcher.numDocs":0},
   [junit4]   2>         "core_node3":{
   [junit4]   2>           
"core":"testNodeWithMultipleReplicasLost_shard1_replica_n3",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":10240,
   [junit4]   2>           "node_name":"127.0.0.1:10010_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>           "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>       "range":"80000000-ffffffff",
   [junit4]   2>       "state":"active"}}}}
   [junit4]   2> 303375 DEBUG 
(TEST-TestSimComputePlanAction.testNodeWithMultipleReplicasLost-seed#[BD1BA4CE4A8EFFE7])
 [    ] o.a.s.c.a.s.SimClusterStateProvider -- submit leader election for 
testNodeWithMultipleReplicasLost / shard2
   [junit4]   2> 303377 DEBUG (ScheduledTrigger-1526-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 303377 DEBUG (ScheduledTrigger-1526-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10010_solr
   [junit4]   2> 303383 DEBUG 
(TEST-TestSimComputePlanAction.testNodeWithMultipleReplicasLost-seed#[BD1BA4CE4A8EFFE7])
 [    ] o.a.s.c.a.s.SimClusterStateProvider -- already has leader 
testNodeWithMultipleReplicasLost / shard1
   [junit4]   2> 303383 DEBUG (simCloudManagerPool-1525-thread-14) [    ] 
o.a.s.c.a.s.SimClusterStateProvider Running leader election for 
testNodeWithMultipleReplicasLost / shard2
   [junit4]   2> 303383 INFO  (simCloudManagerPool-1525-thread-14) [    ] 
o.a.s.c.ActionThrottle Throttling leader attempts - waiting for 3454ms
   [junit4]   2> 303386 DEBUG (ScheduledTrigger-1526-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with 
currently live nodes: 3
   [junit4]   2> 303386 DEBUG (ScheduledTrigger-1526-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10010_solr
   [junit4]   2> 303398 DEBUG (ScheduledTrigger-1526-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 303406 DEBUG (ScheduledTrigger-1526-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with 
currently live nodes: 3
   [junit4]   2> 303406 DEBUG (ScheduledTrigger-1526-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost 
nodes: [127.0.0.1:10010_solr]
   [junit4]   2> 303406 DEBUG (ScheduledTrigger-1526-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"d67133dfb0a9dTaio0mzda7rz53omopvisqsqzh",
   [junit4]   2>   "source":"node_lost_trigger",
   [junit4]   2>   "eventTime":3772507039140509,
   [junit4]   2>   "eventType":"NODELOST",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[3772507039140509],
   [junit4]   2>     "preferredOperation":"movereplica",
   [junit4]   2>     "nodeNames":["127.0.0.1:10010_solr"]}}
   [junit4]   2> 303406 DEBUG (ScheduledTrigger-1526-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, 
.scheduled_maintenance, node_lost_trigger]
   [junit4]   2> 303419 DEBUG (AutoscalingActionExecutor-1527-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"d67133dfb0a9dTaio0mzda7rz53omopvisqsqzh",
   [junit4]   2>   "source":"node_lost_trigger",
   [junit4]   2>   "eventTime":3772507039140509,
   [junit4]   2>   "eventType":"NODELOST",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[3772507039140509],
   [junit4]   2>     "preferredOperation":"movereplica",
   [junit4]   2>     "_enqueue_time_":3772508079564409,
   [junit4]   2>     "nodeNames":["127.0.0.1:10010_solr"]}}
   [junit4]   2> 303419 DEBUG (AutoscalingActionExecutor-1527-thread-1) [    ] 
o.a.s.c.a.ComputePlanAction -- processing event: {
   [junit4]   2>   "id":"d67133dfb0a9dTaio0mzda7rz53omopvisqsqzh",
   [junit4]   2>   "source":"node_lost_trigger",
   [junit4]   2>   "eventTime":3772507039140509,
   [junit4]   2>   "eventType":"NODELOST",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[3772507039140509],
   [junit4]   2>     "preferredOperation":"movereplica",
   [junit4]   2>     "_enqueue_time_":3772508079564409,
   [junit4]   2>     "nodeNames":["127.0.0.1:10010_solr"]}} with context 
properties: {BEFORE_ACTION=[compute_plan]}
   [junit4]   2> 303432 INFO  (AutoscalingActionExecutor-1527-thread-1) [    ] 
o.a.s.c.a.ComputePlanAction Computed Plan: 
action=MOVEREPLICA&collection=testNodeWithMultipleReplicasLost&targetNode=127.0.0.1:10007_solr&inPlaceMove=true&replica=core_node4
   [junit4]   2> 303433 INFO  (AutoscalingActionExecutor-1527-thread-1) [    ] 
o.a.s.c.a.ComputePlanAction Computed Plan: 
action=MOVEREPLICA&collection=testNodeWithMultipleReplicasLost&targetNode=127.0.0.1:10007_solr&inPlaceMove=true&replica=core_node3
   [junit4]   2> 303434 INFO  (AutoscalingActionExecutor-1527-thread-1) [    ] 
o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 3772509437 
sessionWrapper.createTime 3772509286186959, this.sessionWrapper.createTime 
3772509286186959 
   [junit4]   2> 303441 INFO  
(TEST-TestSimComputePlanAction.testNodeWithMultipleReplicasLost-seed#[BD1BA4CE4A8EFFE7])
 [    ] o.a.s.c.a.s.TestSimComputePlanAction 
action=MOVEREPLICA&collection=testNodeWithMultipleReplicasLost&targetNode=127.0.0.1:10007_solr&inPlaceMove=true&replica=core_node4
   [junit4]   2> 303442 INFO  
(TEST-TestSimComputePlanAction.testNodeWithMultipleReplicasLost-seed#[BD1BA4CE4A8EFFE7])
 [    ] o.a.s.c.a.s.TestSimComputePlanAction 
action=MOVEREPLICA&collection=testNodeWithMultipleReplicasLost&targetNode=127.0.0.1:10007_solr&inPlaceMove=true&replica=core_node3
   [junit4]   2> 303442 INFO  
(TEST-TestSimComputePlanAction.testNodeWithMultipleReplicasLost-seed#[BD1BA4CE4A8EFFE7])
 [    ] o.a.s.c.a.s.TestSimComputePlanAction -------------_ FINAL STATE 
--------------
   [junit4]   2> 303442 INFO  
(TEST-TestSimComputePlanAction.testNodeWithMultipleReplicasLost-seed#[BD1BA4CE4A8EFFE7])
 [    ] o.a.s.c.a.s.TestSimComputePlanAction * Node values: {
   [junit4]   2>   "127.0.0.1:10010_solr":{
   [junit4]   2>     "sysprop.java.vendor":"Oracle Corporation",
   [junit4]   2>     "cores":2,
   [junit4]   2>     "host":"127.0.0.1",
   [junit4]   2>     "freedisk":1022,
   [junit4]   2>     "sysprop.java.version":"1.8.0_172",
   [junit4]   2>     "ip_1":"127",
   [junit4]   2>     "ip_3":"0",
   [junit4]   2>     "ip_2":"0",
   [junit4]   2>     "ip_4":"1",
   [junit4]   2>     "node":"127.0.0.1:10010_solr",
   [junit4]   2>     "metrics:solr.jvm:buffers.direct.Count":0,
   [junit4]   2>     "port":10010,
   [junit4]   2>     "heapUsage":123450000,
   [junit4]   2>     "sysLoadAvg":1.0,
   [junit4]   2>     
"metrics:solr.node:ADMIN./admin/authorization.clientErrors:count":0},
   [junit4]   2>   "127.0.0.1:10012_solr":{
   [junit4]   2>     "sysprop.java.vendor":"Oracle Corporation",
   [junit4]   2>     "cores":2,
   [junit4]   2>     "host":"127.0.0.1",
   [junit4]   2>     "freedisk":1022,
   [junit4]   2>     "sysprop.java.version":"1.8.0_172",
   [junit4]   2>     "ip_1":"127",
   [junit4]   2>     "ip_3":"0",
   [junit4]   2>     "ip_2":"0",
   [junit4]   2>     "ip_4":"1",
   [junit4]   2>     "node":"127.0.0.1:10012_solr",
   [junit4]   2>     "metrics:solr.jvm:buffers.direct.Count":0,
   [junit4]   2>     "port":10012,
   [junit4]   2>     "heapUsage":123450000,
   [junit4]   2>     "sysLoadAvg":1.0,
   [junit4]   2>     
"metrics:solr.node:ADMIN./admin/authorization.clientErrors:count":0},
   [junit4]   2>   "127.0.0.1:10007_solr":{
   [junit4]   2>     "sysprop.java.vendor":"Oracle Corporation",
   [junit4]   2>     "cores":1,
   [junit4]   2>     "host":"127.0.0.1",
   [junit4]   2>     "freedisk":999,
   [junit4]   2>     "sysprop.java.version":"1.8.0_172",
   [junit4]   2>     "ip_1":"127",
   [junit4]   2>     "ip_3":"0",
   [junit4]   2>     "ip_2":"0",
   [junit4]   2>     "ip_4":"1",
   [junit4]   2>     "node":"127.0.0.1:10007_solr",
   [junit4]   2>     "metrics:solr.jvm:buffers.direct.Count":0,
   [junit4]   2>     "port":10007,
   [junit4]   2>     "heapUsage":123450000,
   [junit4]   2>     "sysLoadAvg":1.0,
   [junit4]   2>     
"metrics:solr.node:ADMIN./admin/authorization.clientErrors:count":0},
   [junit4]   2>   "127.0.0.1:10011_solr":{
   [junit4]   2>     "sysprop.java.vendor":"Oracle Corporation",
   [junit4]   2>     "cores":2,
   [junit4]   2>     "host":"127.0.0.1",
   [junit4]   2>     "freedisk":1022,
   [junit4]   2>     "sysprop.java.version":"1.8.0_172",
   [junit4]   2>     "ip_1":"127",
   [junit4]   2>     "ip_3":"0",
   [junit4]   2>     "ip_2":"0",
   [junit4]   2>     "ip_4":"1",
   [junit4]   2>     "node":"127.0.0.1:10011_solr",
   [junit4]   2>     "metrics:solr.jvm:buffers.direct.Count":0,
   [junit4]   2>     "port":10011,
   [junit4]   2>     "heapUsage":123450000,
   [junit4]   2>     "sysLoadAvg":1.0,
   [junit4]   2>     
"metrics:solr.node:ADMIN./admin/authorization.clientErrors:count":0}}
   [junit4]   2> 303442 DEBUG (AutoscalingActionExecutor-1527-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
   [junit4]   2> 303442 INFO  
(TEST-TestSimComputePlanAction.testNodeWithMultipleReplicasLost-seed#[BD1BA4CE4A8EFFE7])
 [    ] o.a.s.c.a.s.TestSimComputePlanAction * Live nodes: 
[127.0.0.1:10012_solr, 127.0.0.1:10007_solr, 127.0.0.1:10011_solr]
   [junit4]   2> 303442 DEBUG (AutoscalingActionExecutor-1527-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers Resuming trigger: .scheduled_maintenance after 100ms
   [junit4]   2> 303442 DEBUG (AutoscalingActionExecutor-1527-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger after 100ms
   [junit4]   2> 303442 DEBUG (AutoscalingActionExecutor-1527-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers -- processing took 1185 ms for event 
id=d67133dfb0a9dTaio0mzda7rz53omopvisqsqzh
   [junit4]   2> 303442 INFO  
(TEST-TestSimComputePlanAction.testNodeWithMultipleReplicasLost-seed#[BD1BA4CE4A8EFFE7])
 [    ] o.a.s.c.a.s.TestSimComputePlanAction * Collection .system state: 
DocCollection(.system//clusterstate.json/37)={
   [junit4]   2>   "replicationFactor":"1",
   [junit4]   2>   "pullReplicas":"0",
   [junit4]   2>   "router":{"name":"compositeId"},
   [junit4]   2>   "maxShardsPerNode":"1",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"1",
   [junit4]   2>   "tlogReplicas":"0",
   [junit4]   2>   "autoCreated":"true",
   [junit4]   2>   "shards":{"shard1":{
   [junit4]   2>       "replicas":{"core_node2":{
   [junit4]   2>           "core":".system_shard1_replica_n1",
   [junit4]   2>           "leader":"true",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":6,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":13240,
   [junit4]   2>           "node_name":"127.0.0.1:10007_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "INDEX.sizeInGB":1.2330710887908936E-5,
   [junit4]   2>           "SEARCHER.searcher.numDocs":6}},
   [junit4]   2>       "range":"80000000-7fffffff",
   [junit4]   2>       "state":"active"}}}
   [junit4]   2> 303443 INFO  
(TEST-TestSimComputePlanAction.testNodeWithMultipleReplicasLost-seed#[BD1BA4CE4A8EFFE7])
 [    ] o.a.s.c.a.s.TestSimComputePlanAction * Collection 
testNodeWithMultipleReplicasLost state: 
DocCollection(testNodeWithMultipleReplicasLost//clusterstate.json/37)={
   [junit4]   2>   "replicationFactor":"1",
   [junit4]   2>   "pullReplicas":"0",
   [junit4]   2>   "router":{"name":"compositeId"},
   [junit4]   2>   "maxShardsPerNode":"1",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"3",
   [junit4]   2>   "tlogReplicas":"0",
   [junit4]   2>   "autoCreated":"true",
   [junit4]   2>   "shards":{
   [junit4]   2>     "shard2":{
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node6":{
   [junit4]   2>           
"core":"testNodeWithMultipleReplicasLost_shard2_replica_n6",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":10240,
   [junit4]   2>           "node_name":"127.0.0.1:10011_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>           "SEARCHER.searcher.numDocs":0},
   [junit4]   2>         "core_node4":{
   [junit4]   2>           
"core":"testNodeWithMultipleReplicasLost_shard2_replica_n4",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":10240,
   [junit4]   2>           "node_name":"127.0.0.1:10010_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>           "SEARCHER.searcher.numDocs":0},
   [junit4]   2>         "core_node5":{
   [junit4]   2>           
"core":"testNodeWithMultipleReplicasLost_shard2_replica_n5",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":10240,
   [junit4]   2>           "node_name":"127.0.0.1:10012_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>           "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>       "range":"0-7fffffff",
   [junit4]   2>       "state":"active"},
   [junit4]   2>     "shard1":{
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node1":{
   [junit4]   2>           
"core":"testNodeWithMultipleReplicasLost_shard1_replica_n1",
   [junit4]   2>           "leader":"true",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":10240,
   [junit4]   2>           "node_name":"127.0.0.1:10011_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>           "SEARCHER.searcher.numDocs":0},
   [junit4]   2>         "core_node2":{
   [junit4]   2>           
"core":"testNodeWithMultipleReplicasLost_shard1_replica_n2",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":10240,
   [junit4]   2>           "node_name":"127.0.0.1:10012_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>           "SEARCHER.searcher.numDocs":0},
   [junit4]   2>         "core_node3":{
   [junit4]   2>           
"core":"testNodeWithMultipleReplicasLost_shard1_replica_n3",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":10240,
   [junit4]   2>           "node_name":"127.0.0.1:10010_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>           "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>       "range":"80000000-ffffffff",
   [junit4]   2>       "state":"active"}}}
   [junit4]   2> 303443 INFO  
(TEST-TestSimComputePlanAction.testNodeWithMultipleReplicasLost-seed#[BD1BA4CE4A8EFFE7])
 [    ] o.a.s.SolrTestCaseJ4 ###Ending testNodeWithMultipleReplicasLost
   [junit4]   2> 303443 INFO  
(TEST-TestSimComputePlanAction.testNodeWithMultipleReplicasLost-seed#[BD1BA4CE4A8EFFE7])
 [    ] o.a.s.c.a.s.SimSolrCloudTestCase #######################################
   [junit4]   2> ############ CLUSTER STATE ############
   [junit4]   2> #######################################
   [junit4]   2> ## Live nodes:         3
   [junit4]   2> ## Empty nodes:        0
   [junit4]   2> ## Dead nodes:         1
   [junit4]   2> ##             127.0.0.1:10010_solr
   [junit4]   2> ## Collections:        [.system, 
testNodeWithMultipleReplicasLost]
   [junit4]   2> ## Max replicas per node:      2
   [junit4]   2> ## Min replicas per node:      1
   [junit4]   2> ## Total replicas:             5
   [junit4]   2> ## * .system           1
   [junit4]   2> ##             - active           1
   [junit4]   2> ## * testNodeWithMultipleReplicasLost          4
   [junit4]   2> ##             - active           4
   [junit4]   2> ######### Solr op counts ##########
   [junit4]   2> ##             - CREATE             1
   [junit4]   2> ##             - autoscaling        3
   [junit4]   2> ##             - update             4
   [junit4]   2> ######### Autoscaling event counts ###########
   [junit4]   2> ## * Trigger: node_lost_trigger
   [junit4]   2> ##             - AFTER_ACTION     2
   [junit4]   2> ##             - BEFORE_ACTION     1
   [junit4]   2> ##             - FAILED          1
   [junit4]   2> ##             - STARTED         1
   [junit4]   2> ##             - SUCCEEDED       1
   [junit4]   2> 
   [junit4]   2> 303451 INFO  
(TEST-TestSimComputePlanAction.testNodeAdded-seed#[BD1BA4CE4A8EFFE7]) [    ] 
o.a.s.SolrTestCaseJ4 ###Starting testNodeAdded
   [junit4]   2> 303451 DEBUG 
(TEST-TestSimComputePlanAction.testNodeAdded-seed#[BD1BA4CE4A8EFFE7]) [    ] 
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 
11
   [junit4]   2> 303451 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 11
   [junit4]   2> 303451 DEBUG 
(TEST-TestSimComputePlanAction.testNodeAdded-seed#[BD1BA4CE4A8EFFE7]) [    ] 
o.a.s.c.a.s.SimNodeStateProvider - removing dead node values: 
127.0.0.1:10010_solr
   [junit4]   2> 303452 DEBUG 
(TEST-TestSimComputePlanAction.testNodeAdded-seed#[BD1BA4CE4A8EFFE7]) [    ] 
o.a.s.c.a.s.SimNodeStateProvider --removing value for 127.0.0.1:10011_solr
   [junit4]   2> 303452 DEBUG 
(TEST-TestSimComputePlanAction.testNodeAdded-seed#[BD1BA4CE4A8EFFE7]) [    ] 
o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 
127.0.0.1:10012_solr
   [junit4]   2> 303452 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 303452 DEBUG 
(TEST-TestSimComputePlanAction.testNodeAdded-seed#[BD1BA4CE4A8EFFE7]) [    ] 
o.a.s.c.a.s.SimNodeStateProvider --removing value for 127.0.0.1:10007_solr
   [junit4]   2> 303452 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 303452 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 11, lastZnodeVersion 11
   [junit4]   2> 303452 INFO  
(TEST-TestSimComputePlanAction.testNodeAdded-seed#[BD1BA4CE4A8EFFE7]) [    ] 
o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing 
object cache...
   [junit4]   2> 303452 DEBUG 
(TEST-TestSimComputePlanAction.testNodeAdded-seed#[BD1BA4CE4A8EFFE7]) [    ] 
o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 303452 WARN  (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are 
closed, exiting.
   [junit4]   2> 303452 DEBUG 
(TEST-TestSimComputePlanAction.testNodeAdded-seed#[BD1BA4CE4A8EFFE7]) [    ] 
o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 303452 DEBUG 
(TEST-TestSimComputePlanAction.testNodeAdded-seed#[BD1BA4CE4A8EFFE7]) [    ] 
o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 303452 DEBUG 
(TEST-TestSimComputePlanAction.testNodeAdded-seed#[BD1BA4CE4A8EFFE7]) [    ] 
o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool 
executor
   [junit4]   2> 303452 DEBUG 
(TEST-TestSimComputePlanAction.testNodeAdded-seed#[BD1BA4CE4A8EFFE7]) [    ] 
o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 303452 DEBUG 
(TEST-TestSimComputePlanAction.testNodeAdded-seed#[BD1BA4CE4A8EFFE7]) [    ] 
o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 303452 DEBUG (simCloudManagerPool-1525-thread-14) [    ] 
o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for 
testNodeWithMultipleReplicasLost / shard2: core_node6
   [junit4]   2> 303457 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and 
.scheduled_maintenance triggers
   [junit4]   2> 303457 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode 
version 12
   [junit4]   2> 303457 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 12, lastZnodeVersion -1
   [junit4]   2> 303457 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 12
   [junit4]   2> 303465 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial 
livenodes: [127.0.0.1:10012_solr]
   [junit4]   2> 303466 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 303466 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 12, lastZnodeVersion 12
   [junit4]   2> 303466 DEBUG (ScheduledTrigger-1529-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 1
   [junit4]   2> 303487 DEBUG (ScheduledTrigger-1529-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 1
   [junit4]   2> 303507 DEBUG (ScheduledTrigger-1529-thread-4) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 1
   [junit4]   2> 303527 DEBUG (ScheduledTrigger-1529-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 1
   [junit4]   2> 303548 DEBUG (ScheduledTrigger-1529-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 1
   [junit4]   2> 303554 DEBUG (simCloudManagerPool-1528-thread-1) [    ] 
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 303554 DEBUG (simCloudManagerPool-1528-thread-1) [    ] 
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 
13
   [junit4]   2> 303555 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 13
   [junit4]   2> 303555 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 303556 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 13, lastZnodeVersion 13
   [junit4]   2> 303557 DEBUG (simCloudManagerPool-1528-thread-2) [    ] 
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 303557 DEBUG (simCloudManagerPool-1528-thread-2) [    ] 
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 
14
   [junit4]   2> 303558 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 14
   [junit4]   2> 303558 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 303558 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 14, lastZnodeVersion 14
   [junit4]   2> 303568 DEBUG (ScheduledTrigger-1529-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 1
   [junit4]   2> 303588 DEBUG (ScheduledTrigger-1529-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 1
   [junit4]   2> 303608 DEBUG (ScheduledTrigger-1529-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 1
   [junit4]   2> 303619 DEBUG (simCloudManagerPool-1528-thread-3) [    ] 
o.a.s.c.o.ClusterStateMutator building a new cName: .system
   [junit4]   2> 303628 DEBUG (ScheduledTrigger-1529-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 1
   [junit4]   2> 303631 INFO  (simCloudManagerPool-1528-thread-3) [    ] 
o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 3772519309 
sessionWrapper.createTime 3772519271910409, this.sessionWrapper.createTime 
3772519271910409 
   [junit4]   2> 303643 DEBUG (simCloudManagerPool-1528-thread-4) [    ] 
o.a.s.c.a.s.SimClusterStateProvider -- submit leader election for .system / 
shard1
   [junit4]   2> 303644 DEBUG (simCloudManagerPool-1528-thread-5) [    ] 
o.a.s.c.a.s.SimClusterStateProvider Running leader election for .system / shard1
   [junit4]   2> 303644 DEBUG (simCloudManagerPool-1528-thread-5) [    ] 
o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for .system / shard1: 
core_node4
   [junit4]   2> 303649 DEBUG (ScheduledTrigger-1529-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 1
   [junit4]   2> 303661 DEBUG (simCloudManagerPool-1528-thread-9) [    ] 
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 303661 DEBUG (simCloudManagerPool-1528-thread-9) [    ] 
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 
15
   [junit4]   2> 303662 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 15
   [junit4]   2> 303662 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - Initial 
livenodes: [127.0.0.1:10012_solr]
   [junit4]   2> 303662 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated 
with properties: {event=nodeAdded, waitFor=1, actions=[{name=compute_plan, 
class=solr.ComputePlanAction}, {name=test, 
class=org.apache.solr.cloud.autoscaling.sim.TestSimComputePlanAction$AssertingTriggerAction}],
 enabled=true}
   [junit4]   2> 303662 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 15, lastZnodeVersion 15
   [junit4]   2> 303663 DEBUG (ScheduledTrigger-1529-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 303663 DEBUG (ScheduledTrigger-1529-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
   [junit4]   2> 303663 DEBUG (simCloudManagerPool-1528-thread-10) [    ] 
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 303664 DEBUG (simCloudManagerPool-1528-thread-10) [    ] 
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 
16
   [junit4]   2> 303664 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 16
   [junit4]   2> 303665 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 16, lastZnodeVersion 16
   [junit4]   2> 303666 DEBUG (simCloudManagerPool-1528-thread-11) [    ] 
o.a.s.c.o.ClusterStateMutator building a new cName: testNodeAdded
   [junit4]   2> 303669 DEBUG (ScheduledTrigger-1529-thread-4) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 1
   [junit4]   2> 303676 WARN  (simCloudManagerPool-1528-thread-11) [    ] 
o.a.s.c.a.c.CreateCollectionCmd Specified number of replicas of 4 on collection 
testNodeAdded is higher than the number of Solr instances currently live or 
live and part of your createNodeSet(1). It's unusual to run two replica of the 
same slice on the same Solr-instance.
   [junit4]   2> 303678 INFO  (simCloudManagerPool-1528-thread-11) [    ] 
o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 3772521676 
sessionWrapper.createTime 3772521614170309, this.sessionWrapper.createTime 
3772521614170309 
   [junit4]   2> 303685 DEBUG (ScheduledTrigger-1529-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 303685 DEBUG (ScheduledTrigger-1529-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
   [junit4]   2> 303689 DEBUG (ScheduledTrigger-1529-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 1
   [junit4]   2> 303690 DEBUG (simCloudManagerPool-1528-thread-12) [    ] 
o.a.s.c.a.s.SimClusterStateProvider -- submit leader election for testNodeAdded 
/ shard1
   [junit4]   2> 303695 DEBUG (simCloudManagerPool-1528-thread-16) [    ] 
o.a.s.c.a.s.SimClusterStateProvider Running leader election for testNodeAdded / 
shard1
   [junit4]   2> 303695 DEBUG (simCloudManagerPool-1528-thread-16) [    ] 
o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeAdded / 
shard1: cor

[...truncated too long message...]

ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/null1189140520
     [copy] Copying 238 files to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/null1189140520
   [delete] Deleting directory 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/null1189140520

check-working-copy:
[ivy:cachepath] :: resolving dependencies :: 
org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath]         confs: [default]
[ivy:cachepath]         found 
org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath]         found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath]         found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath]         found org.apache.httpcomponents#httpclient;4.3.6 in 
public
[ivy:cachepath]         found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath]         found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath]         found commons-codec#commons-codec;1.6 in public
[ivy:cachepath]         found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 19ms :: artifacts dl 2ms
        ---------------------------------------------------------------------
        |                  |            modules            ||   artifacts   |
        |       conf       | number| search|dwnlded|evicted|| number|dwnlded|
        ---------------------------------------------------------------------
        |      default     |   8   |   0   |   0   |   0   ||   8   |   0   |
        ---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for 
further details.
[wc-checker] Checking working copy status...

-jenkins-base:

BUILD SUCCESSFUL
Total time: 87 minutes 32 seconds
Archiving artifacts
java.lang.InterruptedException: no matches found within 10000
        at hudson.FilePath$34.hasMatch(FilePath.java:2678)
        at hudson.FilePath$34.invoke(FilePath.java:2557)
        at hudson.FilePath$34.invoke(FilePath.java:2547)
        at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2918)
Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to lucene
                at 
hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
                at 
hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
                at hudson.remoting.Channel.call(Channel.java:955)
                at hudson.FilePath.act(FilePath.java:1036)
                at hudson.FilePath.act(FilePath.java:1025)
                at hudson.FilePath.validateAntFileMask(FilePath.java:2547)
                at 
hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
                at 
hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
                at 
hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
                at 
hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
                at 
hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
                at hudson.model.Build$BuildExecution.post2(Build.java:186)
                at 
hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
                at hudson.model.Run.execute(Run.java:1819)
                at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
                at 
hudson.model.ResourceController.execute(ResourceController.java:97)
                at hudson.model.Executor.run(Executor.java:429)
Caused: hudson.FilePath$TunneledInterruptedException
        at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2920)
        at hudson.remoting.UserRequest.perform(UserRequest.java:212)
        at hudson.remoting.UserRequest.perform(UserRequest.java:54)
        at hudson.remoting.Request$2.run(Request.java:369)
        at 
hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:748)
Caused: java.lang.InterruptedException: java.lang.InterruptedException: no 
matches found within 10000
        at hudson.FilePath.act(FilePath.java:1038)
        at hudson.FilePath.act(FilePath.java:1025)
        at hudson.FilePath.validateAntFileMask(FilePath.java:2547)
        at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
        at 
hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
        at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
        at 
hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
        at 
hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
        at hudson.model.Build$BuildExecution.post2(Build.java:186)
        at 
hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
        at hudson.model.Run.execute(Run.java:1819)
        at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
        at hudson.model.ResourceController.execute(ResourceController.java:97)
        at hudson.model.Executor.run(Executor.java:429)
No artifacts found that match the file pattern 
"**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to