Build: https://jenkins.thetaphi.de/job/Lucene-Solr-BadApples-master-Linux/141/
Java: 64bit/jdk-9.0.4 -XX:+UseCompressedOops -XX:+UseG1GC

1 tests failed.
FAILED:  
org.apache.solr.cloud.autoscaling.sim.TestSimTriggerIntegration.testNodeMarkersRegistration

Error Message:
Path /autoscaling/nodeAdded/127.0.0.1:10027_solr wasn't created

Stack Trace:
java.lang.AssertionError: Path /autoscaling/nodeAdded/127.0.0.1:10027_solr 
wasn't created
        at 
__randomizedtesting.SeedInfo.seed([AE068F72BF210EE1:B6BC077EB114C30E]:0)
        at org.junit.Assert.fail(Assert.java:88)
        at org.junit.Assert.assertTrue(Assert.java:41)
        at 
org.apache.solr.cloud.autoscaling.sim.TestSimTriggerIntegration.testNodeMarkersRegistration(TestSimTriggerIntegration.java:888)
        at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:844)




Build Log:
[...truncated 12745 lines...]
   [junit4] Suite: 
org.apache.solr.cloud.autoscaling.sim.TestSimTriggerIntegration
   [junit4]   2> Creating dataDir: 
/home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.sim.TestSimTriggerIntegration_AE068F72BF210EE1-001/init-core-data-001
   [junit4]   2> 121341 WARN  
(SUITE-TestSimTriggerIntegration-seed#[AE068F72BF210EE1]-worker) [    ] 
o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=2 numCloses=2
   [junit4]   2> 121342 INFO  
(SUITE-TestSimTriggerIntegration-seed#[AE068F72BF210EE1]-worker) [    ] 
o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) 
w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 121342 INFO  
(SUITE-TestSimTriggerIntegration-seed#[AE068F72BF210EE1]-worker) [    ] 
o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: 
@org.apache.solr.util.RandomizeSSL(reason="", ssl=0.0/0.0, value=0.0/0.0, 
clientAuth=0.0/0.0)
   [junit4]   2> 121343 INFO  
(SUITE-TestSimTriggerIntegration-seed#[AE068F72BF210EE1]-worker) [    ] 
o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: 
test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 121345 INFO  
(TEST-TestSimTriggerIntegration.testListeners-seed#[AE068F72BF210EE1]) [    ] 
o.a.s.SolrTestCaseJ4 ###Starting testListeners
   [junit4]   2> 121392 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and 
.scheduled_maintenance triggers
   [junit4]   2> 121392 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode 
version 1
   [junit4]   2> 121392 DEBUG 
(TEST-TestSimTriggerIntegration.testListeners-seed#[AE068F72BF210EE1]) [    ] 
o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 
127.0.0.1:10000_solr
   [junit4]   2> 121393 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 0
   [junit4]   2> 121393 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
   [junit4]   2> 121402 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial 
livenodes: [127.0.0.1:10000_solr]
   [junit4]   2> 121412 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers
   [junit4]   2> 121413 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
   [junit4]   2> 121413 DEBUG (ScheduledTrigger-522-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 1 and last live nodes: 1
   [junit4]   2> 121421 DEBUG 
(TEST-TestSimTriggerIntegration.testListeners-seed#[AE068F72BF210EE1]) [    ] 
o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, 
currentVersion=0
   [junit4]   2> 121429 DEBUG 
(TEST-TestSimTriggerIntegration.testListeners-seed#[AE068F72BF210EE1]) [    ] 
o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 0
   [junit4]   2> 121429 INFO  
(TEST-TestSimTriggerIntegration.testListeners-seed#[AE068F72BF210EE1]) [    ] 
o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history 
in memory.
   [junit4]   2> 121450 DEBUG (ScheduledTrigger-522-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 1
   [junit4]   2> 121454 DEBUG (simCloudManagerPool-521-thread-2) [    ] 
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 121455 DEBUG (simCloudManagerPool-521-thread-2) [    ] 
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 
2
   [junit4]   2> 121456 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
   [junit4]   2> 121456 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers
   [junit4]   2> 121456 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
   [junit4]   2> 121464 DEBUG (simCloudManagerPool-521-thread-3) [    ] 
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 121464 DEBUG (simCloudManagerPool-521-thread-3) [    ] 
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 
3
   [junit4]   2> 121465 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
   [junit4]   2> 121466 INFO  (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.s.TestSimTriggerIntegration TestTriggerAction init
   [junit4]   2> 121466 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - Initial 
livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
   [junit4]   2> 121466 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated 
with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, 
class=org.apache.solr.cloud.autoscaling.sim.TestSimTriggerIntegration$TestTriggerAction},
 {name=test1, 
class=org.apache.solr.cloud.autoscaling.sim.TestSimTriggerIntegration$TestDummyAction}],
 enabled=true}
   [junit4]   2> 121466 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers
   [junit4]   2> 121466 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
   [junit4]   2> 121466 DEBUG (ScheduledTrigger-522-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 121466 DEBUG (ScheduledTrigger-522-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
   [junit4]   2> 121468 DEBUG (simCloudManagerPool-521-thread-4) [    ] 
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 121469 DEBUG (simCloudManagerPool-521-thread-4) [    ] 
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 
4
   [junit4]   2> 121470 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
   [junit4]   2> 121470 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers
   [junit4]   2> 121470 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
   [junit4]   2> 121476 DEBUG (ScheduledTrigger-522-thread-4) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 121477 DEBUG (simCloudManagerPool-521-thread-5) [    ] 
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 121478 DEBUG (simCloudManagerPool-521-thread-5) [    ] 
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 
5
   [junit4]   2> 121479 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
   [junit4]   2> 121479 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers
   [junit4]   2> 121479 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
   [junit4]   2> 121480 DEBUG 
(TEST-TestSimTriggerIntegration.testListeners-seed#[AE068F72BF210EE1]) [    ] 
o.a.s.c.a.s.SimClusterStateProvider -- creating marker: 
/autoscaling/nodeAdded/127.0.0.1:10002_solr
   [junit4]   2> 121489 DEBUG (ScheduledTrigger-522-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 121489 DEBUG (ScheduledTrigger-522-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 121489 DEBUG (ScheduledTrigger-522-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10002_solr at time 
904194349666958
   [junit4]   2> 121497 DEBUG (ScheduledTrigger-522-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3 and last live nodes: 2
   [junit4]   2> 121509 DEBUG (ScheduledTrigger-522-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 121509 DEBUG (ScheduledTrigger-522-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 121509 DEBUG (ScheduledTrigger-522-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger firing 
registered processor for nodes: [127.0.0.1:10002_solr] added at times 
[904194349666958], now=904195368046908
   [junit4]   2> 121511 DEBUG (ScheduledTrigger-522-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"3365c2705128eT6rt02ywaos78pu0mw8fer5dzm",
   [junit4]   2>   "source":"node_added_trigger",
   [junit4]   2>   "eventTime":904194349666958,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[904194349666958],
   [junit4]   2>     "preferredOperation":"movereplica",
   [junit4]   2>     "nodeNames":["127.0.0.1:10002_solr"]}}
   [junit4]   2> 121511 DEBUG (ScheduledTrigger-522-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, 
node_added_trigger]
   [junit4]   2> 121512 DEBUG (simCloudManagerPool-521-thread-6) [    ] 
o.a.s.c.a.s.SimClusterStateProvider -- simCreateCollection .system, 
currentVersion=1
   [junit4]   2> 121513 DEBUG (simCloudManagerPool-521-thread-6) [    ] 
o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, 
currentVersion=1
   [junit4]   2> 121513 DEBUG (simCloudManagerPool-521-thread-6) [    ] 
o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 1
   [junit4]   2> 121530 DEBUG (simCloudManagerPool-521-thread-6) [    ] 
o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, 
currentVersion=2
   [junit4]   2> 121532 DEBUG (simCloudManagerPool-521-thread-6) [    ] 
o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 2
   [junit4]   2> 121550 DEBUG (simCloudManagerPool-521-thread-7) [    ] 
o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, 
currentVersion=3
   [junit4]   2> 121551 DEBUG (simCloudManagerPool-521-thread-7) [    ] 
o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 3
   [junit4]   2> 121564 DEBUG (simCloudManagerPool-521-thread-8) [    ] 
o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, 
currentVersion=4
   [junit4]   2> 121565 DEBUG (simCloudManagerPool-521-thread-8) [    ] 
o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 4
   [junit4]   2> 121575 DEBUG (simCloudManagerPool-521-thread-9) [    ] 
o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, 
currentVersion=5
   [junit4]   2> 121575 DEBUG (simCloudManagerPool-521-thread-9) [    ] 
o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 5
   [junit4]   2> 121576 DEBUG (simCloudManagerPool-521-thread-6) [    ] 
o.a.s.c.a.s.SimClusterStateProvider -- finished createCollection .system, 
currentVersion=6
   [junit4]   2> 121577 INFO  (simCloudManagerPool-521-thread-6) [    ] 
o.a.s.c.CloudTestUtils -- shards without leaders: [shard1]
   [junit4]   2> 121577 DEBUG (simCloudManagerPool-521-thread-10) [    ] 
o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for .system / shard1 
(currentVersion=6): {"core_node1":{
   [junit4]   2>     "core":".system_shard1_replica_n1",
   [junit4]   2>     "shard":"shard1",
   [junit4]   2>     "collection":".system",
   [junit4]   2>     "node_name":"127.0.0.1:10002_solr",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "state":"active",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>     "SEARCHER.searcher.numDocs":0}}
   [junit4]   2> 121579 DEBUG (simCloudManagerPool-521-thread-6) [    ] 
o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, 
currentVersion=6
   [junit4]   2> 121579 DEBUG (simCloudManagerPool-521-thread-6) [    ] 
o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 6
   [junit4]   2> 121582 DEBUG (ScheduledTrigger-522-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Removing new node from tracking: 127.0.0.1:10002_solr
   [junit4]   2> 121582 DEBUG (AutoscalingActionExecutor-523-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"3365c2705128eT6rt02ywaos78pu0mw8fer5dzm",
   [junit4]   2>   "source":"node_added_trigger",
   [junit4]   2>   "eventTime":904194349666958,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[904194349666958],
   [junit4]   2>     "preferredOperation":"movereplica",
   [junit4]   2>     "_enqueue_time_":904195463587958,
   [junit4]   2>     "nodeNames":["127.0.0.1:10002_solr"]}}
   [junit4]   2> 121587 DEBUG (AutoscalingActionExecutor-523-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
   [junit4]   2> 121587 DEBUG (AutoscalingActionExecutor-523-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger after 100ms
   [junit4]   2> 121587 DEBUG (AutoscalingActionExecutor-523-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers -- processing took 256 ms for event 
id=3365c2705128eT6rt02ywaos78pu0mw8fer5dzm
   [junit4]   2> 121645 DEBUG 
(TEST-TestSimTriggerIntegration.testListeners-seed#[AE068F72BF210EE1]) [    ] 
o.a.s.c.a.s.SimClusterStateProvider -- creating marker: 
/autoscaling/nodeAdded/127.0.0.1:10003_solr
   [junit4]   2> 121688 DEBUG (ScheduledTrigger-522-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 4 and last live nodes: 3
   [junit4]   2> 121688 DEBUG (ScheduledTrigger-522-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 121688 DEBUG (ScheduledTrigger-522-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
   [junit4]   2> 121688 DEBUG (ScheduledTrigger-522-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10003_solr at time 
904204287342458
   [junit4]   2> 121708 DEBUG (ScheduledTrigger-522-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 121708 DEBUG (ScheduledTrigger-522-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 4 and last live nodes: 4
   [junit4]   2> 121708 DEBUG (ScheduledTrigger-522-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
   [junit4]   2> 121708 DEBUG (ScheduledTrigger-522-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger firing 
registered processor for nodes: [127.0.0.1:10003_solr] added at times 
[904204287342458], now=904205301477758
   [junit4]   2> 121708 DEBUG (ScheduledTrigger-522-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"3365e7759f77aT6rt02ywaos78pu0mw8fer5dzp",
   [junit4]   2>   "source":"node_added_trigger",
   [junit4]   2>   "eventTime":904204287342458,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[904204287342458],
   [junit4]   2>     "preferredOperation":"movereplica",
   [junit4]   2>     "nodeNames":["127.0.0.1:10003_solr"]}}
   [junit4]   2> 121708 DEBUG (ScheduledTrigger-522-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, 
node_added_trigger]
   [junit4]   2> 121709 DEBUG (ScheduledTrigger-522-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Removing new node from tracking: 127.0.0.1:10003_solr
   [junit4]   2> 121709 DEBUG (AutoscalingActionExecutor-523-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"3365e7759f77aT6rt02ywaos78pu0mw8fer5dzp",
   [junit4]   2>   "source":"node_added_trigger",
   [junit4]   2>   "eventTime":904204287342458,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[904204287342458],
   [junit4]   2>     "preferredOperation":"movereplica",
   [junit4]   2>     "_enqueue_time_":904205315638808,
   [junit4]   2>     "nodeNames":["127.0.0.1:10003_solr"]}}
   [junit4]   2> 121713 WARN  (AutoscalingActionExecutor-523-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers Exception executing actions
   [junit4]   2> org.apache.solr.cloud.autoscaling.TriggerActionException: 
Error processing action for trigger event: {
   [junit4]   2>   "id":"3365e7759f77aT6rt02ywaos78pu0mw8fer5dzp",
   [junit4]   2>   "source":"node_added_trigger",
   [junit4]   2>   "eventTime":904204287342458,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[904204287342458],
   [junit4]   2>     "preferredOperation":"movereplica",
   [junit4]   2>     "_enqueue_time_":904205315638808,
   [junit4]   2>     "nodeNames":["127.0.0.1:10003_solr"]}}
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$add$3(ScheduledTriggers.java:314)
 [java/:?]
   [junit4]   2>        at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:514) [?:?]
   [junit4]   2>        at 
java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
   [junit4]   2>        at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
 [java/:?]
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167) 
[?:?]
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641) 
[?:?]
   [junit4]   2>        at java.lang.Thread.run(Thread.java:844) [?:?]
   [junit4]   2> Caused by: java.lang.RuntimeException: failure
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.sim.TestSimTriggerIntegration$TestDummyAction.process(TestSimTriggerIntegration.java:943)
 ~[test/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$add$3(ScheduledTriggers.java:311)
 ~[java/:?]
   [junit4]   2>        ... 6 more
   [junit4]   2> 121714 DEBUG (AutoscalingActionExecutor-523-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
   [junit4]   2> 121714 DEBUG (AutoscalingActionExecutor-523-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger after 100ms
   [junit4]   2> 121714 DEBUG (AutoscalingActionExecutor-523-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers -- processing took 252 ms for event 
id=3365e7759f77aT6rt02ywaos78pu0mw8fer5dzp
   [junit4]   2> 121770 DEBUG 
(TEST-TestSimTriggerIntegration.testListeners-seed#[AE068F72BF210EE1]) [    ] 
o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 121770 DEBUG 
(TEST-TestSimTriggerIntegration.testListeners-seed#[AE068F72BF210EE1]) [    ] 
o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 121770 INFO  (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are 
closed, exiting.
   [junit4]   2> 121770 DEBUG 
(TEST-TestSimTriggerIntegration.testListeners-seed#[AE068F72BF210EE1]) [    ] 
o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 121770 DEBUG 
(TEST-TestSimTriggerIntegration.testListeners-seed#[AE068F72BF210EE1]) [    ] 
o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool 
executor
   [junit4]   2> 121771 DEBUG 
(TEST-TestSimTriggerIntegration.testListeners-seed#[AE068F72BF210EE1]) [    ] 
o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 121771 DEBUG 
(TEST-TestSimTriggerIntegration.testListeners-seed#[AE068F72BF210EE1]) [    ] 
o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 121771 INFO  
(TEST-TestSimTriggerIntegration.testListeners-seed#[AE068F72BF210EE1]) [    ] 
o.a.s.SolrTestCaseJ4 ###Ending testListeners
   [junit4]   2> 121773 INFO  
(TEST-TestSimTriggerIntegration.testCooldown-seed#[AE068F72BF210EE1]) [    ] 
o.a.s.SolrTestCaseJ4 ###Starting testCooldown
   [junit4]   2> 121776 DEBUG 
(TEST-TestSimTriggerIntegration.testCooldown-seed#[AE068F72BF210EE1]) [    ] 
o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 
127.0.0.1:10004_solr
   [junit4]   2> 121776 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and 
.scheduled_maintenance triggers
   [junit4]   2> 121776 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode 
version 1
   [junit4]   2> 121777 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 0
   [junit4]   2> 121777 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
   [junit4]   2> 121782 DEBUG 
(TEST-TestSimTriggerIntegration.testCooldown-seed#[AE068F72BF210EE1]) [    ] 
o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, 
currentVersion=0
   [junit4]   2> 121782 DEBUG 
(TEST-TestSimTriggerIntegration.testCooldown-seed#[AE068F72BF210EE1]) [    ] 
o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 0
   [junit4]   2> 121782 INFO  
(TEST-TestSimTriggerIntegration.testCooldown-seed#[AE068F72BF210EE1]) [    ] 
o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history 
in memory.
   [junit4]   2> 121782 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial 
livenodes: [127.0.0.1:10004_solr]
   [junit4]   2> 121783 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers
   [junit4]   2> 121783 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
   [junit4]   2> 121783 DEBUG (ScheduledTrigger-528-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 1
   [junit4]   2> 121791 DEBUG (simCloudManagerPool-527-thread-2) [    ] 
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 121791 DEBUG (simCloudManagerPool-527-thread-2) [    ] 
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 
2
   [junit4]   2> 121791 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
   [junit4]   2> 121794 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers
   [junit4]   2> 121794 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
   [junit4]   2> 121795 DEBUG (simCloudManagerPool-527-thread-3) [    ] 
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 121795 DEBUG (simCloudManagerPool-527-thread-3) [    ] 
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 
3
   [junit4]   2> 121795 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
   [junit4]   2> 121796 INFO  (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.s.TestSimTriggerIntegration TestTriggerAction init
   [junit4]   2> 121796 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_cooldown_trigger - 
Initial livenodes: [127.0.0.1:10004_solr, 127.0.0.1:10005_solr]
   [junit4]   2> 121796 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_cooldown_trigger 
instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, 
class=org.apache.solr.cloud.autoscaling.sim.TestSimTriggerIntegration$TestTriggerAction}],
 enabled=true}
   [junit4]   2> 121796 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers
   [junit4]   2> 121796 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
   [junit4]   2> 121796 DEBUG (ScheduledTrigger-528-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_cooldown_trigger
   [junit4]   2> 121796 DEBUG (ScheduledTrigger-528-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
   [junit4]   2> 121796 DEBUG (simCloudManagerPool-527-thread-4) [    ] 
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 121796 DEBUG (simCloudManagerPool-527-thread-4) [    ] 
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 
4
   [junit4]   2> 121798 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
   [junit4]   2> 121798 DEBUG 
(TEST-TestSimTriggerIntegration.testCooldown-seed#[AE068F72BF210EE1]) [    ] 
o.a.s.c.a.s.SimClusterStateProvider -- creating marker: 
/autoscaling/nodeAdded/127.0.0.1:10006_solr
   [junit4]   2> 121798 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers
   [junit4]   2> 121798 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
   [junit4]   2> 121803 DEBUG (ScheduledTrigger-528-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3 and last live nodes: 2
   [junit4]   2> 121817 DEBUG (ScheduledTrigger-528-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_cooldown_trigger
   [junit4]   2> 121817 DEBUG (ScheduledTrigger-528-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 121817 DEBUG (ScheduledTrigger-528-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10006_solr at time 
904210745012808
   [junit4]   2> 121823 DEBUG (ScheduledTrigger-528-thread-4) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3 and last live nodes: 3
   [junit4]   2> 121837 DEBUG (ScheduledTrigger-528-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_cooldown_trigger
   [junit4]   2> 121837 DEBUG (ScheduledTrigger-528-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 121837 DEBUG (ScheduledTrigger-528-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_cooldown_trigger firing 
registered processor for nodes: [127.0.0.1:10006_solr] added at times 
[904210745012808], now=904211764127308
   [junit4]   2> 121837 DEBUG (ScheduledTrigger-528-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"3365ff8423248T6rt02ywaos78pu0mw8fer5dzt",
   [junit4]   2>   "source":"node_added_cooldown_trigger",
   [junit4]   2>   "eventTime":904210745012808,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[904210745012808],
   [junit4]   2>     "preferredOperation":"movereplica",
   [junit4]   2>     "nodeNames":["127.0.0.1:10006_solr"]}}
   [junit4]   2> 121838 DEBUG (ScheduledTrigger-528-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, 
node_added_cooldown_trigger]
   [junit4]   2> 121838 DEBUG (simCloudManagerPool-527-thread-5) [    ] 
o.a.s.c.a.s.SimClusterStateProvider -- simCreateCollection .system, 
currentVersion=1
   [junit4]   2> 121838 DEBUG (simCloudManagerPool-527-thread-5) [    ] 
o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, 
currentVersion=1
   [junit4]   2> 121838 DEBUG (simCloudManagerPool-527-thread-5) [    ] 
o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 1
   [junit4]   2> 121848 DEBUG (simCloudManagerPool-527-thread-5) [    ] 
o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, 
currentVersion=2
   [junit4]   2> 121849 DEBUG (simCloudManagerPool-527-thread-5) [    ] 
o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 2
   [junit4]   2> 121860 DEBUG (simCloudManagerPool-527-thread-6) [    ] 
o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, 
currentVersion=3
   [junit4]   2> 121860 DEBUG (simCloudManagerPool-527-thread-6) [    ] 
o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 3
   [junit4]   2> 121871 DEBUG (simCloudManagerPool-527-thread-8) [    ] 
o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, 
currentVersion=4
   [junit4]   2> 121871 DEBUG (simCloudManagerPool-527-thread-8) [    ] 
o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 4
   [junit4]   2> 121881 DEBUG (simCloudManagerPool-527-thread-7) [    ] 
o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, 
currentVersion=5
   [junit4]   2> 121882 DEBUG (simCloudManagerPool-527-thread-7) [    ] 
o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 5
   [junit4]   2> 121882 DEBUG (simCloudManagerPool-527-thread-5) [    ] 
o.a.s.c.a.s.SimClusterStateProvider -- finished createCollection .system, 
currentVersion=6
   [junit4]   2> 121882 INFO  (simCloudManagerPool-527-thread-5) [    ] 
o.a.s.c.CloudTestUtils -- shards without leaders: [shard1]
   [junit4]   2> 121882 DEBUG (simCloudManagerPool-527-thread-9) [    ] 
o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for .system / shard1 
(currentVersion=6): {"core_node1":{
   [junit4]   2>     "core":".system_shard1_replica_n1",
   [junit4]   2>     "shard":"shard1",
   [junit4]   2>     "collection":".system",
   [junit4]   2>     "node_name":"127.0.0.1:10005_solr",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "state":"active",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>     "SEARCHER.searcher.numDocs":0}}
   [junit4]   2> 121884 DEBUG (simCloudManagerPool-527-thread-5) [    ] 
o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, 
currentVersion=6
   [junit4]   2> 121885 DEBUG (simCloudManagerPool-527-thread-5) [    ] 
o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 6
   [junit4]   2> 121890 DEBUG (ScheduledTrigger-528-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Removing new node from tracking: 127.0.0.1:10006_solr
   [junit4]   2> 121890 DEBUG (AutoscalingActionExecutor-529-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"3365ff8423248T6rt02ywaos78pu0mw8fer5dzt",
   [junit4]   2>   "source":"node_added_cooldown_trigger",
   [junit4]   2>   "eventTime":904210745012808,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[904210745012808],
   [junit4]   2>     "preferredOperation":"movereplica",
   [junit4]   2>     "_enqueue_time_":904211784407858,
   [junit4]   2>     "nodeNames":["127.0.0.1:10006_solr"]}}
   [junit4]   2> 121892 DEBUG (AutoscalingActionExecutor-529-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
   [junit4]   2> 121892 DEBUG (AutoscalingActionExecutor-529-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_cooldown_trigger after 
100ms
   [junit4]   2> 121892 DEBUG (AutoscalingActionExecutor-529-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers -- processing took 112 ms for event 
id=3365ff8423248T6rt02ywaos78pu0mw8fer5dzt
   [junit4]   2> 121991 DEBUG 
(TEST-TestSimTriggerIntegration.testCooldown-seed#[AE068F72BF210EE1]) [    ] 
o.a.s.c.a.s.SimClusterStateProvider -- creating marker: 
/autoscaling/nodeAdded/127.0.0.1:10007_solr
   [junit4]   2> 121992 DEBUG (ScheduledTrigger-528-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_cooldown_trigger
   [junit4]   2> 121992 DEBUG (ScheduledTrigger-528-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 4 and last live nodes: 3
   [junit4]   2> 121992 DEBUG (ScheduledTrigger-528-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
   [junit4]   2> 121992 DEBUG (ScheduledTrigger-528-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10007_solr at time 
904219528682358
   [junit4]   2> 122013 DEBUG (ScheduledTrigger-528-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 4 and last live nodes: 4
   [junit4]   2> 122013 DEBUG (ScheduledTrigger-528-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_cooldown_trigger
   [junit4]   2> 122013 DEBUG (ScheduledTrigger-528-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
   [junit4]   2> 122013 DEBUG (ScheduledTrigger-528-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_cooldown_trigger firing 
registered processor for nodes: [127.0.0.1:10007_solr] added at times 
[904219528682358], now=904220546341958
   [junit4]   2> 122013 DEBUG (ScheduledTrigger-528-thread-4) [    ] 
o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"3366203ce5b76T6rt02ywaos78pu0mw8fer5dzw",
   [junit4]   2>   "source":"node_added_cooldown_trigger",
   [junit4]   2>   "eventTime":904219528682358,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[904219528682358],
   [junit4]   2>     "preferredOperation":"movereplica",
   [junit4]   2>     "nodeNames":["127.0.0.1:10007_solr"]}}
   [junit4]   2> 122013 DEBUG (ScheduledTrigger-528-thread-4) [    ] 
o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, 
node_added_cooldown_trigger]
   [junit4]   2> 122014 DEBUG (ScheduledTrigger-528-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Removing new node from tracking: 127.0.0.1:10007_solr
   [junit4]   2> 122014 DEBUG (AutoscalingActionExecutor-529-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"3366203ce5b76T6rt02ywaos78pu0mw8fer5dzw",
   [junit4]   2>   "source":"node_added_cooldown_trigger",
   [junit4]   2>   "eventTime":904219528682358,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[904219528682358],
   [junit4]   2>     "preferredOperation":"movereplica",
   [junit4]   2>     "_enqueue_time_":904220558135708,
   [junit4]   2>     "nodeNames":["127.0.0.1:10007_solr"]}}
   [junit4]   2> 122014 DEBUG (AutoscalingActionExecutor-529-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
   [junit4]   2> 122014 DEBUG (AutoscalingActionExecutor-529-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_cooldown_trigger after 
100ms
   [junit4]   2> 122014 DEBUG (AutoscalingActionExecutor-529-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers -- processing took 27 ms for event 
id=3366203ce5b76T6rt02ywaos78pu0mw8fer5dzw
   [junit4]   2> 122114 DEBUG (ScheduledTrigger-528-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_cooldown_trigger
   [junit4]   2> 122114 DEBUG (ScheduledTrigger-528-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 4 and last live nodes: 4
   [junit4]   2> 122114 DEBUG (ScheduledTrigger-528-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
   [junit4]   2> 122134 DEBUG (ScheduledTrigger-528-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_cooldown_trigger
   [junit4]   2> 122134 DEBUG (ScheduledTrigger-528-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 4 and last live nodes: 4
   [junit4]   2> 122135 DEBUG (ScheduledTrigger-528-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
   [junit4]   2> 122155 DEBUG (ScheduledTrigger-528-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 4 and last live nodes: 4
   [junit4]   2> 122155 DEBUG (ScheduledTrigger-528-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_cooldown_trigger
   [junit4]   2> 122155 DEBUG (ScheduledTrigger-528-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
   [junit4]   2> 122174 DEBUG 
(TEST-TestSimTriggerIntegration.testCooldown-seed#[AE068F72BF210EE1]) [    ] 
o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 122174 INFO  (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are 
closed, exiting.
   [junit4]   2> 122175 DEBUG 
(TEST-TestSimTriggerIntegration.testCooldown-seed#[AE068F72BF210EE1]) [    ] 
o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 122175 DEBUG 
(TEST-TestSimTriggerIntegration.testCooldown-seed#[AE068F72BF210EE1]) [    ] 
o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 122175 DEBUG 
(TEST-TestSimTriggerIntegration.testCooldown-seed#[AE068F72BF210EE1]) [    ] 
o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool 
executor
   [junit4]   2> 122175 DEBUG 
(TEST-TestSimTriggerIntegration.testCooldown-seed#[AE068F72BF210EE1]) [    ] 
o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 122175 DEBUG 
(TEST-TestSimTriggerIntegration.testCooldown-seed#[AE068F72BF210EE1]) [    ] 
o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 122175 INFO  
(TEST-TestSimTriggerIntegration.testCooldown-seed#[AE068F72BF210EE1]) [    ] 
o.a.s.SolrTestCaseJ4 ###Ending testCooldown
   [junit4]   2> 122177 INFO  
(TEST-TestSimTriggerIntegration.testEventFromRestoredState-seed#[AE068F72BF210EE1])
 [    ] o.a.s.SolrTestCaseJ4 ###Starting testEventFromRestoredState
   [junit4]   2> 122180 DEBUG 
(TEST-TestSimTriggerIntegration.testEventFromRestoredState-seed#[AE068F72BF210EE1])
 [    ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 
127.0.0.1:10008_solr
   [junit4]   2> 122180 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and 
.scheduled_maintenance triggers
   [junit4]   2> 122180 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode 
version 1
   [junit4]   2> 122180 DEBUG 
(TEST-TestSimTriggerIntegration.testEventFromRestoredState-seed#[AE068F72BF210EE1])
 [    ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, 
currentVersion=0
   [junit4]   2> 122180 DEBUG 
(TEST-TestSimTriggerIntegration.testEventFromRestoredState-seed#[AE068F72BF210EE1])
 [    ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 0
   [junit4]   2> 122180 INFO  
(TEST-TestSimTriggerIntegration.testEventFromRestoredState-seed#[AE068F72BF210EE1])
 [    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics 
history in memory.
   [junit4]   2> 122181 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 0
   [junit4]   2> 122181 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
   [junit4]   2> 122181 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial 
livenodes: [127.0.0.1:10009_solr, 127.0.0.1:10008_solr]
   [junit4]   2> 122182 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers
   [junit4]   2> 122182 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
   [junit4]   2> 122182 DEBUG (ScheduledTrigger-534-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 122182 DEBUG (simCloudManagerPool-533-thread-2) [    ] 
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 122182 DEBUG (simCloudManagerPool-533-thread-2) [    ] 
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 
2
   [junit4]   2> 122183 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
   [junit4]   2> 122184 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers
   [junit4]   2> 122184 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
   [junit4]   2> 122185 DEBUG (simCloudManagerPool-533-thread-3) [    ] 
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 122185 DEBUG (simCloudManagerPool-533-thread-3) [    ] 
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 
3
   [junit4]   2> 122186 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
   [junit4]   2> 122186 INFO  (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.s.TestSimTriggerIntegration TestTriggerAction init
   [junit4]   2> 122186 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - Initial 
livenodes: [127.0.0.1:10009_solr, 127.0.0.1:10008_solr]
   [junit4]   2> 122186 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated 
with properties: {event=nodeAdded, waitFor=10, actions=[{name=test, 
class=org.apache.solr.cloud.autoscaling.sim.TestSimTriggerIntegration$TestTriggerAction}],
 enabled=true}
   [junit4]   2> 122186 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers
   [junit4]   2> 122186 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
   [junit4]   2> 122186 DEBUG 
(TEST-TestSimTriggerIntegration.testEventFromRestoredState-seed#[AE068F72BF210EE1])
 [    ] o.a.s.c.a.s.SimClusterStateProvider -- creating marker: 
/autoscaling/nodeAdded/127.0.0.1:10010_solr
   [junit4]   2> 122186 DEBUG (ScheduledTrigger-534-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 122186 DEBUG (ScheduledTrigger-534-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 122186 DEBUG (ScheduledTrigger-534-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10010_solr at time 
904229215353358
   [junit4]   2> 122202 DEBUG (ScheduledTrigger-534-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3 and last live nodes: 2
   [junit4]   2> 122207 DEBUG (ScheduledTrigger-534-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 122207 DEBUG (ScheduledTrigger-534-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 122222 DEBUG (ScheduledTrigger-534-thread-4) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3 and last live nodes: 3
   [junit4]   2> 122227 DEBUG (ScheduledTrigger-534-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 122227 DEBUG (ScheduledTrigger-534-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 122243 DEBUG (ScheduledTrigger-534-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3 and last live nodes: 3
   [junit4]   2> 122247 DEBUG (ScheduledTrigger-534-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 122247 DEBUG (ScheduledTrigger-534-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 122263 DEBUG (ScheduledTrigger-534-thread-4) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3 and last live nodes: 3
   [junit4]   2> 122268 DEBUG (ScheduledTrigger-534-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 122268 DEBUG (ScheduledTrigger-534-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 122283 DEBUG (ScheduledTrigger-534-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3 and last live nodes: 3
   [junit4]   2> 122288 DEBUG (ScheduledTrigger-534-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 122288 DEBUG (ScheduledTrigger-534-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 122304 DEBUG (ScheduledTrigger-534-thread-4) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3 and last live nodes: 3
   [junit4]   2> 122308 DEBUG (ScheduledTrigger-534-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 122308 DEBUG (ScheduledTrigger-534-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 122324 DEBUG (ScheduledTrigger-534-thread-4) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3 and last live nodes: 3
   [junit4]   2> 122328 DEBUG (ScheduledTrigger-534-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 122328 DEBUG (ScheduledTrigger-534-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 122345 DEBUG (ScheduledTrigger-534-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3 and last live nodes: 3
   [junit4]   2> 122349 DEBUG (ScheduledTrigger-534-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 122349 DEBUG (ScheduledTrigger-534-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 122365 DEBUG (ScheduledTrigger-534-thread-4) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3 and last live nodes: 3
   [junit4]   2> 122369 DEBUG (ScheduledTrigger-534-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 122369 DEBUG (ScheduledTrigger-534-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 122385 DEBUG (ScheduledTrigger-534-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3 and last live nodes: 3
   [junit4]   2> 122389 DEBUG (ScheduledTrigger-534-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 122389 DEBUG (ScheduledTrigger-534-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 122389 DEBUG (ScheduledTrigger-534-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger firing 
registered processor for nodes: [127.0.0.1:10010_solr] added at times 
[904229215353358], now=904239370953908
   [junit4]   2> 122389 DEBUG (ScheduledTrigger-534-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"33664452d3a0eT6rt02ywaos78pu0mw8fer5e00",
   [junit4]   2>   "source":"node_added_trigger",
   [junit4]   2>   "eventTime":904229215353358,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[904229215353358],
   [junit4]   2>     "preferredOperation":"movereplica",
   [junit4]   2>     "nodeNames":["127.0.0.1:10010_solr"]}}
   [junit4]   2> 122390 DEBUG (ScheduledTrigger-534-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, 
node_added_trigger]
   [junit4]   2> 122390 DEBUG (simCloudManagerPool-533-thread-4) [    ] 
o.a.s.c.a.s.SimClusterStateProvider -- simCreateCollection .system, 
currentVersion=1
   [junit4]   2> 122390 DEBUG (simCloudManagerPool-533-thread-4) [    ] 
o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, 
currentVersion=1
   [junit4]   2> 122390 DEBUG (simCloudManagerPool-533-thread-4) [    ] 
o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 1
   [junit4]   2> 122400 DEBUG (simCloudManagerPool-533-thread-4) [    ] 
o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, 
currentVersion=2
   [junit4]   2> 122401 DEBUG (simCloudManagerPool-533-thread-4) [    ] 
o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 2
   [junit4]   2> 122412 DEBUG (simCloudManagerPool-533-thread-5) [    ] 
o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, 
currentVersion=3
   [junit4]   2> 122412 DEBUG (simCloudManagerPool-533-thread-5) [    ] 
o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 3
   [junit4]   2> 122422 DEBUG (simCloudManagerPool-533-thread-6) [    ] 
o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, 
currentVersion=4
   [junit4]   2> 122423 DEBUG (simCloudManagerPool-533-thread-6) [    ] 
o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 4
   [junit4]   2> 122433 DEBUG (simCloudManagerPool-533-thread-7) [    ] 
o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, 
currentVersion=5
   [junit4]   2> 122434 DEBUG (simCloudManagerPool-533-thread-7) [    ] 
o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 5
   [junit4]   2> 122434 DEBUG (simCloudManagerPool-533-thread-4) [    ] 
o.a.s.c.a.s.SimClusterStateProvider -- finished createCollection .system, 
currentVersion=6
   [junit4]   2> 122434 INFO  (simCloudManagerPool-533-thread-4) [    ] 
o.a.s.c.CloudTestUtils -- shards without leaders: [shard1]
   [junit4]   2> 122434 DEBUG (simCloudManagerPool-533-thread-8) [    ] 
o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for .system / shard1 
(currentVersion=6): {"core_node1":{
   [junit4]   2>     "core":".system_shard1_replica_n1",
   [junit4]   2>     "shard":"shard1",
   [junit4]   2>     "collection":".system",
   [junit4]   2>     "node_name":"127.0.0.1:10008_solr",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "state":"active",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>     "SEARCHER.searcher.numDocs":0}}
   [junit4]   2> 122451 DEBUG (simCloudManagerPool-533-thread-4) [    ] 
o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, 
currentVersion=6
   [junit4]   2> 122451 DEBUG (simCloudManagerPool-533-thread-4) [    ] 
o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 6
   [junit4]   2> 122453 DEBUG (ScheduledTrigger-534-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Removing new node from tracking: 127.0.0.1:10010_solr
   [junit4]   2> 122453 DEBUG (AutoscalingActionExecutor-535-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"33664452d3a0eT6rt02ywaos78pu0mw8fer5e00",
   [junit4]   2>   "source":"node_added_trigger",
   [junit4]   2>   "eventTime":904229215353358,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[904229215353358],
   [junit4]   2>     "preferredOperation":"movereplica",
   [junit4]   2>     "_enqueue_time_":904239389448858,
   [junit4]   2>     "nodeNames":["127.0.0.1:10010_solr"]}}
   [junit4]   2> 122453 DEBUG 
(TEST-TestSimTriggerIntegration.testEventFromRestoredState-seed#[AE068F72BF210EE1])
 [    ] o.a.s.c.a.s.SimClusterStateProvider -- creating marker: 
/autoscaling/nodeAdded/127.0.0.1:10011_solr
   [junit4]   2> 122458 DEBUG (AutoscalingActionExecutor-535-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
   [junit4]   2> 122458 DEBUG (AutoscalingActionExecutor-535-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger after 100ms
   [junit4]   2> 122458 DEBUG (AutoscalingActionExecutor-535-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers -- processing took 268 ms for event 
id=33664452d3a0eT6rt02ywaos78pu0mw8fer5e00
   [junit4]   2> 122559 DEBUG (ScheduledTrigger-534-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 4 and last live nodes: 3
   [junit4]   2> 122559 DEBUG (ScheduledTrigger-534-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 122559 DEBUG (ScheduledTrigger-534-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
   [junit4]   2> 122559 DEBUG (ScheduledTrigger-534-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10011_solr at time 
904247839758808
   [junit4]   2> 122579 DEBUG (ScheduledTrigger-534-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 4 and last live nodes: 4
   [junit4]   2> 122579 DEBUG (ScheduledTrigger-534-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 122579 DEBUG (ScheduledTrigger-534-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
   [junit4]   2> 122599 DEBUG (ScheduledTrigger-534-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 122599 DEBUG (ScheduledTrigger-534-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 4 and last live nodes: 4
   [junit4]   2> 122599 DEBUG (ScheduledTrigger-534-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
   [junit4]   2> 122619 DEBUG (ScheduledTrigger-534-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 122619 DEBUG (ScheduledTrigger-534-thread-4) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 4 and last live nodes: 4
   [junit4]   2> 122619 DEBUG (ScheduledTrigger-534-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
   [junit4]   2> 122640 DEBUG (ScheduledTrigger-534-thread-4) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 4 and last live nodes: 4
   [junit4]   2> 122640 DEBUG (ScheduledTrigger-534-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 122640 DEBUG (ScheduledTrigger-534-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
   [junit4]   2> 122654 INFO  
(TEST-TestSimTriggerIntegration.testEventFromRestoredState-seed#[AE068F72BF210EE1])
 [    ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and 
clearing object cache...
   [junit4]   2> 122654 INFO  (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are 
closed, exiting.
   [junit4]   2> 122654 DEBUG 
(TEST-TestSimTriggerIntegration.testEventFromRestoredState-seed#[AE068F72BF210EE1])
 [    ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool 
executor now
   [junit4]   2> 122654 DEBUG 
(TEST-TestSimTriggerIntegration.testEventFromRestoredState-seed#[AE068F72BF210EE1])
 [    ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 122654 DEBUG 
(TEST-TestSimTriggerIntegration.testEventFromRestoredState-seed#[AE068F72BF210EE1])
 [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 122654 DEBUG 
(TEST-TestSimTriggerIntegration.testEventFromRestoredState-seed#[AE068F72BF210EE1])
 [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread 
pool executor
   [junit4]   2> 122655 DEBUG 
(TEST-TestSimTriggerIntegration.testEventFromRestoredState-seed#[AE068F72BF210EE1])
 [    ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 122655 DEBUG 
(TEST-TestSimTriggerIntegration.testEventFromRestoredState-seed#[AE068F72BF210EE1])
 [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed 
explicitly
   [junit4]   2> 122655 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode 
version 3
   [junit4]   2> 122656 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 0
   [junit4]   2> 122656 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
   [junit4]   2> 122656 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial 
livenodes: [127.0.0.1:10010_solr, 127.0.0.1:10009_solr, 127.0.0.1:10008_solr, 
127.0.0.1:10011_solr]
   [junit4]   2> 122656 INFO  (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.s.TestSimTriggerIntegration TestTriggerAction init
   [junit4]   2> 122656 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - Initial 
livenodes: [127.0.0.1:10010_solr, 127.0.0.1:10009_solr, 127.0.0.1:10008_solr, 
127.0.0.1:10011_solr]
   [junit4]   2> 122656 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated 
with properties: {event=nodeAdded, waitFor=10, actions=[{name=test, 
class=org.apache.solr.cloud.autoscaling.sim.TestSimTriggerIntegration$TestTriggerAction}],
 enabled=true}
   [junit4]   2> 122656 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.NodeAddedTrigger Adding node from marker path: 127.0.0.1:10010_solr
   [junit4]   2> 122656 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.NodeAddedTrigger Adding node from marker path: 127.0.0.1:10011_solr
   [junit4]   2> 122656 DEBUG (ScheduledTrigger-539-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 4 and last live nodes: 4
   [junit4]   2> 122658 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers
   [junit4]   2> 122658 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
   [junit4]   2> 122658 DEBUG (ScheduledTrigger-539-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 122658 DEBUG (ScheduledTrigger-539-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
   [junit4]   2> 122676 DEBUG (ScheduledTrigger-539-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 4 and last live nodes: 4
   [junit4]   2> 122679 DEBUG (ScheduledTrigger-539-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 122679 DEBUG (ScheduledTrigger-539-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
   [junit4]   2> 122697 DEBUG (ScheduledTrigger-539-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 4 and last live nodes: 4
   [junit4]   2> 122699 DEBUG (ScheduledTrigger-539-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 122699 DEBUG (ScheduledTrigger-539-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
   [junit4]   2> 122717 DEBUG (ScheduledTrigger-539-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 4 and last live nodes: 4
   [junit4]   2> 122719 DEBUG (ScheduledTrigger-539-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 122719 DEBUG (ScheduledTrigger-539-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
   [junit4]   2> 122737 DEBUG (ScheduledTrigger-539-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 4 and last live nodes: 4
   [junit4]   2> 122739 DEBUG (ScheduledTrigger-539-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 122739 DEBUG (ScheduledTrigger-539-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
   [junit4]   2> 122758 DEBUG (ScheduledTrigger-539-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 4 and last live nodes: 4
   [junit4]   2> 122760 DEBUG (ScheduledTrigger-539-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 122760 DEBUG (ScheduledTrigger-539-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
   [junit4]   2> 122760 DEBUG (ScheduledTrigger-539-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger firing 
registered processor for nodes: [127.0.0.1:10011_solr] added at times 
[904247839758808], now=904257888451708
   [junit4]   2> 122760 DEBUG (ScheduledTrigger-539-thread-3) [    ] 
o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"336689b4719d8T6rt02ywaos78pu0mw8fer5e03",
   [junit4]   2>   "source":"node_added_trigger",
   [junit4]   2>   "eventTime":904247839758808,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[904247839758808],
   [junit4]   2>     "preferredOperation":"movereplica",
   [junit4]   2>     "nodeNames":["127.0.0.1:10011_solr"]}}
   [junit4]   2> 122760 DEBUG (ScheduledTrigger-539-thread-3) [    ] 
o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, 
node_added_trigger]
   [junit4]   2> 122761 DEBUG (ScheduledTrigger-539-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Removing new node from tracking: 127.0.0.1:10011_solr
   [junit4]   2> 122761 DEBUG (AutoscalingActionExecutor-540-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"336689b4719d8T6rt02ywaos78pu0mw8fer5e03",
   [junit4]   2>   "source":"node_added_trigger",
   [junit4]   2>   "eventTime":904247839758808,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[904247839758808],
   [junit4]   2>     "preferredOperation":"movereplica",
   [junit4]   2>     "_enqueue_time_":904257913213358,
   [junit4]   2>     "nodeNames":["127.0.0.1:10011_solr"]}}
   [junit4]   2> 122761 INFO  (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are 
closed, exiting.
   [junit4]   2> 122762 DEBUG 
(TEST-TestSimTriggerIntegration.testEventFromRestoredState-seed#[AE068F72BF210EE1])
 [    ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool 
executor now
   [junit4]   2> 122762 DEBUG 
(TEST-TestSimTriggerIntegration.testEventFromRestoredState-seed#[AE068F72BF210EE1])
 [    ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 122762 DEBUG 
(TEST-TestSimTriggerIntegration.testEventFromRestoredState-seed#[AE068F72BF210EE1])
 [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 122762 WARN  (AutoscalingActionExecutor-540-thread-1) [    ] 
o.a.s.c.a.SystemLogListener Exception sending event to collection .system
   [junit4]   2> java.io.IOException: java.util.concurrent.ExecutionException: 
java.io.IOException: already closed
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.sim.SimCloudManager.request(SimCloudManager.java:654)
 ~[test/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.SystemLogListener.onEvent(SystemLogListener.java:118)
 [java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerListeners.fireListeners(ScheduledTriggers.java:806)
 [java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerListeners.fireListeners(ScheduledTriggers.java:773)
 [java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$add$3(ScheduledTriggers.java:324)
 [java/:?]
   [junit4]   2>        at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:514) [?:?]
   [junit4]   2>        at 
java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
   [junit4]   2>        at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
 [java/:?]
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167) 
[?:?]
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641) 
[?:?]
   [junit4]   2>        at java.lang.Thread.run(Thread.java:844) [?:?]
   [junit4]   2> Caused by: java.util.concurrent.ExecutionException: 
java.io.IOException: already closed
   [junit4]   2>        at 
java.util.concurrent.FutureTask.report(FutureTask.java:122) ~[?:?]
   [junit4]   2>        at 
java.util.concurrent.FutureTask.get(FutureTask.java:191) ~[?:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.sim.SimCloudManager.request(SimCloudManager.java:652)
 ~[test/:?]
   [junit4]   2>        ... 10 more
   [junit4]   2> Caused by: java.io.IOException: already closed
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.sim.SimClusterStateProvider.ensureNotClosed(SimClusterStateProvider.java:2208)
 ~[test/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.sim.SimClusterStateProvider.simUpdate(SimClusterStateProvider.java:1425)
 ~[test/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.sim.SimCloudManager.simHandleSolrRequest(SimCloudManager.java:741)
 ~[test/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.sim.SimCloudManager.lambda$request$15(SimCloudManager.java:651)
 ~[test/:?]
   [junit4]   2>        ... 5 more
   [junit4]   2> 122762 DEBUG (AutoscalingActionExecutor-540-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers -- processing took 72 ms for event 
id=336689b4719d8T6rt02ywaos78pu0mw8fer5e03
   [junit4]   2> 122762 DEBUG 
(TEST-TestSimTriggerIntegration.testEventFromRestoredState-seed#[AE068F72BF210EE1])
 [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread 
pool executor
   [junit4]   2> 122762 DEBUG 
(TEST-TestSimTriggerIntegration.testEventFromRestoredState-seed#[AE068F72BF210EE1])
 [    ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 122762 DEBUG 
(TEST-TestSimTriggerIntegration.testEventFromRestoredState-seed#[AE068F72BF210EE1])
 [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed 
explicitly
   [junit4]   2> 122763 INFO  
(TEST-TestSimTriggerIntegration.testEventFromRestoredState-seed#[AE068F72BF210EE1])
 [    ] o.a.s.SolrTestCaseJ4 ###Ending testEventFromRestoredState
   [junit4]   2> 122764 INFO  
(TEST-TestSimTriggerIntegration.testTriggerThrottling-seed#[AE068F72BF210EE1]) 
[    ] o.a.s.SolrTestCaseJ4 ###Starting testTriggerThrottling
   [junit4]   2> 122766 DEBUG 
(TEST-TestSimTriggerIntegration.testTriggerThrottling-seed#[AE068F72BF210EE1]) 
[    ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 
127.0.0.1:10012_solr
   [junit4]   2> 122767 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and 
.scheduled_maintenance triggers
   [junit4]   2> 122767 DEBUG 
(TEST-TestSimTriggerIntegration.testTriggerThrottling-seed#[AE068F72BF210EE1]) 
[    ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, 
currentVersion=0
   [junit4]   2> 122767 DEBUG 
(TEST-TestSimTriggerIntegration.testTriggerThrottling-seed#[AE068F72BF210EE1]) 
[    ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 0
   [junit4]   2> 122767 INFO  
(TEST-TestSimTriggerIntegration.testTriggerThrottling-seed#[AE068F72BF210EE1]) 
[    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics 
history in memory.
   [junit4]   2> 122767 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode 
version 1
   [junit4]   2> 122767 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 0
   [junit4]   2> 122767 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
   [junit4]   2> 122767 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial 
livenodes: [127.0.0.1:10013_solr, 127.0.0.1:10012_solr]
   [junit4]   2> 122768 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers
   [junit4]   2> 122768 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
   [junit4]   2> 122768 DEBUG (ScheduledTrigger-543-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 122768 DEBUG (simCloudManagerPool-542-thread-2) [    ] 
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 122768 DEBUG (simCloudManagerPool-542-thread-2) [    ] 
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 
2
   [junit4]   2> 122768 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
   [junit4]   2> 122768 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers
   [junit4]   2> 122768 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
   [junit4]   2> 122769 DEBUG (simCloudManagerPool-542-thread-3) [    ] 
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 122769 DEBUG (simCloudManagerPool-542-thread-3) [    ] 
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 
3
   [junit4]   2> 122770 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
   [junit4]   2> 122770 INFO  (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.s.TestSimTriggerIntegration TestTriggerAction init
   [junit4]   2> 122770 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 - Initial 
livenodes: [127.0.0.1:10013_solr, 127.0.0.1:10012_solr]
   [junit4]   2> 122770 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 instantiated 
with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, 
class=org.apache.solr.cloud.autoscaling.sim.TestSimTriggerIntegration$ThrottlingTesterAction}],
 enabled=true}
   [junit4]   2> 122770 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers
   [junit4]   2> 122770 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
   [junit4]   2> 122770 DEBUG (ScheduledTrigger-543-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 122770 DEBUG (ScheduledTrigger-543-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
   [junit4]   2> 122770 DEBUG (simCloudManagerPool-542-thread-4) [    ] 
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 122771 DEBUG (simCloudManagerPool-542-thread-4) [    ] 
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 
4
   [junit4]   2> 122771 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
   [junit4]   2> 122771 INFO  (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.s.TestSimTriggerIntegration TestTriggerAction init
   [junit4]   2> 122771 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 - Initial 
livenodes: [127.0.0.1:10013_solr, 127.0.0.1:10012_solr]
   [junit4]   2> 122771 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 instantiated 
with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, 
class=org.apache.solr.cloud.autoscaling.sim.TestSimTriggerIntegration$ThrottlingTesterAction}],
 enabled=true}
   [junit4]   2> 122771 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers
   [junit4]   2> 122771 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
   [junit4]   2> 122771 DEBUG (ScheduledTrigger-543-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 122771 DEBUG (ScheduledTrigger-543-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
   [junit4]   2> 122771 DEBUG 
(TEST-TestSimTriggerIntegration.testTriggerThrottling-seed#[AE068F72BF210EE1]) 
[    ] o.a.s.c.a.s.SimClusterStateProvider -- creating marker: 
/autoscaling/nodeAdded/127.0.0.1:10014_solr
   [junit4]   2> 122788 DEBUG (ScheduledTrigger-543-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3 and last live nodes: 2
   [junit4]   2> 122791 DEBUG (ScheduledTrigger-543-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 122791 DEBUG (ScheduledTrigger-543-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 122791 DEBUG (ScheduledTrigger-543-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10014_solr at time 
904259443391208
   [junit4]   2> 122791 DEBUG (ScheduledTrigger-543-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 firing 
registered processor for nodes: [127.0.0.1:10014_solr] added at times 
[904259443391208], now=904259444438958
   [junit4]   2> 122791 DEBUG (ScheduledTrigger-543-thread-4) [    ] 
o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"3366b4ee87ae8T6rt02ywaos78pu0mw8fer5e07",
   [junit4]   2>   "source":"node_added_trigger1",
   [junit4]   2>   "eventTime":904259443391208,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[904259443391208],
   [junit4]   2>     "preferredOperation":"movereplica",
   [junit4]   2>     "nodeNames":["127.0.0.1:10014_solr"]}}
   [junit4]   2> 122791 DEBUG (ScheduledTrigger-543-thread-4) [    ] 
o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, 
node_added_trigger2, node_added_trigger1]
   [junit4]   2> 122792 DEBUG (simCloudManagerPool-542-thread-5) [    ] 
o.a.s.c.a.s.SimClusterStateProvider -- simCreateCollection .system, 
currentVersion=1
   [junit4]   2> 122792 DEBUG (simCloudManagerPool-542-thread-5) [    ] 
o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, 
currentVersion=1
   [junit4]   2> 122792 DEBUG (simCloudManagerPool-542-thread-5) [    ] 
o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 1
   [junit4]   2> 122803 DEBUG (simCloudManagerPool-542-thread-5) [    ] 
o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, 
currentVersion=2
   [junit4]   2> 122803 DEBUG (simCloudManagerPool-542-thread-5) [    ] 
o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 2
   [junit4]   2> 122814 DEBUG (simCloudManagerPool-542-thread-6) [    ] 
o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, 
currentVersion=3
   [junit4]   2> 122815 DEBUG (simCloudManagerPool-542-thread-6) [    ] 
o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 3
   [junit4]   2> 122825 DEBUG (simCloudManagerPool-542-thread-7) [    ] 
o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, 
currentVersion=4
   [junit4]   2> 122826 DEBUG (simCloudManagerPool-542-thread-7) [    ] 
o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 4
   [junit4]   2> 122836 DEBUG (simCloudManagerPool-542-thread-8) [    ] 
o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, 
currentVersion=5
   [junit4]   2> 122836 DEBUG (simCloudManagerPool-542-thread-8) [    ] 
o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 5
   [junit4]   2> 122837 DEBUG (simCloudManagerPool-542-thread-5) [    ] 
o.a.s.c.a.s.SimClusterStateProvider -- finished createCollection .system, 
currentVersion=6
   [junit4]   2> 122837 INFO  (simCloudManagerPool-542-thread-5) [    ] 
o.a.s.c.CloudTestUtils -- shards without leaders: [shard1]
   [junit4]   2> 122837 DEBUG (simCloudManagerPool-542-thread-9) [    ] 
o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for .system / shard1 
(currentVersion=6): {"core_node1":{
   [junit4]   2>     "core":".system_shard1_replica_n1",
   [junit4]   2>     "shard":"shard1",
   [junit4]   2>     "collection":".system",
   [junit4]   2>     "node_name":"127.0.0.1:10014_solr",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "state":"active",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>     "SEARCHER.searcher.numDocs":0}}
   [junit4]   2> 122839 DEBUG (simCloudManagerPool-542-thread-5) [    ] 
o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, 
currentVersion=6
   [junit4]   2> 122839 DEBUG (simCloudManagerPool-542-thread-5) [    ] 
o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 6
   [junit4]   2> 122839 DEBUG (ScheduledTrigger-543-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Removing new node from tracking: 127.0.0.1:10014_solr
   [junit4]   2> 122839 DEBUG (AutoscalingActionExecutor-544-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"3366b4ee87ae8T6rt02ywaos78pu0mw8fer5e07",
   [junit4]   2>   "source":"node_added_trigger1",
   [junit4]   2>   "eventTime":904259443391208,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[904259443391208],
   [junit4]   2>     "preferredOperation":"movereplica",
   [junit4]   2>     "_enqueue_time_":904259461073108,
   [junit4]   2>     "nodeNames":["127.0.0.1:10014_solr"]}}
   [junit4]   2> 122840 INFO  (AutoscalingActionExecutor-544-thread-1) [    ] 
o.a.s.c.a.s.TestSimTriggerIntegration action executed from node_added_trigger1
   [junit4]   2> 122840 DEBUG (AutoscalingActionExecutor-544-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
   [junit4]   2> 122840 DEBUG (AutoscalingActionExecutor-544-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger2 after 100ms
   [junit4]   2> 122840 DEBUG (AutoscalingActionExecutor-544-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger1 after 100ms
   [junit4]   2> 122840 DEBUG (AutoscalingActionExecutor-544-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers -- processing took 50 ms for event 
id=3366b4ee87ae8T6rt02ywaos78pu0mw8fer5e07
   [junit4]   2> 122941 DEBUG (ScheduledTrigger-543-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3 and last live nodes: 3
   [junit4]   2> 122941 DEBUG (ScheduledTrigger-543-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 122941 DEBUG (ScheduledTrigger-543-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 122941 DEBUG (ScheduledTrigger-543-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 122941 DEBUG (ScheduledTrigger-543-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 122941 DEBUG (ScheduledTrigger-543-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10014_solr at time 
904266939169508
   [junit4]   2> 122941 DEBUG (ScheduledTrigger-543-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 firing 
registered processor for nodes: [127.0.0.1:10014_solr] added at times 
[904266939169508], now=904266940301258
   [junit4]   2> 122941 DEBUG (ScheduledTrigger-543-thread-3) [    ] 
o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"3366d0db0fae4T6rt02ywaos78pu0mw8fer5e0a",
   [junit4]   2>   "source":"node_added_trigger2",
   [junit4]   2>   "eventTime":904266939169508,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[904266939169508],
   [junit4]   2>     "preferredOperation":"movereplica",
   [junit4]   2>     "nodeNames":["127.0.0.1:10014_solr"]}}
   [junit4]   2> 122941 DEBUG (ScheduledTrigger-543-thread-3) [    ] 
o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, 
node_added_trigger2, node_added_trigger1]
   [junit4]   2> 122942 DEBUG (ScheduledTrigger-543-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Removing new node from tracking: 127.0.0.1:10014_solr
   [junit4]   2> 122942 DEBUG (AutoscalingActionExecutor-544-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"3366d0db0fae4T6rt02ywaos78pu0mw8fer5e0a",
   [junit4]   2>   "source":"node_added_trigger2",
   [junit4]   2>   "eventTime":904266939169508,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[904266939169508],
   [junit4]   2>     "preferredOperation":"movereplica",
   [junit4]   2>     "_enqueue_time_":904266954150308,
   [junit4]   2>     "nodeNames":["127.0.0.1:10014_solr"]}}
   [junit4]   2> 122942 INFO  (AutoscalingActionExecutor-544-thread-1) [    ] 
o.a.s.c.a.s.TestSimTriggerIntegration last action at 904261882979458 time = 
904266987697908
   [junit4]   2> 122942 INFO  (AutoscalingActionExecutor-544-thread-1) [    ] 
o.a.s.c.a.s.TestSimTriggerIntegration action executed from node_added_trigger2
   [junit4]   2> 122942 DEBUG (AutoscalingActionExecutor-544-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
   [junit4]   2> 122942 DEBUG (AutoscalingActionExecutor-544-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger2 after 100ms
   [junit4]   2> 122942 DEBUG (AutoscalingActionExecutor-544-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers Resuming trigger:

[...truncated too long message...]

odes: 3 and last live nodes: 3
   [junit4]   2> 7629 DEBUG (ScheduledTrigger-53-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 7649 DEBUG (ScheduledTrigger-53-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3 and last live nodes: 3
   [junit4]   2> 7649 DEBUG (ScheduledTrigger-53-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 7649 DEBUG (ScheduledTrigger-53-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with 
currently live nodes: 3 and last live nodes: 3
   [junit4]   2> 7649 DEBUG (ScheduledTrigger-53-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 7669 DEBUG (ScheduledTrigger-53-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 7669 DEBUG (ScheduledTrigger-53-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with 
currently live nodes: 3 and last live nodes: 3
   [junit4]   2> 7669 DEBUG (ScheduledTrigger-53-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3 and last live nodes: 3
   [junit4]   2> 7669 DEBUG (ScheduledTrigger-53-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 7690 DEBUG (ScheduledTrigger-53-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 7690 DEBUG (ScheduledTrigger-53-thread-4) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3 and last live nodes: 3
   [junit4]   2> 7690 DEBUG (ScheduledTrigger-53-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 7690 DEBUG (ScheduledTrigger-53-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with 
currently live nodes: 3 and last live nodes: 3
   [junit4]   2> 7710 DEBUG (ScheduledTrigger-53-thread-4) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3 and last live nodes: 3
   [junit4]   2> 7710 DEBUG (ScheduledTrigger-53-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with 
currently live nodes: 3 and last live nodes: 3
   [junit4]   2> 7710 DEBUG (ScheduledTrigger-53-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 7710 DEBUG (ScheduledTrigger-53-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 7730 DEBUG (ScheduledTrigger-53-thread-4) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3 and last live nodes: 3
   [junit4]   2> 7730 DEBUG (ScheduledTrigger-53-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 7730 DEBUG (ScheduledTrigger-53-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with 
currently live nodes: 3 and last live nodes: 3
   [junit4]   2> 7730 DEBUG (ScheduledTrigger-53-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 7751 DEBUG (ScheduledTrigger-53-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 7751 DEBUG (ScheduledTrigger-53-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with 
currently live nodes: 3 and last live nodes: 3
   [junit4]   2> 7751 DEBUG (ScheduledTrigger-53-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3 and last live nodes: 3
   [junit4]   2> 7751 DEBUG (ScheduledTrigger-53-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 7763 DEBUG 
(TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[AE068F72BF210EE1])
 [    ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool 
executor now
   [junit4]   2> 7763 DEBUG 
(TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[AE068F72BF210EE1])
 [    ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 7763 DEBUG 
(TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[AE068F72BF210EE1])
 [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 7763 DEBUG 
(TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[AE068F72BF210EE1])
 [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread 
pool executor
   [junit4]   2> 7763 DEBUG 
(TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[AE068F72BF210EE1])
 [    ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 7763 DEBUG 
(TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[AE068F72BF210EE1])
 [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed 
explicitly
   [junit4]   2> 7763 WARN  (Simulated OverseerAutoScalingTriggerThread) [    ] 
o.a.s.c.a.OverseerTriggerThread Interrupted
   [junit4]   2> java.lang.InterruptedException: null
   [junit4]   2>        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2037)
 ~[?:?]
   [junit4]   2>        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2071)
 ~[?:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:189)
 [java/:?]
   [junit4]   2>        at java.lang.Thread.run(Thread.java:844) [?:?]
   [junit4]   2> 7763 INFO  
(TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[AE068F72BF210EE1])
 [    ] o.a.s.SolrTestCaseJ4 ###Ending testNodeMarkersRegistration
   [junit4]   2> NOTE: reproduce with: ant test  
-Dtestcase=TestSimTriggerIntegration -Dtests.method=testNodeMarkersRegistration 
-Dtests.seed=AE068F72BF210EE1 -Dtests.multiplier=3 -Dtests.slow=true 
-Dtests.badapples=true -Dtests.locale=el-GR -Dtests.timezone=Australia/Hobart 
-Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] FAILURE 1.36s J1 | 
TestSimTriggerIntegration.testNodeMarkersRegistration <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: Path 
/autoscaling/nodeAdded/127.0.0.1:10027_solr should have been deleted
   [junit4]    >        at 
__randomizedtesting.SeedInfo.seed([AE068F72BF210EE1:B6BC077EB114C30E]:0)
   [junit4]    >        at 
org.apache.solr.cloud.autoscaling.sim.TestSimTriggerIntegration.testNodeMarkersRegistration(TestSimTriggerIntegration.java:892)
   [junit4]    >        at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]    >        at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]    >        at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]    >        at 
java.base/java.lang.reflect.Method.invoke(Method.java:564)
   [junit4]    >        at java.base/java.lang.Thread.run(Thread.java:844)
   [junit4]   2> NOTE: leaving temporary files on disk at: 
/home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.sim.TestSimTriggerIntegration_AE068F72BF210EE1-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene80): {}, 
docValues:{}, maxPointsInLeafNode=1655, maxMBSortInHeap=5.221475914951917, 
sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@e9cc71),
 locale=el-GR, timezone=Australia/Hobart
   [junit4]   2> NOTE: Linux 4.15.0-42-generic amd64/Oracle Corporation 9.0.4 
(64-bit)/cpus=8,threads=1,free=406213200,total=536870912
   [junit4]   2> NOTE: All tests run in this JVM: [TestSimTriggerIntegration]
   [junit4] Completed [1/5 (1!)] on J1 in 6.83s, 11 tests, 1 failure, 3 skipped 
<<< FAILURES!

[...truncated 79 lines...]
BUILD FAILED
/home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/lucene/common-build.xml:1572:
 The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/lucene/common-build.xml:1099:
 There were test failures: 5 suites, 55 tests, 1 failure, 15 ignored (15 
assumptions) [seed: AE068F72BF210EE1]

Total time: 14 seconds

[repro] Setting last failure code to 256

[repro] Failures:
[repro]   1/5 failed: 
org.apache.solr.cloud.autoscaling.sim.TestSimTriggerIntegration
[repro] Exiting with code 256
+ mv lucene/build lucene/build.repro
+ mv solr/build solr/build.repro
+ mv lucene/build.orig lucene/build
+ mv solr/build.orig solr/build
Archiving artifacts
Setting 
ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
[WARNINGS] Parsing warnings in console log with parser Java Compiler (javac)
Setting 
ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting 
ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
<Git Blamer> Using GitBlamer to create author and commit information for all 
warnings.
<Git Blamer> GIT_COMMIT=24ae0d16cd5446f27309270f6534d4c19fb4fe43, 
workspace=/var/lib/jenkins/workspace/Lucene-Solr-BadApples-master-Linux
[WARNINGS] Computing warning deltas based on reference build #140
Recording test results
Setting 
ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
Setting 
ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting 
ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting 
ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting 
ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Reply via email to