Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.3/64/

1 tests failed.
FAILED:  
org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testEventQueue

Error Message:
action wasn't interrupted

Stack Trace:
java.lang.AssertionError: action wasn't interrupted
        at 
__randomizedtesting.SeedInfo.seed([A5F19B037E49F612:6C44D9AD772E30E7]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at 
org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testEventQueue(TestTriggerIntegration.java:641)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 14209 lines...]
   [junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration
   [junit4]   2> Creating dataDir: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.3/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.sim.TestTriggerIntegration_A5F19B037E49F612-001/init-core-data-001
   [junit4]   2> 3055931 WARN  
(SUITE-TestTriggerIntegration-seed#[A5F19B037E49F612]-worker) [    ] 
o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=29 numCloses=29
   [junit4]   2> 3055931 INFO  
(SUITE-TestTriggerIntegration-seed#[A5F19B037E49F612]-worker) [    ] 
o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) 
w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 3055933 INFO  
(SUITE-TestTriggerIntegration-seed#[A5F19B037E49F612]-worker) [    ] 
o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: 
@org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 3055933 INFO  
(SUITE-TestTriggerIntegration-seed#[A5F19B037E49F612]-worker) [    ] 
o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: 
test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 3055942 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
   [junit4]   2> 3055942 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode 
version 0
   [junit4]   2> 3055942 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10006_solr, 
127.0.0.1:10005_solr]
   [junit4]   2> 3055943 INFO  
(TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[A5F19B037E49F612]) [    
] o.a.s.SolrTestCaseJ4 ###Starting testNodeLostTrigger
   [junit4]   2> 3055945 DEBUG 
(TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[A5F19B037E49F612]) [    
] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode 
version 1
   [junit4]   2> 3055945 INFO  
(TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[A5F19B037E49F612]) [    
] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing 
object cache...
   [junit4]   2> 3055947 DEBUG 
(TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[A5F19B037E49F612]) [    
] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed 
explicitly
   [junit4]   2> 3055947 WARN  (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Interrupted
   [junit4]   2> java.lang.InterruptedException
   [junit4]   2>        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1220)
   [junit4]   2>        at 
java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:335)
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:163)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 3055947 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
   [junit4]   2> 3055947 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode 
version 2
   [junit4]   2> 3055948 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10006_solr, 
127.0.0.1:10005_solr]
   [junit4]   2> 3055948 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion -1
   [junit4]   2> 3055948 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
   [junit4]   2> 3055948 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 3055948 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
   [junit4]   2> 3055948 DEBUG (ScheduledTrigger-9642-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2
   [junit4]   2> 3055969 DEBUG (ScheduledTrigger-9642-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2
   [junit4]   2> 3055989 DEBUG (ScheduledTrigger-9642-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2
   [junit4]   2> 3056010 DEBUG (ScheduledTrigger-9642-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2
   [junit4]   2> 3056030 DEBUG (ScheduledTrigger-9642-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2
   [junit4]   2> 3056049 DEBUG (simCloudManagerPool-9641-thread-1) [    ] 
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 3056049 DEBUG (simCloudManagerPool-9641-thread-1) [    ] 
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 
3
   [junit4]   2> 3056052 DEBUG (ScheduledTrigger-9642-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2
   [junit4]   2> 3056052 DEBUG (simCloudManagerPool-9641-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10006_solr, 
127.0.0.1:10005_solr]
   [junit4]   2> 3056052 DEBUG (simCloudManagerPool-9641-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10006_solr, 
127.0.0.1:10005_solr]
   [junit4]   2> 3056052 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
   [junit4]   2> 3056053 INFO  (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
   [junit4]   2> 3056053 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 3056053 DEBUG (ScheduledTrigger-9642-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with 
currently live nodes: 2
   [junit4]   2> 3056053 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
   [junit4]   2> 3056072 DEBUG (ScheduledTrigger-9642-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 1
   [junit4]   2> 3056072 DEBUG (ScheduledTrigger-9642-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10005_solr
   [junit4]   2> 3056073 DEBUG (ScheduledTrigger-9642-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with 
currently live nodes: 1
   [junit4]   2> 3056073 DEBUG (ScheduledTrigger-9642-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10005_solr
   [junit4]   2> 3056092 DEBUG (ScheduledTrigger-9642-thread-4) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 1
   [junit4]   2> 3056094 DEBUG (ScheduledTrigger-9642-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with 
currently live nodes: 1
   [junit4]   2> 3056112 DEBUG (ScheduledTrigger-9642-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 1
   [junit4]   2> 3056114 DEBUG (ScheduledTrigger-9642-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with 
currently live nodes: 1
   [junit4]   2> 3056114 DEBUG (ScheduledTrigger-9642-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost 
nodes: [127.0.0.1:10005_solr]
   [junit4]   2> 3056114 DEBUG (ScheduledTrigger-9642-thread-2) [    ] 
o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"5c7c9863f863d7Td40et349ergsrftgiu0xy1ipe",
   [junit4]   2>   "source":"node_lost_trigger",
   [junit4]   2>   "eventTime":26032691811476439,
   [junit4]   2>   "eventType":"NODELOST",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[26032691811476439],
   [junit4]   2>     "nodeNames":["127.0.0.1:10005_solr"]}}
   [junit4]   2> 3056114 DEBUG (ScheduledTrigger-9642-thread-2) [    ] 
o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, 
node_lost_trigger]
   [junit4]   2> 3056115 DEBUG (AutoscalingActionExecutor-9643-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"5c7c9863f863d7Td40et349ergsrftgiu0xy1ipe",
   [junit4]   2>   "source":"node_lost_trigger",
   [junit4]   2>   "eventTime":26032691811476439,
   [junit4]   2>   "eventType":"NODELOST",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[26032691811476439],
   [junit4]   2>     "_enqueue_time_":26032693874082989,
   [junit4]   2>     "nodeNames":["127.0.0.1:10005_solr"]}}
   [junit4]   2> 3056117 DEBUG (simCloudManagerPool-9641-thread-3) [    ] 
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 3056117 DEBUG (simCloudManagerPool-9641-thread-3) [    ] 
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 
4
   [junit4]   2> 3056117 DEBUG (AutoscalingActionExecutor-9643-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
   [junit4]   2> 3056117 DEBUG (simCloudManagerPool-9641-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10006_solr]
   [junit4]   2> 3056117 DEBUG (AutoscalingActionExecutor-9643-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger after 100ms
   [junit4]   2> 3056117 DEBUG (simCloudManagerPool-9641-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10006_solr]
   [junit4]   2> 3056117 DEBUG (AutoscalingActionExecutor-9643-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers -- processing took 86 ms for event 
id=5c7c9863f863d7Td40et349ergsrftgiu0xy1ipe
   [junit4]   2> 3056117 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
   [junit4]   2> 3056118 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 3056118 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
   [junit4]   2> 3056158 INFO  
(TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[A5F19B037E49F612]) [    
] o.a.s.SolrTestCaseJ4 ###Ending testNodeLostTrigger
   [junit4]   2> 3056158 INFO  
(TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[A5F19B037E49F612]) [    
] o.a.s.c.a.s.SimSolrCloudTestCase 
   [junit4]   2> 
   [junit4]   2> 3056158 INFO  
(TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[A5F19B037E49F612]) [    
] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
   [junit4]   2> 3056158 INFO  
(TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[A5F19B037E49F612]) [    
] o.a.s.c.a.s.SimSolrCloudTestCase ############ FINAL CLUSTER STATS ############
   [junit4]   2> 3056158 INFO  
(TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[A5F19B037E49F612]) [    
] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
   [junit4]   2> 
   [junit4]   2> 3056158 INFO  
(TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[A5F19B037E49F612]) [    
] o.a.s.c.a.s.SimSolrCloudTestCase ## Live nodes:         1
   [junit4]   2> 3056158 INFO  
(TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[A5F19B037E49F612]) [    
] o.a.s.c.a.s.SimSolrCloudTestCase ## Empty nodes:        1
   [junit4]   2> 3056158 INFO  
(TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[A5F19B037E49F612]) [    
] o.a.s.c.a.s.SimSolrCloudTestCase ## Dead nodes:         1
   [junit4]   2> 3056158 INFO  
(TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[A5F19B037E49F612]) [    
] o.a.s.c.a.s.SimSolrCloudTestCase ##             127.0.0.1:10005_solr
   [junit4]   2> 3056158 INFO  
(TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[A5F19B037E49F612]) [    
] o.a.s.c.a.s.SimSolrCloudTestCase ## Collections:        []
   [junit4]   2> 3056159 INFO  
(TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[A5F19B037E49F612]) [    
] o.a.s.c.a.s.SimSolrCloudTestCase ## Max replicas per node:      0
   [junit4]   2> 3056159 INFO  
(TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[A5F19B037E49F612]) [    
] o.a.s.c.a.s.SimSolrCloudTestCase ## Min replicas per node:      0
   [junit4]   2> 3056159 INFO  
(TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[A5F19B037E49F612]) [    
] o.a.s.c.a.s.SimSolrCloudTestCase ## Total replicas:             0
   [junit4]   2> 3056159 INFO  
(TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[A5F19B037E49F612]) [    
] o.a.s.c.a.s.SimSolrCloudTestCase ######### Final Solr op counts ##########
   [junit4]   2> 3056159 INFO  
(TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[A5F19B037E49F612]) [    
] o.a.s.c.a.s.SimSolrCloudTestCase ##             - autoscaling        2
   [junit4]   2> 3056159 INFO  
(TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[A5F19B037E49F612]) [    
] o.a.s.c.a.s.SimSolrCloudTestCase ##             - update             2
   [junit4]   2> 3056159 INFO  
(TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[A5F19B037E49F612]) [    
] o.a.s.c.a.s.SimSolrCloudTestCase ######### Autoscaling event counts 
###########
   [junit4]   2> 3056160 INFO  
(TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[A5F19B037E49F612]) [    
] o.a.s.c.a.s.SimSolrCloudTestCase ## * Trigger: node_lost_trigger
   [junit4]   2> 3056160 INFO  
(TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[A5F19B037E49F612]) [    
] o.a.s.c.a.s.SimSolrCloudTestCase ##             - STARTED         1
   [junit4]   2> 3056161 INFO  
(TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[A5F19B037E49F612]) [    
] o.a.s.c.a.s.SimSolrCloudTestCase ##             - SUCCEEDED       1
   [junit4]   2> 3056164 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[A5F19B037E49F612])
 [    ] o.a.s.SolrTestCaseJ4 ###Starting testNodeAddedTriggerRestoreState
   [junit4]   2> 3056164 DEBUG 
(TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[A5F19B037E49F612])
 [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode 
version 5
   [junit4]   2> 3056164 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
   [junit4]   2> 3056165 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 3056165 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 3056165 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
   [junit4]   2> 3056165 DEBUG 
(TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[A5F19B037E49F612])
 [    ] o.a.s.c.a.s.SimNodeStateProvider - removing dead node values: 
127.0.0.1:10005_solr
   [junit4]   2> 3056165 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[A5F19B037E49F612])
 [    ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and 
clearing object cache...
   [junit4]   2> 3056165 WARN  (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Interrupted
   [junit4]   2> java.lang.InterruptedException
   [junit4]   2>        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014)
   [junit4]   2>        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048)
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:175)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 3056166 DEBUG 
(TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[A5F19B037E49F612])
 [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed 
explicitly
   [junit4]   2> 3056167 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
   [junit4]   2> 3056167 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode 
version 6
   [junit4]   2> 3056167 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10006_solr, 
127.0.0.1:10007_solr]
   [junit4]   2> 3056167 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion -1
   [junit4]   2> 3056167 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
   [junit4]   2> 3056168 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 3056168 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion 6
   [junit4]   2> 3056168 DEBUG (ScheduledTrigger-9645-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2
   [junit4]   2> 3056188 DEBUG (ScheduledTrigger-9645-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2
   [junit4]   2> 3056209 DEBUG (ScheduledTrigger-9645-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2
   [junit4]   2> 3056229 DEBUG (ScheduledTrigger-9645-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2
   [junit4]   2> 3056249 DEBUG (ScheduledTrigger-9645-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2
   [junit4]   2> 3056267 DEBUG (simCloudManagerPool-9644-thread-1) [    ] 
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 3056268 DEBUG (simCloudManagerPool-9644-thread-1) [    ] 
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 
7
   [junit4]   2> 3056268 DEBUG (simCloudManagerPool-9644-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:10006_solr, 
127.0.0.1:10007_solr]
   [junit4]   2> 3056268 DEBUG (simCloudManagerPool-9644-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_restore_trigger 
instantiated with properties: {event=nodeAdded, waitFor=5, actions=[{name=test, 
class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestTriggerAction}],
 enabled=true}
   [junit4]   2> 3056268 DEBUG (simCloudManagerPool-9644-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10006_solr, 
127.0.0.1:10007_solr]
   [junit4]   2> 3056268 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 7
   [junit4]   2> 3056268 INFO  (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
   [junit4]   2> 3056269 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion 7
   [junit4]   2> 3056269 DEBUG (ScheduledTrigger-9645-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_restore_trigger
   [junit4]   2> 3056269 DEBUG (ScheduledTrigger-9645-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3056269 DEBUG (ScheduledTrigger-9645-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10008_solr at time 
26032701615364839
   [junit4]   2> 3056269 DEBUG (ScheduledTrigger-9645-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3056290 DEBUG (ScheduledTrigger-9645-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_restore_trigger
   [junit4]   2> 3056290 DEBUG (ScheduledTrigger-9645-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3056290 DEBUG (ScheduledTrigger-9645-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3056300 DEBUG (simCloudManagerPool-9644-thread-2) [    ] 
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 3056300 DEBUG (simCloudManagerPool-9644-thread-2) [    ] 
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 
8
   [junit4]   2> 3056300 DEBUG (simCloudManagerPool-9644-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:10006_solr, 
127.0.0.1:10008_solr, 127.0.0.1:10007_solr]
   [junit4]   2> 3056300 DEBUG (simCloudManagerPool-9644-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_restore_trigger 
instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, 
class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestTriggerAction}],
 enabled=true}
   [junit4]   2> 3056300 DEBUG (simCloudManagerPool-9644-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10006_solr, 
127.0.0.1:10008_solr, 127.0.0.1:10007_solr]
   [junit4]   2> 3056300 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 8
   [junit4]   2> 3056301 INFO  (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
   [junit4]   2> 3056301 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.NodeAddedTrigger Adding node from marker path: 127.0.0.1:10008_solr
   [junit4]   2> 3056301 DEBUG (ScheduledTrigger-9645-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_restore_trigger
   [junit4]   2> 3056301 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 8, lastZnodeVersion 8
   [junit4]   2> 3056301 DEBUG (ScheduledTrigger-9645-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3056302 DEBUG (ScheduledTrigger-9645-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_restore_trigger firing 
registered processor for nodes: [127.0.0.1:10008_solr] added at times 
[26032703213671289], now=26032703228965939
   [junit4]   2> 3056302 DEBUG (ScheduledTrigger-9645-thread-3) [    ] 
o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"5c7c9b0b981379Td40et349ergsrftgiu0xy1iph",
   [junit4]   2>   "source":"node_added_restore_trigger",
   [junit4]   2>   "eventTime":26032703213671289,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[26032703213671289],
   [junit4]   2>     "nodeNames":["127.0.0.1:10008_solr"]}}
   [junit4]   2> 3056302 DEBUG (ScheduledTrigger-9645-thread-3) [    ] 
o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, 
node_added_restore_trigger]
   [junit4]   2> 3056303 DEBUG (AutoscalingActionExecutor-9646-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"5c7c9b0b981379Td40et349ergsrftgiu0xy1iph",
   [junit4]   2>   "source":"node_added_restore_trigger",
   [junit4]   2>   "eventTime":26032703213671289,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[26032703213671289],
   [junit4]   2>     "_enqueue_time_":26032703257020689,
   [junit4]   2>     "nodeNames":["127.0.0.1:10008_solr"]}}
   [junit4]   2> 3056303 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[A5F19B037E49F612])
 [    ] o.a.s.SolrTestCaseJ4 ###Ending testNodeAddedTriggerRestoreState
   [junit4]   2> 3056303 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[A5F19B037E49F612])
 [    ] o.a.s.c.a.s.SimSolrCloudTestCase 
   [junit4]   2> 
   [junit4]   2> 3056303 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[A5F19B037E49F612])
 [    ] o.a.s.c.a.s.SimSolrCloudTestCase 
#############################################
   [junit4]   2> 3056303 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[A5F19B037E49F612])
 [    ] o.a.s.c.a.s.SimSolrCloudTestCase ############ FINAL CLUSTER STATS 
############
   [junit4]   2> 3056303 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[A5F19B037E49F612])
 [    ] o.a.s.c.a.s.SimSolrCloudTestCase 
#############################################
   [junit4]   2> 
   [junit4]   2> 3056303 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[A5F19B037E49F612])
 [    ] o.a.s.c.a.s.SimSolrCloudTestCase ## Live nodes:            3
   [junit4]   2> 3056303 DEBUG (AutoscalingActionExecutor-9646-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
   [junit4]   2> 3056303 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[A5F19B037E49F612])
 [    ] o.a.s.c.a.s.SimSolrCloudTestCase ## Empty nodes:   3
   [junit4]   2> 3056303 DEBUG (AutoscalingActionExecutor-9646-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_restore_trigger after 
100ms
   [junit4]   2> 3056303 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[A5F19B037E49F612])
 [    ] o.a.s.c.a.s.SimSolrCloudTestCase ## Dead nodes:            0
   [junit4]   2> 3056303 DEBUG (AutoscalingActionExecutor-9646-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers -- processing took 36 ms for event 
id=5c7c9b0b981379Td40et349ergsrftgiu0xy1iph
   [junit4]   2> 3056304 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[A5F19B037E49F612])
 [    ] o.a.s.c.a.s.SimSolrCloudTestCase ## Collections:   []
   [junit4]   2> 3056304 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[A5F19B037E49F612])
 [    ] o.a.s.c.a.s.SimSolrCloudTestCase ## Max replicas per node: 0
   [junit4]   2> 3056304 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[A5F19B037E49F612])
 [    ] o.a.s.c.a.s.SimSolrCloudTestCase ## Min replicas per node: 0
   [junit4]   2> 3056304 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[A5F19B037E49F612])
 [    ] o.a.s.c.a.s.SimSolrCloudTestCase ## Total replicas:                0
   [junit4]   2> 3056304 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[A5F19B037E49F612])
 [    ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Final Solr op counts 
##########
   [junit4]   2> 3056304 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[A5F19B037E49F612])
 [    ] o.a.s.c.a.s.SimSolrCloudTestCase ##                - autoscaling        
2
   [junit4]   2> 3056304 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[A5F19B037E49F612])
 [    ] o.a.s.c.a.s.SimSolrCloudTestCase ##                - update             
2
   [junit4]   2> 3056304 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[A5F19B037E49F612])
 [    ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Autoscaling event counts 
###########
   [junit4]   2> 3056304 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[A5F19B037E49F612])
 [    ] o.a.s.c.a.s.SimSolrCloudTestCase ## * Trigger: 
node_added_restore_trigger
   [junit4]   2> 3056304 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[A5F19B037E49F612])
 [    ] o.a.s.c.a.s.SimSolrCloudTestCase ##                - STARTED         1
   [junit4]   2> 3056304 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[A5F19B037E49F612])
 [    ] o.a.s.c.a.s.SimSolrCloudTestCase ##                - SUCCEEDED       1
   [junit4] IGNOR/A 0.00s J1 | TestTriggerIntegration.testListeners
   [junit4]    > Assumption #1: 'badapple' test group is disabled 
(@BadApple(bugUrl=https://issues.apache.org/jira/browse/SOLR-12028))
   [junit4]   2> 3056308 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[A5F19B037E49F612]) [    
] o.a.s.SolrTestCaseJ4 ###Starting testNodeAddedTrigger
   [junit4]   2> 3056308 DEBUG 
(TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[A5F19B037E49F612]) [    
] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode 
version 9
   [junit4]   2> 3056308 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 9
   [junit4]   2> 3056309 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 3056309 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 3056309 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 9, lastZnodeVersion 9
   [junit4]   2> 3056310 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[A5F19B037E49F612]) [    
] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing 
object cache...
   [junit4]   2> 3056310 WARN  (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Interrupted
   [junit4]   2> java.lang.InterruptedException
   [junit4]   2>        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014)
   [junit4]   2>        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048)
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:175)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 3056311 DEBUG 
(TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[A5F19B037E49F612]) [    
] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed 
explicitly
   [junit4]   2> 3056311 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
   [junit4]   2> 3056312 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode 
version 10
   [junit4]   2> 3056317 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10006_solr, 
127.0.0.1:10007_solr]
   [junit4]   2> 3056317 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 10, lastZnodeVersion -1
   [junit4]   2> 3056317 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 
10
   [junit4]   2> 3056317 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 3056317 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 10, lastZnodeVersion 10
   [junit4]   2> 3056318 DEBUG (ScheduledTrigger-9648-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2
   [junit4]   2> 3056338 DEBUG (ScheduledTrigger-9648-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2
   [junit4]   2> 3056358 DEBUG (ScheduledTrigger-9648-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2
   [junit4]   2> 3056379 DEBUG (ScheduledTrigger-9648-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2
   [junit4]   2> 3056399 DEBUG (ScheduledTrigger-9648-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2
   [junit4]   2> 3056412 DEBUG (simCloudManagerPool-9647-thread-1) [    ] 
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 3056412 DEBUG (simCloudManagerPool-9647-thread-1) [    ] 
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 
11
   [junit4]   2> 3056413 DEBUG (simCloudManagerPool-9647-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10006_solr, 
127.0.0.1:10007_solr]
   [junit4]   2> 3056413 DEBUG (simCloudManagerPool-9647-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:10006_solr, 
127.0.0.1:10007_solr]
   [junit4]   2> 3056413 DEBUG (simCloudManagerPool-9647-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated 
with properties: {event=nodeAdded, waitFor=3, actions=[{name=test, 
class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestTriggerAction}],
 enabled=true}
   [junit4]   2> 3056413 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 
11
   [junit4]   2> 3056413 INFO  (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
   [junit4]   2> 3056414 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 11, lastZnodeVersion 11
   [junit4]   2> 3056414 DEBUG (ScheduledTrigger-9648-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 3056414 DEBUG (ScheduledTrigger-9648-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
   [junit4]   2> 3056419 DEBUG (ScheduledTrigger-9648-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3056434 DEBUG (ScheduledTrigger-9648-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 3056434 DEBUG (ScheduledTrigger-9648-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3056434 DEBUG (ScheduledTrigger-9648-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10009_solr at time 
26032709867730639
   [junit4]   2> 3056439 DEBUG (ScheduledTrigger-9648-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3056455 DEBUG (ScheduledTrigger-9648-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 3056455 DEBUG (ScheduledTrigger-9648-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3056460 DEBUG (ScheduledTrigger-9648-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3056475 DEBUG (ScheduledTrigger-9648-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 3056475 DEBUG (ScheduledTrigger-9648-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3056480 DEBUG (ScheduledTrigger-9648-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3056495 DEBUG (ScheduledTrigger-9648-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 3056495 DEBUG (ScheduledTrigger-9648-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3056495 DEBUG (ScheduledTrigger-9648-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger firing 
registered processor for nodes: [127.0.0.1:10009_solr] added at times 
[26032709867730639], now=26032712914088839
   [junit4]   2> 3056495 DEBUG (ScheduledTrigger-9648-thread-4) [    ] 
o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"5c7c9c9834f6cfTd40et349ergsrftgiu0xy1ipk",
   [junit4]   2>   "source":"node_added_trigger",
   [junit4]   2>   "eventTime":26032709867730639,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[26032709867730639],
   [junit4]   2>     "nodeNames":["127.0.0.1:10009_solr"]}}
   [junit4]   2> 3056496 DEBUG (ScheduledTrigger-9648-thread-4) [    ] 
o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, 
node_added_trigger]
   [junit4]   2> 3056496 DEBUG (AutoscalingActionExecutor-9649-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"5c7c9c9834f6cfTd40et349ergsrftgiu0xy1ipk",
   [junit4]   2>   "source":"node_added_trigger",
   [junit4]   2>   "eventTime":26032709867730639,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[26032709867730639],
   [junit4]   2>     "_enqueue_time_":26032712939902289,
   [junit4]   2>     "nodeNames":["127.0.0.1:10009_solr"]}}
   [junit4]   2> 3056501 DEBUG (simCloudManagerPool-9647-thread-3) [    ] 
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 3056501 DEBUG (simCloudManagerPool-9647-thread-3) [    ] 
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 
12
   [junit4]   2> 3056502 DEBUG (simCloudManagerPool-9647-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10006_solr, 
127.0.0.1:10009_solr, 127.0.0.1:10007_solr]
   [junit4]   2> 3056502 DEBUG (AutoscalingActionExecutor-9649-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
   [junit4]   2> 3056502 DEBUG (AutoscalingActionExecutor-9649-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger after 100ms
   [junit4]   2> 3056502 DEBUG (AutoscalingActionExecutor-9649-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers -- processing took 298 ms for event 
id=5c7c9c9834f6cfTd40et349ergsrftgiu0xy1ipk
   [junit4]   2> 3056502 DEBUG (simCloudManagerPool-9647-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:10006_solr, 
127.0.0.1:10009_solr, 127.0.0.1:10007_solr]
   [junit4]   2> 3056503 DEBUG (simCloudManagerPool-9647-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated 
with properties: {event=nodeAdded, waitFor=3, actions=[{name=test, 
class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestTriggerAction}],
 enabled=true}
   [junit4]   2> 3056503 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 
12
   [junit4]   2> 3056503 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 12, lastZnodeVersion 12
   [junit4]   2> 3056543 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[A5F19B037E49F612]) [    
] o.a.s.SolrTestCaseJ4 ###Ending testNodeAddedTrigger
   [junit4]   2> 3056543 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[A5F19B037E49F612]) [    
] o.a.s.c.a.s.SimSolrCloudTestCase 
   [junit4]   2> 
   [junit4]   2> 3056543 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[A5F19B037E49F612]) [    
] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
   [junit4]   2> 3056543 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[A5F19B037E49F612]) [    
] o.a.s.c.a.s.SimSolrCloudTestCase ############ FINAL CLUSTER STATS ############
   [junit4]   2> 3056543 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[A5F19B037E49F612]) [    
] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
   [junit4]   2> 
   [junit4]   2> 3056543 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[A5F19B037E49F612]) [    
] o.a.s.c.a.s.SimSolrCloudTestCase ## Live nodes:                3
   [junit4]   2> 3056543 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[A5F19B037E49F612]) [    
] o.a.s.c.a.s.SimSolrCloudTestCase ## Empty nodes:       3
   [junit4]   2> 3056543 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[A5F19B037E49F612]) [    
] o.a.s.c.a.s.SimSolrCloudTestCase ## Dead nodes:                0
   [junit4]   2> 3056544 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[A5F19B037E49F612]) [    
] o.a.s.c.a.s.SimSolrCloudTestCase ## Collections:       []
   [junit4]   2> 3056544 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[A5F19B037E49F612]) [    
] o.a.s.c.a.s.SimSolrCloudTestCase ## Max replicas per node:     0
   [junit4]   2> 3056544 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[A5F19B037E49F612]) [    
] o.a.s.c.a.s.SimSolrCloudTestCase ## Min replicas per node:     0
   [junit4]   2> 3056544 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[A5F19B037E49F612]) [    
] o.a.s.c.a.s.SimSolrCloudTestCase ## Total replicas:            0
   [junit4]   2> 3056544 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[A5F19B037E49F612]) [    
] o.a.s.c.a.s.SimSolrCloudTestCase ######### Final Solr op counts ##########
   [junit4]   2> 3056544 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[A5F19B037E49F612]) [    
] o.a.s.c.a.s.SimSolrCloudTestCase ##            - autoscaling        2
   [junit4]   2> 3056544 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[A5F19B037E49F612]) [    
] o.a.s.c.a.s.SimSolrCloudTestCase ##            - update             2
   [junit4]   2> 3056544 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[A5F19B037E49F612]) [    
] o.a.s.c.a.s.SimSolrCloudTestCase ######### Autoscaling event counts 
###########
   [junit4]   2> 3056544 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[A5F19B037E49F612]) [    
] o.a.s.c.a.s.SimSolrCloudTestCase ## * Trigger: node_added_trigger
   [junit4]   2> 3056544 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[A5F19B037E49F612]) [    
] o.a.s.c.a.s.SimSolrCloudTestCase ##            - STARTED         1
   [junit4]   2> 3056544 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[A5F19B037E49F612]) [    
] o.a.s.c.a.s.SimSolrCloudTestCase ##            - SUCCEEDED       1
   [junit4] IGNOR/A 0.00s J1 | TestTriggerIntegration.testCooldown
   [junit4]    > Assumption #1: 'badapple' test group is disabled 
(@BadApple(bugUrl=https://issues.apache.org/jira/browse/SOLR-12028))
   [junit4] IGNOR/A 0.00s J1 | 
TestTriggerIntegration.testNodeLostTriggerRestoreState
   [junit4]    > Assumption #1: 'badapple' test group is disabled 
(@BadApple(bugUrl=https://issues.apache.org/jira/browse/SOLR-12028))
   [junit4]   2> 3056548 INFO  
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[A5F19B037E49F612]) [   
 ] o.a.s.SolrTestCaseJ4 ###Starting testTriggerThrottling
   [junit4]   2> 3056548 DEBUG 
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[A5F19B037E49F612]) [   
 ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode 
version 13
   [junit4]   2> 3056548 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 
13
   [junit4]   2> 3056549 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 3056549 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 3056549 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 13, lastZnodeVersion 13
   [junit4]   2> 3056549 INFO  
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[A5F19B037E49F612]) [   
 ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and 
clearing object cache...
   [junit4]   2> 3056549 WARN  (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Interrupted
   [junit4]   2> java.lang.InterruptedException
   [junit4]   2>        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014)
   [junit4]   2>        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048)
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:175)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 3056550 DEBUG 
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[A5F19B037E49F612]) [   
 ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed 
explicitly
   [junit4]   2> 3056550 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
   [junit4]   2> 3056550 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode 
version 14
   [junit4]   2> 3056550 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10009_solr, 
127.0.0.1:10007_solr]
   [junit4]   2> 3056551 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 14, lastZnodeVersion -1
   [junit4]   2> 3056551 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 
14
   [junit4]   2> 3056553 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 3056553 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 14, lastZnodeVersion 14
   [junit4]   2> 3056553 DEBUG (ScheduledTrigger-9651-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2
   [junit4]   2> 3056573 DEBUG (ScheduledTrigger-9651-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2
   [junit4]   2> 3056594 DEBUG (ScheduledTrigger-9651-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2
   [junit4]   2> 3056614 DEBUG (ScheduledTrigger-9651-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2
   [junit4]   2> 3056634 DEBUG (ScheduledTrigger-9651-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2
   [junit4]   2> 3056654 DEBUG (ScheduledTrigger-9651-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2
   [junit4]   2> 3056662 DEBUG (simCloudManagerPool-9650-thread-1) [    ] 
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 3056662 DEBUG (simCloudManagerPool-9650-thread-1) [    ] 
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 
15
   [junit4]   2> 3056662 DEBUG (simCloudManagerPool-9650-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10009_solr, 
127.0.0.1:10007_solr]
   [junit4]   2> 3056663 DEBUG (simCloudManagerPool-9650-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:10009_solr, 
127.0.0.1:10007_solr]
   [junit4]   2> 3056663 DEBUG (simCloudManagerPool-9650-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 instantiated 
with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, 
class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$ThrottlingTesterAction}],
 enabled=true}
   [junit4]   2> 3056663 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 
15
   [junit4]   2> 3056663 INFO  (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
   [junit4]   2> 3056663 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 15, lastZnodeVersion 15
   [junit4]   2> 3056663 DEBUG (ScheduledTrigger-9651-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 3056664 DEBUG (ScheduledTrigger-9651-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
   [junit4]   2> 3056664 DEBUG (simCloudManagerPool-9650-thread-2) [    ] 
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 3056664 DEBUG (simCloudManagerPool-9650-thread-2) [    ] 
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 
16
   [junit4]   2> 3056664 DEBUG (simCloudManagerPool-9650-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:10009_solr, 
127.0.0.1:10007_solr]
   [junit4]   2> 3056664 DEBUG (simCloudManagerPool-9650-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 instantiated 
with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, 
class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$ThrottlingTesterAction}],
 enabled=true}
   [junit4]   2> 3056664 DEBUG (simCloudManagerPool-9650-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10009_solr, 
127.0.0.1:10007_solr]
   [junit4]   2> 3056664 DEBUG (simCloudManagerPool-9650-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:10009_solr, 
127.0.0.1:10007_solr]
   [junit4]   2> 3056665 DEBUG (simCloudManagerPool-9650-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 instantiated 
with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, 
class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$ThrottlingTesterAction}],
 enabled=true}
   [junit4]   2> 3056665 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 
16
   [junit4]   2> 3056665 INFO  (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
   [junit4]   2> 3056665 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 16, lastZnodeVersion 16
   [junit4]   2> 3056665 DEBUG (ScheduledTrigger-9651-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 3056665 DEBUG (ScheduledTrigger-9651-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3056665 DEBUG (ScheduledTrigger-9651-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10010_solr at time 
26032721426907789
   [junit4]   2> 3056666 DEBUG (ScheduledTrigger-9651-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 firing 
registered processor for nodes: [127.0.0.1:10010_solr] added at times 
[26032721426907789], now=26032721430394739
   [junit4]   2> 3056666 DEBUG (ScheduledTrigger-9651-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"5c7c9f4930028dTd40et349ergsrftgiu0xy1ipn",
   [junit4]   2>   "source":"node_added_trigger2",
   [junit4]   2>   "eventTime":26032721426907789,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[26032721426907789],
   [junit4]   2>     "nodeNames":["127.0.0.1:10010_solr"]}}
   [junit4]   2> 3056666 DEBUG (ScheduledTrigger-9651-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, 
node_added_trigger2, node_added_trigger1]
   [junit4]   2> 3056667 DEBUG (AutoscalingActionExecutor-9652-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"5c7c9f4930028dTd40et349ergsrftgiu0xy1ipn",
   [junit4]   2>   "source":"node_added_trigger2",
   [junit4]   2>   "eventTime":26032721426907789,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[26032721426907789],
   [junit4]   2>     "_enqueue_time_":26032721452894889,
   [junit4]   2>     "nodeNames":["127.0.0.1:10010_solr"]}}
   [junit4]   2> 3056667 INFO  (AutoscalingActionExecutor-9652-thread-1) [    ] 
o.a.s.c.a.s.TestTriggerIntegration action executed from node_added_trigger2
   [junit4]   2> 3056667 DEBUG (AutoscalingActionExecutor-9652-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
   [junit4]   2> 3056667 DEBUG (AutoscalingActionExecutor-9652-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger2 after 100ms
   [junit4]   2> 3056667 DEBUG (AutoscalingActionExecutor-9652-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger1 after 100ms
   [junit4]   2> 3056667 DEBUG (AutoscalingActionExecutor-9652-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers -- processing took 39 ms for event 
id=5c7c9f4930028dTd40et349ergsrftgiu0xy1ipn
   [junit4]   2> 3056767 DEBUG (ScheduledTrigger-9651-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3056768 DEBUG (ScheduledTrigger-9651-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 3056768 DEBUG (ScheduledTrigger-9651-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3056768 DEBUG (ScheduledTrigger-9651-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10010_solr at time 
26032726541646089
   [junit4]   2> 3056768 DEBUG (ScheduledTrigger-9651-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 firing 
registered processor for nodes: [127.0.0.1:10010_solr] added at times 
[26032726541646089], now=26032726545455489
   [junit4]   2> 3056768 DEBUG (ScheduledTrigger-9651-thread-3) [    ] 
o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"5c7ca07a0cb909Td40et349ergsrftgiu0xy1ipq",
   [junit4]   2>   "source":"node_added_trigger1",
   [junit4]   2>   "eventTime":26032726541646089,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[26032726541646089],
   [junit4]   2>     "nodeNames":["127.0.0.1:10010_solr"]}}
   [junit4]   2> 3056768 DEBUG (ScheduledTrigger-9651-thread-3) [    ] 
o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, 
node_added_trigger2, node_added_trigger1]
   [junit4]   2> 3056769 DEBUG (ScheduledTrigger-9651-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 3056769 DEBUG (ScheduledTrigger-9651-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3056769 DEBUG (AutoscalingActionExecutor-9652-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"5c7ca07a0cb909Td40et349ergsrftgiu0xy1ipq",
   [junit4]   2>   "source":"node_added_trigger1",
   [junit4]   2>   "eventTime":26032726541646089,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[26032726541646089],
   [junit4]   2>     "_enqueue_time_":26032726559128789,
   [junit4]   2>     "nodeNames":["127.0.0.1:10010_solr"]}}
   [junit4]   2> 3056770 INFO  (AutoscalingActionExecutor-9652-thread-1) [    ] 
o.a.s.c.a.s.TestTriggerIntegration last action at 26032721495632289 time = 
26032726631097439
   [junit4]   2> 3056770 INFO  (AutoscalingActionExecutor-9652-thread-1) [    ] 
o.a.s.c.a.s.TestTriggerIntegration action executed from node_added_trigger1
   [junit4]   2> 3056770 DEBUG (AutoscalingActionExecutor-9652-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
   [junit4]   2> 3056770 DEBUG (AutoscalingActionExecutor-9652-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger2 after 100ms
   [junit4]   2> 3056770 DEBUG (AutoscalingActionExecutor-9652-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger1 after 100ms
   [junit4]   2> 3056770 DEBUG (AutoscalingActionExecutor-9652-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers -- processing took 51 ms for event 
id=5c7ca07a0cb909Td40et349ergsrftgiu0xy1ipq
   [junit4]   2> 3056771 DEBUG (simCloudManagerPool-9650-thread-6) [    ] 
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 3056771 DEBUG (simCloudManagerPool-9650-thread-6) [    ] 
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 
17
   [junit4]   2> 3056771 DEBUG (simCloudManagerPool-9650-thread-6) [    ] 
o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:10010_solr, 
127.0.0.1:10009_solr, 127.0.0.1:10007_solr]
   [junit4]   2> 3056771 DEBUG (simCloudManagerPool-9650-thread-6) [    ] 
o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 instantiated 
with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, 
class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$ThrottlingTesterAction}],
 enabled=true}
   [junit4]   2> 3056771 DEBUG (simCloudManagerPool-9650-thread-6) [    ] 
o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10010_solr, 
127.0.0.1:10009_solr, 127.0.0.1:10007_solr]
   [junit4]   2> 3056771 DEBUG (simCloudManagerPool-9650-thread-6) [    ] 
o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:10010_solr, 
127.0.0.1:10009_solr, 127.0.0.1:10007_solr]
   [junit4]   2> 3056771 DEBUG (simCloudManagerPool-9650-thread-6) [    ] 
o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 instantiated 
with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, 
class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$ThrottlingTesterAction}],
 enabled=true}
   [junit4]   2> 3056771 DEBUG (simCloudManagerPool-9650-thread-6) [    ] 
o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10010_solr, 
127.0.0.1:10009_solr, 127.0.0.1:10007_solr]
   [junit4]   2> 3056771 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 
17
   [junit4]   2> 3056772 INFO  (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
   [junit4]   2> 3056772 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 17, lastZnodeVersion 17
   [junit4]   2> 3056772 DEBUG (ScheduledTrigger-9651-thread-4) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with 
currently live nodes: 3
   [junit4]   2> 3056773 DEBUG (simCloudManagerPool-9650-thread-8) [    ] 
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 3056773 DEBUG (simCloudManagerPool-9650-thread-8) [    ] 
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 
18
   [junit4]   2> 3056773 DEBUG (simCloudManagerPool-9650-thread-8) [    ] 
o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:10010_solr, 
127.0.0.1:10009_solr, 127.0.0.1:10007_solr]
   [junit4]   2> 3056773 DEBUG (simCloudManagerPool-9650-thread-8) [    ] 
o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 instantiated 
with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, 
class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$ThrottlingTesterAction}],
 enabled=true}
   [junit4]   2> 3056773 DEBUG (simCloudManagerPool-9650-thread-8) [    ] 
o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10010_solr, 
127.0.0.1:10009_solr, 127.0.0.1:10007_solr]
   [junit4]   2> 3056773 DEBUG (simCloudManagerPool-9650-thread-8) [    ] 
o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:10010_solr, 
127.0.0.1:10009_solr, 127.0.0.1:10007_solr]
   [junit4]   2> 3056773 DEBUG (simCloudManagerPool-9650-thread-8) [    ] 
o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 instantiated 
with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, 
class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$ThrottlingTesterAction}],
 enabled=true}
   [junit4]   2> 3056773 DEBUG (simCloudManagerPool-9650-thread-8) [    ] 
o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10010_solr, 
127.0.0.1:10009_solr, 127.0.0.1:10007_solr]
   [junit4]   2> 3056773 DEBUG (simCloudManagerPool-9650-thread-8) [    ] 
o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10010_solr, 
127.0.0.1:10009_solr, 127.0.0.1:10007_solr]
   [junit4]   2> 3056774 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 
18
   [junit4]   2> 3056774 INFO  (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
   [junit4]   2> 3056774 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 18, lastZnodeVersion 18
   [junit4]   2> 3056774 DEBUG (ScheduledTrigger-9651-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger2 with 
currently live nodes: 3
   [junit4]   2> 3056792 DEBUG (ScheduledTrigger-9651-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with 
currently live nodes: 2
   [junit4]   2> 3056793 DEBUG (ScheduledTrigger-9651-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10010_solr
   [junit4]   2> 3056793 DEBUG (ScheduledTrigger-9651-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost 
nodes: [127.0.0.1:10010_solr]
   [junit4]   2> 3056793 DEBUG (ScheduledTrigger-9651-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"5c7ca0c42c67a5Td40et349ergsrftgiu0xy1ipt",
   [junit4]   2>   "source":"node_lost_trigger1",
   [junit4]   2>   "eventTime":26032727785236389,
   [junit4]   2>   "eventType":"NODELOST",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[26032727785236389],
   [junit4]   2>     "nodeNames":["127.0.0.1:10010_solr"]}}
   [junit4]   2> 3056793 DEBUG (ScheduledTrigger-9651-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_lost_trigger2, 
.auto_add_replicas, node_added_trigger2, node_lost_trigger1, 
node_added_trigger1]
   [junit4]   2> 3056794 DEBUG (AutoscalingActionExecutor-9652-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"5c7ca0c42c67a5Td40et349ergsrftgiu0xy1ipt",
   [junit4]   2>   "source":"node_lost_trigger1",
   [junit4]   2>   "eventTime":26032727785236389,
   [junit4]   2>   "eventType":"NODELOST",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[26032727785236389],
   [junit4]   2>     "_enqueue_time_":26032727799634939,
   [junit4]   2>     "nodeNames":["127.0.0.1:10010_solr"]}}
   [junit4]   2> 3056794 INFO  (AutoscalingActionExecutor-9652-thread-1) [    ] 
o.a.s.c.a.s.TestTriggerIntegration action executed from node_lost_trigger1
   [junit4]   2> 3056795 DEBUG (AutoscalingActionExecutor-9652-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger2 after 100ms
   [junit4]   2> 3056795 DEBUG (AutoscalingActionExecutor-9652-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
   [junit4]   2> 3056795 DEBUG (AutoscalingActionExecutor-9652-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger2 after 100ms
   [junit4]   2> 3056795 DEBUG (AutoscalingActionExecutor-9652-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger1 after 100ms
   [junit4]   2> 3056795 DEBUG (AutoscalingActionExecutor-9652-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger1 after 100ms
   [junit4]   2> 3056795 DEBUG (AutoscalingActionExecutor-9652-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers -- processing took 41 ms for event 
id=5c7ca0c42c67a5Td40et349ergsrftgiu0xy1ipt
   [junit4]   2> 3056897 DEBUG (ScheduledTrigger-9651-thread-4) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2
   [junit4]   2> 3056897 DEBUG (ScheduledTrigger-9651-thread-4) [    ] 
o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10010_solr
   [junit4]   2> 3056897 DEBUG (ScheduledTrigger-9651-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with 
currently live nodes: 2
   [junit4]   2> 3056898 DEBUG (ScheduledTrigger-9651-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 3056898 DEBUG (ScheduledTrigger-9651-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
   [junit4]   2> 3056898 DEBUG (ScheduledTrigger-9651-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger2 with 
currently live nodes: 2
   [junit4]   2> 3056898 DEBUG (ScheduledTrigger-9651-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10010_solr
   [junit4]   2> 3056898 DEBUG (ScheduledTrigger-9651-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost 
nodes: [127.0.0.1:10010_solr]
   [junit4]   2> 3056898 DEBUG (ScheduledTrigger-9651-thread-3) [    ] 
o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"5c7ca1fe31f793Td40et349ergsrftgiu0xy1ipw",
   [junit4]   2>   "source":"node_lost_trigger2",
   [junit4]   2>   "eventTime":26032733053646739,
   [junit4]   2>   "eventType":"NODELOST",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[26032733053646739],
   [junit4]   2>     "nodeNames":["127.0.0.1:10010_solr"]}}
   [junit4]   2> 3056898 DEBUG (ScheduledTrigger-9651-thread-3) [    ] 
o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_lost_trigger2, 
.auto_add_replicas, node_added_trigger2, node_lost_trigger1, 
node_added_trigger1]
   [junit4]   2> 3056899 DEBUG (ScheduledTrigger-9651-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 3056899 DEBUG (ScheduledTrigger-9651-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
   [junit4]   2> 3056901 DEBUG (AutoscalingActionExecutor-9652-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"5c7ca1fe31f793Td40et349ergsrftgiu0xy1ipw",
   [junit4]   2>   "source":"node_lost_trigger2",
   [junit4]   2>   "eventTime":26032733053646739,
   [junit4]   2>   "eventType":"NODELOST",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[26032733053646739],
   [junit4]   2>     "_enqueue_time_":26032733067809139,
   [junit4]   2>     "nodeNames":["127.0.0.1:10010_solr"]}}
   [junit4]   2> 3056901 INFO  (AutoscalingActionExecutor-9652-thread-1) [    ] 
o.a.s.c.a.s.TestTriggerIntegration last action at 26032727865685789 time = 
26032733204029789
   [junit4]   2> 3056901 INFO  (AutoscalingActionExecutor-9652-thread-1) [    ] 
o.a.s.c.a.s.TestTriggerIntegration action executed from node_lost_trigger2
   [junit4]   2> 3056901 INFO  
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[A5F19B037E49F612]) [   
 ] o.a.s.SolrTestCaseJ4 ###Ending testTriggerThrottling
   [junit4]   2> 3056901 INFO  
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[A5F19B037E49F612]) [   
 ] o.a.s.c.a.s.SimSolrCloudTestCase 
   [junit4]   2> 
   [junit4]   2> 3056901 INFO  
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[A5F19B037E49F612]) [   
 ] o.a.s.c.a.s.SimSolrCloudTestCase 
#############################################
   [junit4]   2> 3056902 INFO  
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[A5F19B037E49F612]) [   
 ] o.a.s.c.a.s.SimSolrCloudTestCase ############ FINAL CLUSTER STATS 
############
   [junit4]   2> 3056902 INFO  
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[A5F19B037E49F612]) [   
 ] o.a.s.c.a.s.SimSolrCloudTestCase 
#############################################
   [junit4]   2> 
   [junit4]   2> 3056902 INFO  
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[A5F19B037E49F612]) [   
 ] o.a.s.c.a.s.SimSolrCloudTestCase ## Live nodes:               2
   [junit4]   2> 3056902 INFO  
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[A5F19B037E49F612]) [   
 ] o.a.s.c.a.s.SimSolrCloudTestCase ## Empty nodes:      2
   [junit4]   2> 3056902 INFO  
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[A5F19B037E49F612]) [   
 ] o.a.s.c.a.s.SimSolrCloudTestCase ## Dead nodes:               1
   [junit4]   2> 3056902 INFO  
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[A5F19B037E49F612]) [   
 ] o.a.s.c.a.s.SimSolrCloudTestCase ##           127.0.0.1:10010_solr
   [junit4]   2> 3056902 INFO  
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[A5F19B037E49F612]) [   
 ] o.a.s.c.a.s.SimSolrCloudTestCase ## Collections:      []
   [junit4]   2> 3056902 INFO  
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[A5F19B037E49F612]) [   
 ] o.a.s.c.a.s.SimSolrCloudTestCase ## Max replicas per node:    0
   [junit4]   2> 3056902 INFO  
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[A5F19B037E49F612]) [   
 ] o.a.s.c.a.s.SimSolrCloudTestCase ## Min replicas per node:    0
   [junit4]   2> 3056902 INFO  
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[A5F19B037E49F612]) [   
 ] o.a.s.c.a.s.SimSolrCloudTestCase ## Total replicas:           0
   [junit4]   2> 3056902 INFO  
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[A5F19B037E49F612]) [   
 ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Final Solr op counts ##########
   [junit4]   2> 3056902 INFO  
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[A5F19B037E49F612]) [   
 ] o.a.s.c.a.s.SimSolrCloudTestCase ##           - autoscaling        4
   [junit4]   2> 3056902 INFO  
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[A5F19B037E49F612]) [   
 ] o.a.s.c.a.s.SimSolrCloudTestCase ##           - update             7
   [junit4]   2> 3056902 INFO  
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[A5F19B037E49F612]) [   
 ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Autoscaling event counts 
###########
   [junit4]   2> 3056902 INFO  
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[A5F19B037E49F612]) [   
 ] o.a.s.c.a.s.SimSolrCloudTestCase ## * Trigger: node_added_trigger1
   [junit4]   2> 3056902 INFO  
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[A5F19B037E49F612]) [   
 ] o.a.s.c.a.s.SimSolrCloudTestCase ##           - STARTED         1
   [junit4]   2> 3056902 INFO  
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[A5F19B037E49F612]) [   
 ] o.a.s.c.a.s.SimSolrCloudTestCase ##           - SUCCEEDED       1
   [junit4]   2> 3056902 INFO  
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[A5F19B037E49F612]) [   
 ] o.a.s.c.a.s.SimSolrCloudTestCase ## * Trigger: node_added_trigger2
   [junit4]   2> 3056903 INFO  
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[A5F19B037E49F612]) [   
 ] o.a.s.c.a.s.SimSolrCloudTestCase ##           - STARTED         1
   [junit4]   2> 3056903 INFO  
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[A5F19B037E49F612]) [   
 ] o.a.s.c.a.s.SimSolrCloudTestCase ##           - SUCCEEDED       1
   [junit4]   2> 3056903 INFO  
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[A5F19B037E49F612]) [   
 ] o.a.s.c.a.s.SimSolrCloudTestCase ## * Trigger: node_lost_trigger1
   [junit4]   2> 3056903 INFO  
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[A5F19B037E49F612]) [   
 ] o.a.s.c.a.s.SimSolrCloudTestCase ##           - STARTED         1
   [junit4]   2> 3056903 INFO  
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[A5F19B037E49F612]) [   
 ] o.a.s.c.a.s.SimSolrCloudTestCase ##           - SUCCEEDED       1
   [junit4]   2> 3056903 INFO  
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[A5F19B037E49F612]) [   
 ] o.a.s.c.a.s.SimSolrCloudTestCase ## * Trigger: node_lost_trigger2
   [junit4]   2> 3056903 INFO  
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[A5F19B037E49F612]) [   
 ] o.a.s.c.a.s.SimSolrCloudTestCase ##           - STARTED         1
   [junit4] IGNOR/A 0.00s J1 | TestTriggerIntegration.testEventFromRestoredState
   [junit4]    > Assumption #1: 'badapple' test group is disabled 
(@BadApple(bugUrl=https://issues.apache.org/jira/browse/SOLR-12028))
   [junit4]   2> 3056916 INFO  
(TEST-TestTriggerIntegration.testEventQueue-seed#[A5F19B037E49F612]) [    ] 
o.a.s.SolrTestCaseJ4 ###Starting testEventQueue
   [junit4]   2> 3056916 DEBUG 
(TEST-TestTriggerIntegration.testEventQueue-seed#[A5F19B037E49F612]) [    ] 
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 
19
   [junit4]   2> 3056916 DEBUG 
(TEST-TestTriggerIntegration.testEventQueue-seed#[A5F19B037E49F612]) [    ] 
o.a.s.c.a.s.SimNodeStateProvider - removing dead node values: 
127.0.0.1:10010_solr
   [junit4]   2> 3056916 INFO  
(TEST-TestTriggerIntegration.testEventQueue-seed#[A5F19B037E49F612]) [    ] 
o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing 
object cache...
   [junit4]   2> 3056916 WARN  (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are 
closed, exiting.
   [junit4]   2> 3056917 WARN  (AutoscalingActionExecutor-9652-thread-1) [    ] 
o.a.s.c.a.SystemLogListener Exception sending event to collection .system
   [junit4]   2> java.io.IOException: java.lang.InterruptedException
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.sim.SimCloudManager.request(SimCloudManager.java:462)
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.SystemLogListener.onEvent(SystemLogListener.java:122)
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerListeners.fireListeners(ScheduledTriggers.java:748)
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerListeners.fireListeners(ScheduledTriggers.java:718)
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:317)
   [junit4]   2>        at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
   [junit4]   2>        at 
java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]   2>        at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> Caused by: java.lang.InterruptedException
   [junit4]   2>        at 
java.util.concurrent.FutureTask.awaitDone(FutureTask.java:404)
   [junit4]   2>        at 
java.util.concurrent.FutureTask.get(FutureTask.java:191)
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.sim.SimCloudManager.request(SimCloudManager.java:460)
   [junit4]   2>        ... 10 more
   [junit4]   2> 3056918 DEBUG (AutoscalingActionExecutor-9652-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers -- processing took 841 ms for event 
id=5c7ca1fe31f793Td40et349ergsrftgiu0xy1ipw
   [junit4]   2> 3056918 DEBUG 
(TEST-TestTriggerIntegration.testEventQueue-seed#[A5F19B037E49F612]) [    ] 
o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 3056919 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
   [junit4]   2> 3056919 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode 
version 20
   [junit4]   2> 3056919 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10009_solr, 
127.0.0.1:10007_solr]
   [junit4]   2> 3056919 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 20, lastZnodeVersion -1
   [junit4]   2> 3056920 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 
20
   [junit4]   2> 3056920 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 3056920 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 20, lastZnodeVersion 20
   [junit4]   2> 3056920 DEBUG (ScheduledTrigger-9654-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2
   [junit4]   2> 3056941 DEBUG (ScheduledTrigger-9654-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2
   [junit4]   2> 3056961 DEBUG (ScheduledTrigger-9654-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2
   [junit4]   2> 3056981 DEBUG (ScheduledTrigger-9654-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2
   [junit4]   2> 3057002 DEBUG (ScheduledTrigger-9654-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2
   [junit4]   2> 3057021 DEBUG (simCloudManagerPool-9653-thread-1) [    ] 
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 3057021 DEBUG (simCloudManagerPool-9653-thread-1) [    ] 
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 
21
   [junit4]   2> 3057021 DEBUG (simCloudManagerPool-9653-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10009_solr, 
127.0.0.1:10007_solr]
   [junit4]   2> 3057021 INFO  (simCloudManagerPool-9653-thread-1) [    ] 
o.a.s.c.a.s.TestTriggerIntegration TestEventQueueAction instantiated
   [junit4]   2> 3057021 DEBUG (simCloudManagerPool-9653-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:10009_solr, 
127.0.0.1:10007_solr]
   [junit4]   2> 3057021 DEBUG (simCloudManagerPool-9653-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 instantiated 
with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, 
class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestEventQueueAction}],
 enabled=true}
   [junit4]   2> 3057022 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 
21
   [junit4]   2> 3057022 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
   [junit4]   2> 3057022 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 21, lastZnodeVersion 21
   [junit4]   2> 3057022 DEBUG (ScheduledTrigger-9654-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2
   [junit4]   2> 3057022 DEBUG (ScheduledTrigger-9654-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 3057022 DEBUG (ScheduledTrigger-9654-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
   [junit4]   2> 3057042 DEBUG (ScheduledTrigger-9654-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3057042 DEBUG (ScheduledTrigger-9654-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 3057042 DEBUG (ScheduledTrigger-9654-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3057042 DEBUG (ScheduledTrigger-9654-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10011_solr at time 
26032740277773689
   [junit4]   2> 3057062 DEBUG (ScheduledTrigger-9654-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3057063 DEBUG (ScheduledTrigger-9654-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 3057063 DEBUG (ScheduledTrigger-9654-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3057063 DEBUG (ScheduledTrigger-9654-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 firing 
registered processor for nodes: [127.0.0.1:10011_solr] added at times 
[26032740277773689], now=26032741292794239
   [junit4]   2> 3057063 DEBUG (ScheduledTrigger-9654-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"5c7ca3acc96579Td40et349ergsrftgiu0xy1ipz",
   [junit4]   2>   "source":"node_added_trigger1",
   [junit4]   2>   "eventTime":26032740277773689,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[26032740277773689],
   [junit4]   2>     "nodeNames":["127.0.0.1:10011_solr"]}}
   [junit4]   2> 3057063 DEBUG (ScheduledTrigger-9654-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, 
node_added_trigger1]
   [junit4]   2> 3057064 DEBUG (AutoscalingActionExecutor-9655-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"5c7ca3acc96579Td40et349ergsrftgiu0xy1ipz",
   [junit4]   2>   "source":"node_added_trigger1",
   [junit4]   2>   "eventTime":26032740277773689,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[26032740277773689],
   [junit4]   2>     "_enqueue_time_":26032741320903739,
   [junit4]   2>     "nodeNames":["127.0.0.1:10011_solr"]}}
   [junit4]   2> 3057066 INFO  (AutoscalingActionExecutor-9655-thread-1) [    ] 
o.a.s.c.a.s.TestTriggerIntegration -- event: {
   [junit4]   2>   "id":"5c7ca3acc96579Td40et349ergsrftgiu0xy1ipz",
   [junit4]   2>   "source":"node_added_trigger1",
   [junit4]   2>   "eventTime":26032740277773689,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[26032740277773689],
   [junit4]   2>     "_enqueue_time_":26032741320903739,
   [junit4]   2>     "nodeNames":["127.0.0.1:10011_solr"]}}
   [junit4]   2> 3057066 INFO  
(TEST-TestTriggerIntegration.testEventQueue-seed#[A5F19B037E49F612]) [    ] 
o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing 
object cache...
   [junit4]   2> 3057066 WARN  (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are 
closed, exiting.
   [junit4]   2> 3057067 WARN  (AutoscalingActionExecutor-9655-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers Exception executing actions
   [junit4]   2> org.apache.lucene.store.AlreadyClosedException: 
ScheduledTrigger node_added_trigger1 has been closed.
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerWrapper.dequeue(ScheduledTriggers.java:529)
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:314)
   [junit4]   2>        at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
   [junit4]   2>        at 
java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]   2>        at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 3057069 DEBUG 
(TEST-TestTriggerIntegration.testEventQueue-seed#[A5F19B037E49F612]) [    ] 
o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 3057070 DEBUG (AutoscalingActionExecutor-9655-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers -- processing took 290 ms for event 
id=5c7ca3acc96579Td40et349ergsrftgiu0xy1ipz
   [junit4]   2> 3057077 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode 
version 21
   [junit4]   2> 3057077 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10007_solr, 
127.0.0.1:10011_solr]
   [junit4]   2> 3057077 INFO  (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.s.TestTriggerIntegration TestEventQueueAction instantiated
   [junit4]   2> 3057077 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:10007_solr, 
127.0.0.1:10011_solr]
   [junit4]   2> 3057078 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 instanti

[...truncated too long message...]

  ] o.a.s.c.a.s.SimSolrCloudTestCase ## Min replicas per node:  0
   [junit4]   2> 3057238 INFO  
(TEST-TestTriggerIntegration.testEventQueue-seed#[A5F19B037E49F612]) [    ] 
o.a.s.c.a.s.SimSolrCloudTestCase ## Total replicas:          0
   [junit4]   2> 3057238 INFO  
(TEST-TestTriggerIntegration.testEventQueue-seed#[A5F19B037E49F612]) [    ] 
o.a.s.c.a.s.SimSolrCloudTestCase ######### Final Solr op counts ##########
   [junit4]   2> 3057238 INFO  
(TEST-TestTriggerIntegration.testEventQueue-seed#[A5F19B037E49F612]) [    ] 
o.a.s.c.a.s.SimSolrCloudTestCase ##          - autoscaling        1
   [junit4]   2> 3057238 INFO  
(TEST-TestTriggerIntegration.testEventQueue-seed#[A5F19B037E49F612]) [    ] 
o.a.s.c.a.s.SimSolrCloudTestCase ##          - update             3
   [junit4]   2> 3057238 INFO  
(TEST-TestTriggerIntegration.testEventQueue-seed#[A5F19B037E49F612]) [    ] 
o.a.s.c.a.s.SimSolrCloudTestCase ######### Autoscaling event counts ###########
   [junit4]   2> 3057238 INFO  
(TEST-TestTriggerIntegration.testEventQueue-seed#[A5F19B037E49F612]) [    ] 
o.a.s.c.a.s.SimSolrCloudTestCase ## * Trigger: node_added_trigger1
   [junit4]   2> 3057238 INFO  
(TEST-TestTriggerIntegration.testEventQueue-seed#[A5F19B037E49F612]) [    ] 
o.a.s.c.a.s.SimSolrCloudTestCase ##          - STARTED         2
   [junit4]   2> 3057238 INFO  
(TEST-TestTriggerIntegration.testEventQueue-seed#[A5F19B037E49F612]) [    ] 
o.a.s.c.a.s.SimSolrCloudTestCase ##          - SUCCEEDED       1
   [junit4]   2> NOTE: reproduce with: ant test  
-Dtestcase=TestTriggerIntegration -Dtests.method=testEventQueue 
-Dtests.seed=A5F19B037E49F612 -Dtests.multiplier=2 -Dtests.slow=true 
-Dtests.locale=nl -Dtests.timezone=America/Rankin_Inlet -Dtests.asserts=true 
-Dtests.file.encoding=US-ASCII
   [junit4] FAILURE 0.34s J1 | TestTriggerIntegration.testEventQueue <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: action wasn't 
interrupted
   [junit4]    >        at 
__randomizedtesting.SeedInfo.seed([A5F19B037E49F612:6C44D9AD772E30E7]:0)
   [junit4]    >        at 
org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testEventQueue(TestTriggerIntegration.java:641)
   [junit4]    >        at java.lang.Thread.run(Thread.java:748)
   [junit4] IGNOR/A 0.00s J1 | TestTriggerIntegration.testSearchRate
   [junit4]    > Assumption #1: 'badapple' test group is disabled 
(@BadApple(bugUrl=https://issues.apache.org/jira/browse/SOLR-12028))
   [junit4] IGNOR/A 0.00s J1 | 
TestTriggerIntegration.testNodeMarkersRegistration
   [junit4]    > Assumption #1: 'badapple' test group is disabled 
(@BadApple(bugUrl=https://issues.apache.org/jira/browse/SOLR-12028))
   [junit4]   2> 3057241 WARN  (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Interrupted
   [junit4]   2> java.lang.InterruptedException
   [junit4]   2>        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014)
   [junit4]   2>        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048)
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:175)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 3057242 DEBUG 
(SUITE-TestTriggerIntegration-seed#[A5F19B037E49F612]-worker) [    ] 
o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> NOTE: leaving temporary files on disk at: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.3/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.sim.TestTriggerIntegration_A5F19B037E49F612-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {}, 
docValues:{}, maxPointsInLeafNode=1482, maxMBSortInHeap=6.418393627124171, 
sim=RandomSimilarity(queryNorm=true): {}, locale=nl, 
timezone=America/Rankin_Inlet
   [junit4]   2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 
1.8.0_172 (64-bit)/cpus=4,threads=1,free=202915864,total=519569408
   [junit4]   2> NOTE: All tests run in this JVM: [FileUtilsTest, 
DocumentBuilderTest, TestLegacyFieldReuse, SystemInfoHandlerTest, 
HLLSerializationTest, HdfsAutoAddReplicasIntegrationTest, 
DeleteInactiveReplicaTest, DistributedQueueTest, RequestLoggingTest, 
MetricsConfigTest, BlockJoinFacetSimpleTest, TestDistribStateManager, 
TestDistribIDF, TestStandardQParsers, UpdateLogTest, SliceStateTest, 
TestLargeCluster, TestConfigSetImmutable, DirectoryFactoryTest, 
TestBulkSchemaAPI, BitVectorTest, TestLegacyFieldCache, NodeAddedTriggerTest, 
TestManagedSynonymFilterFactory, QueryParsingTest, TestAddFieldRealTimeGet, 
TestSolrConfigHandlerCloud, ClassificationUpdateProcessorIntegrationTest, 
TriggerIntegrationTest, OverseerTest, AutoScalingHandlerTest, TestReload, 
TestEmbeddedSolrServerSchemaAPI, SimplePostToolTest, TestCloudManagedSchema, 
TestDistributedMap, CustomCollectionTest, HdfsBasicDistributedZkTest, 
TestCloudRecovery, DistributedFacetPivotLongTailTest, 
AssignBackwardCompatibilityTest, DateRangeFieldTest, TestQueryWrapperFilter, 
TestJsonFacetRefinement, PreAnalyzedUpdateProcessorTest, 
CollectionsAPIAsyncDistributedZkTest, DistributedExpandComponentTest, 
TestCharFilters, FastVectorHighlighterTest, TestRTGBase, 
TestPostingsSolrHighlighter, TestDistributedGrouping, TestFilteredDocIdSet, 
TestBM25SimilarityFactory, TestLRUCache, 
TestOverriddenPrefixQueryForCustomFieldType, TestApiFramework, 
TestUninvertingReader, TestPartialUpdateDeduplication, TestMacros, 
PathHierarchyTokenizerFactoryTest, TestInitParams, ShardRoutingCustomTest, 
TestInitQParser, AlternateDirectoryTest, SecurityConfHandlerTest, TestLockTree, 
DistributedSuggestComponentTest, TestSerializedLuceneMatchVersion, 
HighlighterTest, DistributedMLTComponentTest, BasicFunctionalityTest, 
MultiThreadedOCPTest, TestXmlQParser, InfoHandlerTest, ZkControllerTest, 
TestMissingGroups, SpellCheckCollatorWithCollapseTest, CollectionsAPISolrJTest, 
TestLuceneIndexBackCompat, ReplaceNodeNoTargetTest, TestPolicyCloud, 
QueryResultKeyTest, UpdateRequestProcessorFactoryTest, CdcrUpdateLogTest, 
TestMaxTokenLenTokenizer, OverseerRolesTest, TestHdfsBackupRestoreCore, 
TestReRankQParserPlugin, TestConfigSetsAPI, SolrInfoBeanTest, 
SolrMetricReporterTest, TestSchemaResource, TestNamedUpdateProcessors, 
PrimitiveFieldTypeTest, TestSchemalessBufferedUpdates, 
CloudExitableDirectoryReaderTest, TestHdfsUpdateLog, RegexBytesRefFilterTest, 
SignatureUpdateProcessorFactoryTest, LIROnShardRestartTest, 
XsltUpdateRequestHandlerTest, AtomicUpdateProcessorFactoryTest, SOLR749Test, 
V2ApiIntegrationTest, SolrIndexMetricsTest, 
VMParamsZkACLAndCredentialsProvidersTest, PeerSyncTest, 
TestRequestStatusCollectionAPI, TestHdfsCloudBackupRestore, 
TestRetrieveFieldsOptimizer, StressHdfsTest, CdcrVersionReplicationTest, 
SimpleCollectionCreateDeleteTest, SuggesterFSTTest, TestCryptoKeys, 
SortByFunctionTest, JsonLoaderTest, TestFastLRUCache, TestFieldCacheSortRandom, 
TestManagedResource, ManagedSchemaRoundRobinCloudTest, 
TestUseDocValuesAsStored2, TestTlogReplica, TestRecovery, 
TestManagedSchemaThreadSafety, SolrCoreTest, LegacyCloudClusterPropTest, 
ChaosMonkeySafeLeaderTest, BasicDistributedZk2Test, UnloadDistributedZkTest, 
SyncSliceTest, ShardRoutingTest, LeaderElectionTest, TestZkChroot, 
TestFaceting, TermVectorComponentDistributedTest, TestJoin, TestRangeQuery, 
ShowFileRequestHandlerTest, SolrIndexSplitterTest, TestFoldingMultitermQuery, 
DocValuesTest, SuggesterTSTTest, SuggesterTest, TestTrie, SpatialFilterTest, 
SuggesterWFSTTest, TestCSVLoader, PolyFieldTest, WordBreakSolrSpellCheckerTest, 
SchemaVersionSpecificBehaviorTest, SolrCoreCheckLockOnStartupTest, 
FieldMutatingUpdateProcessorTest, TestAtomicUpdateErrorCases, 
QueryEqualityTest, DirectUpdateHandlerOptimizeTest, 
DefaultValueUpdateProcessorTest, TestSolrDeletionPolicy1, CacheHeaderTest, 
TestSurroundQueryParser, LukeRequestHandlerTest, DisMaxRequestHandlerTest, 
TestQueryUtils, TestWriterPerf, TestReversedWildcardFilterFactory, 
TermsComponentTest, XmlUpdateRequestHandlerTest, RequestHandlersTest, 
TermVectorComponentTest, TestIndexingPerformance, MoreLikeThisHandlerTest, 
RequiredFieldsTest, IndexSchemaRuntimeFieldTest, LoggingHandlerTest, 
SolrPluginUtilsTest, IndexSchemaTest, UniqFieldsUpdateProcessorFactoryTest, 
BinaryUpdateRequestHandlerTest, TestPHPSerializedResponseWriter, 
CSVRequestHandlerTest, TestLFUCache, TestFuzzyAnalyzedSuggestions, 
TestXIncludeConfig, TestPerFieldSimilarity, 
TestLMJelinekMercerSimilarityFactory, TestIBSimilarityFactory, 
URLClassifyProcessorTest, PreAnalyzedFieldTest, TestSuggestSpellingConverter, 
DateFieldTest, SpellingQueryConverterTest, ClusterStateTest, TestUtils, 
CursorPagingTest, TestCrossCoreJoin, TestCursorMarkWithoutUniqueKey, 
TestDistributedMissingSort, TestHighlightDedupGrouping, TestTolerantSearch, 
TestEmbeddedSolrServerConstructors, ActionThrottleTest, 
AsyncCallRequestStatusResponseTest, ChaosMonkeySafeLeaderWithPullReplicasTest, 
CleanupOldIndexTest, ConcurrentCreateRoutedAliasTest, ConfigSetsAPITest, 
ConnectionManagerTest, CreateCollectionCleanupTest, CreateRoutedAliasTest, 
DistribCursorPagingTest, DocValuesNotIndexedTest, ForceLeaderTest, 
LeaderFailoverAfterPartitionTest, LeaderFailureAfterFreshStartTest, 
NodeMutatorTest, OutOfBoxZkACLAndCredentialsProvidersTest, 
OverriddenZkACLAndCredentialsProvidersTest, PeerSyncReplicationTest, 
RestartWhileUpdatingTest, SSLMigrationTest, SaslZkACLProviderTest, 
SolrCloudExampleTest, TestClusterProperties, TestConfigSetsAPIExclusivity, 
TestConfigSetsAPIZkFailure, TestDownShardTolerantSearch, 
TestExclusionRuleCollectionAccess, TestLeaderElectionWithEmptyReplica, 
TestRandomRequestDistribution, TestRequestForwarding, TestSegmentSorting, 
TestSolrCloudWithKerberosAlt, TestStressInPlaceUpdates, 
CollectionTooManyReplicasTest, TestCollectionsAPIViaSolrCloudCluster, 
TestReplicaProperties, ExecutePlanActionTest, TestPolicyCloud, 
TestTriggerIntegration]
   [junit4] Completed [591/783 (1!)] on J1 in 1.33s, 11 tests, 1 failure, 6 
skipped <<< FAILURES!

[...truncated 50177 lines...]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to