Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/1795/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseG1GC

3 tests failed.
FAILED:  
org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testEventQueue

Error Message:
action wasn't interrupted

Stack Trace:
java.lang.AssertionError: action wasn't interrupted
        at 
__randomizedtesting.SeedInfo.seed([F64294E2F6294EBA:3FF7D64CFF4E884F]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at 
org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testEventQueue(TestTriggerIntegration.java:654)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  
org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testEventQueue

Error Message:
action wasn't interrupted

Stack Trace:
java.lang.AssertionError: action wasn't interrupted
        at 
__randomizedtesting.SeedInfo.seed([F64294E2F6294EBA:3FF7D64CFF4E884F]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at 
org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testEventQueue(TestTriggerIntegration.java:654)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  
org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testEventQueue

Error Message:
action did not start

Stack Trace:
java.lang.AssertionError: action did not start
        at 
__randomizedtesting.SeedInfo.seed([F64294E2F6294EBA:3FF7D64CFF4E884F]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at 
org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testEventQueue(TestTriggerIntegration.java:640)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 14144 lines...]
   [junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration
   [junit4]   2> Creating dataDir: 
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.sim.TestTriggerIntegration_F64294E2F6294EBA-001/init-core-data-001
   [junit4]   2> 3238810 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and 
.scheduled_maintenance triggers
   [junit4]   2> 3238810 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode 
version 0
   [junit4] IGNOR/A 0.00s J1 | TestTriggerIntegration.testListeners
   [junit4]    > Assumption #1: 'badapple' test group is disabled 
(@BadApple(bugUrl=https://issues.apache.org/jira/browse/SOLR-12028))
   [junit4]   2> 3238811 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion -1
   [junit4]   2> 3238811 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 0
   [junit4] IGNOR/A 0.00s J1 | 
TestTriggerIntegration.testNodeLostTriggerRestoreState
   [junit4]    > Assumption #1: 'badapple' test group is disabled 
(@BadApple(bugUrl=https://issues.apache.org/jira/browse/SOLR-12028))
   [junit4] IGNOR/A 0.00s J1 | TestTriggerIntegration.testNodeLostTrigger
   [junit4]    > Assumption #1: 'badapple' test group is disabled 
(@BadApple(bugUrl=https://issues.apache.org/jira/browse/SOLR-12028))
   [junit4] IGNOR/A 0.00s J1 | TestTriggerIntegration.testSearchRate
   [junit4]    > Assumption #1: 'badapple' test group is disabled 
(@BadApple(bugUrl=https://issues.apache.org/jira/browse/SOLR-12028))
   [junit4]   2> 3238812 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial 
livenodes: [127.0.0.1:10021_solr, 127.0.0.1:10020_solr]
   [junit4] IGNOR/A 0.00s J1 | TestTriggerIntegration.testEventFromRestoredState
   [junit4]    > Assumption #1: 'badapple' test group is disabled 
(@BadApple(bugUrl=https://issues.apache.org/jira/browse/SOLR-12028))
   [junit4]   2> 3238812 DEBUG (ScheduledTrigger-11981-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2
   [junit4] IGNOR/A 0.00s J1 | TestTriggerIntegration.testCooldown
   [junit4]    > Assumption #1: 'badapple' test group is disabled 
(@BadApple(bugUrl=https://issues.apache.org/jira/browse/SOLR-12028))
   [junit4]   2> 3238812 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 3238812 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion 0
   [junit4]   2> 3238823 DEBUG 
(TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F64294E2F6294EBA]) [    
] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode 
version 1
   [junit4]   2> 3238823 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
   [junit4]   2> 3238823 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F64294E2F6294EBA]) [    
] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing 
object cache...
   [junit4]   2> 3238830 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 3238830 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 3238830 WARN  (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Interrupted
   [junit4]   2> java.lang.InterruptedException: null
   [junit4]   2>        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1220)
 ~[?:1.8.0_162]
   [junit4]   2>        at 
java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:335)
 ~[?:1.8.0_162]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:169)
 [java/:?]
   [junit4]   2>        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_162]
   [junit4]   2> 3238830 DEBUG 
(TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F64294E2F6294EBA]) [    
] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 3238831 DEBUG 
(TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F64294E2F6294EBA]) [    
] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 3238831 DEBUG 
(TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F64294E2F6294EBA]) [    
] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 3238831 DEBUG 
(TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F64294E2F6294EBA]) [    
] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool 
executor
   [junit4]   2> 3238831 DEBUG 
(TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F64294E2F6294EBA]) [    
] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 3238831 DEBUG 
(TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F64294E2F6294EBA]) [    
] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed 
explicitly
   [junit4]   2> 3238832 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and 
.scheduled_maintenance triggers
   [junit4]   2> 3238832 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode 
version 2
   [junit4]   2> 3238832 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion -1
   [junit4]   2> 3238832 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
   [junit4]   2> 3238832 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial 
livenodes: [127.0.0.1:10021_solr, 127.0.0.1:10020_solr]
   [junit4]   2> 3238833 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 3238833 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
   [junit4]   2> 3238833 DEBUG (ScheduledTrigger-11984-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2
   [junit4]   2> 3238855 DEBUG (ScheduledTrigger-11984-thread-4) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2
   [junit4]   2> 3238875 DEBUG (ScheduledTrigger-11984-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2
   [junit4]   2> 3238900 DEBUG (ScheduledTrigger-11984-thread-4) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2
   [junit4]   2> 3238920 DEBUG (ScheduledTrigger-11984-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2
   [junit4]   2> 3238932 DEBUG (simCloudManagerPool-11983-thread-1) [    ] 
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 3238932 DEBUG (simCloudManagerPool-11983-thread-1) [    ] 
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 
3
   [junit4]   2> 3238933 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
   [junit4]   2> 3238933 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 3238933 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
   [junit4]   2> 3238935 DEBUG (simCloudManagerPool-11983-thread-2) [    ] 
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 3238935 DEBUG (simCloudManagerPool-11983-thread-2) [    ] 
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 
4
   [junit4]   2> 3238935 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
   [junit4]   2> 3238936 INFO  (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
   [junit4]   2> 3238936 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - Initial 
livenodes: [127.0.0.1:10021_solr, 127.0.0.1:10020_solr]
   [junit4]   2> 3238936 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated 
with properties: {event=nodeAdded, waitFor=3, actions=[{name=test, 
class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestTriggerAction}],
 enabled=true}
   [junit4]   2> 3238936 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
   [junit4]   2> 3238936 DEBUG (ScheduledTrigger-11984-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 3238936 DEBUG (ScheduledTrigger-11984-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3238936 DEBUG (ScheduledTrigger-11984-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10022_solr at time 
2479955644076439
   [junit4]   2> 3238940 DEBUG (ScheduledTrigger-11984-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3238957 DEBUG (ScheduledTrigger-11984-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 3238957 DEBUG (ScheduledTrigger-11984-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3238961 DEBUG (ScheduledTrigger-11984-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3238977 DEBUG (ScheduledTrigger-11984-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 3238977 DEBUG (ScheduledTrigger-11984-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3238981 DEBUG (ScheduledTrigger-11984-thread-4) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3239000 DEBUG (ScheduledTrigger-11984-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 3239000 DEBUG (ScheduledTrigger-11984-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3239000 DEBUG (ScheduledTrigger-11984-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger firing 
registered processor for nodes: [127.0.0.1:10022_solr] added at times 
[2479955644076439], now=2479958836909489
   [junit4]   2> 3239000 DEBUG (ScheduledTrigger-11984-thread-4) [    ] 
o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"8cf81ab395197T71cpd1fni40admcqk3y2085dm",
   [junit4]   2>   "source":"node_added_trigger",
   [junit4]   2>   "eventTime":2479955644076439,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[2479955644076439],
   [junit4]   2>     "nodeNames":["127.0.0.1:10022_solr"]}}
   [junit4]   2> 3239000 DEBUG (ScheduledTrigger-11984-thread-4) [    ] 
o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, 
node_added_trigger]
   [junit4]   2> 3239001 DEBUG (ScheduledTrigger-11984-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - removing 
marker path: /autoscaling/nodeAdded/127.0.0.1:10022_solr
   [junit4]   2> 3239002 DEBUG (AutoscalingActionExecutor-11985-thread-1) [    
] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"8cf81ab395197T71cpd1fni40admcqk3y2085dm",
   [junit4]   2>   "source":"node_added_trigger",
   [junit4]   2>   "eventTime":2479955644076439,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[2479955644076439],
   [junit4]   2>     "_enqueue_time_":2479958868013589,
   [junit4]   2>     "nodeNames":["127.0.0.1:10022_solr"]}}
   [junit4]   2> 3239003 DEBUG (AutoscalingActionExecutor-11985-thread-1) [    
] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
   [junit4]   2> 3239003 DEBUG (AutoscalingActionExecutor-11985-thread-1) [    
] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger after 100ms
   [junit4]   2> 3239003 DEBUG (AutoscalingActionExecutor-11985-thread-1) [    
] o.a.s.c.a.ScheduledTriggers -- processing took 98 ms for event 
id=8cf81ab395197T71cpd1fni40admcqk3y2085dm
   [junit4]   2> 3239004 DEBUG (simCloudManagerPool-11983-thread-4) [    ] 
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 3239004 DEBUG (simCloudManagerPool-11983-thread-4) [    ] 
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 
5
   [junit4]   2> 3239004 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
   [junit4]   2> 3239004 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
   [junit4]   2> 3239045 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F64294E2F6294EBA]) [    
] o.a.s.c.a.s.SimSolrCloudTestCase 
   [junit4]   2> 
   [junit4]   2> 3239045 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F64294E2F6294EBA]) [    
] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
   [junit4]   2> 3239045 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F64294E2F6294EBA]) [    
] o.a.s.c.a.s.SimSolrCloudTestCase ############ FINAL CLUSTER STATS ############
   [junit4]   2> 3239045 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F64294E2F6294EBA]) [    
] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
   [junit4]   2> 
   [junit4]   2> 3239045 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F64294E2F6294EBA]) [    
] o.a.s.c.a.s.SimSolrCloudTestCase ## Live nodes:                3
   [junit4]   2> 3239045 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F64294E2F6294EBA]) [    
] o.a.s.c.a.s.SimSolrCloudTestCase ## Empty nodes:       3
   [junit4]   2> 3239045 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F64294E2F6294EBA]) [    
] o.a.s.c.a.s.SimSolrCloudTestCase ## Dead nodes:                0
   [junit4]   2> 3239045 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F64294E2F6294EBA]) [    
] o.a.s.c.a.s.SimSolrCloudTestCase ## Collections:       []
   [junit4]   2> 3239045 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F64294E2F6294EBA]) [    
] o.a.s.c.a.s.SimSolrCloudTestCase ## Max replicas per node:     0
   [junit4]   2> 3239045 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F64294E2F6294EBA]) [    
] o.a.s.c.a.s.SimSolrCloudTestCase ## Min replicas per node:     0
   [junit4]   2> 3239045 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F64294E2F6294EBA]) [    
] o.a.s.c.a.s.SimSolrCloudTestCase ## Total replicas:            0
   [junit4]   2> 3239045 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F64294E2F6294EBA]) [    
] o.a.s.c.a.s.SimSolrCloudTestCase ######### Final Solr op counts ##########
   [junit4]   2> 3239045 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F64294E2F6294EBA]) [    
] o.a.s.c.a.s.SimSolrCloudTestCase ##            - autoscaling        3
   [junit4]   2> 3239045 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F64294E2F6294EBA]) [    
] o.a.s.c.a.s.SimSolrCloudTestCase ##            - update             2
   [junit4]   2> 3239045 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F64294E2F6294EBA]) [    
] o.a.s.c.a.s.SimSolrCloudTestCase ######### Autoscaling event counts 
###########
   [junit4]   2> 3239046 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F64294E2F6294EBA]) [    
] o.a.s.c.a.s.SimSolrCloudTestCase ## * Trigger: node_added_trigger
   [junit4]   2> 3239047 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F64294E2F6294EBA]) [    
] o.a.s.c.a.s.SimSolrCloudTestCase ##            - STARTED         1
   [junit4]   2> 3239047 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F64294E2F6294EBA]) [    
] o.a.s.c.a.s.SimSolrCloudTestCase ##            - SUCCEEDED       1
   [junit4] IGNOR/A 0.00s J1 | 
TestTriggerIntegration.testNodeMarkersRegistration
   [junit4]    > Assumption #1: 'badapple' test group is disabled 
(@BadApple(bugUrl=https://issues.apache.org/jira/browse/SOLR-12028))
   [junit4]   2> 3239060 DEBUG 
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [   
 ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode 
version 6
   [junit4]   2> 3239060 INFO  
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [   
 ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and 
clearing object cache...
   [junit4]   2> 3239060 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
   [junit4]   2> 3239060 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 3239060 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 3239061 DEBUG 
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [   
 ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 3239061 DEBUG 
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [   
 ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 3239061 DEBUG 
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [   
 ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 3239061 DEBUG 
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [   
 ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool 
executor
   [junit4]   2> 3239061 DEBUG 
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [   
 ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 3239061 DEBUG 
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [   
 ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed 
explicitly
   [junit4]   2> 3239060 WARN  (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Interrupted
   [junit4]   2> java.lang.InterruptedException: null
   [junit4]   2>        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1220)
 ~[?:1.8.0_162]
   [junit4]   2>        at 
java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:335)
 ~[?:1.8.0_162]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:169)
 [java/:?]
   [junit4]   2>        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_162]
   [junit4]   2> 3239062 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and 
.scheduled_maintenance triggers
   [junit4]   2> 3239062 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode 
version 7
   [junit4]   2> 3239062 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion -1
   [junit4]   2> 3239062 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 7
   [junit4]   2> 3239063 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial 
livenodes: [127.0.0.1:10021_solr, 127.0.0.1:10020_solr]
   [junit4]   2> 3239063 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 3239063 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion 7
   [junit4]   2> 3239063 DEBUG (ScheduledTrigger-11987-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2
   [junit4]   2> 3239090 DEBUG (ScheduledTrigger-11987-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2
   [junit4]   2> 3239117 DEBUG (ScheduledTrigger-11987-thread-4) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2
   [junit4]   2> 3239137 DEBUG (ScheduledTrigger-11987-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2
   [junit4]   2> 3239157 DEBUG (ScheduledTrigger-11987-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2
   [junit4]   2> 3239163 DEBUG (simCloudManagerPool-11986-thread-1) [    ] 
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 3239163 DEBUG (simCloudManagerPool-11986-thread-1) [    ] 
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 
8
   [junit4]   2> 3239169 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 8
   [junit4]   2> 3239169 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 3239169 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 8, lastZnodeVersion 8
   [junit4]   2> 3239171 DEBUG (simCloudManagerPool-11986-thread-2) [    ] 
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 3239171 DEBUG (simCloudManagerPool-11986-thread-2) [    ] 
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 
9
   [junit4]   2> 3239172 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 9
   [junit4]   2> 3239172 INFO  (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
   [junit4]   2> 3239172 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 - Initial 
livenodes: [127.0.0.1:10021_solr, 127.0.0.1:10020_solr]
   [junit4]   2> 3239172 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 instantiated 
with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, 
class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$ThrottlingTesterAction}],
 enabled=true}
   [junit4]   2> 3239172 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 9, lastZnodeVersion 9
   [junit4]   2> 3239172 DEBUG (ScheduledTrigger-11987-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 3239172 DEBUG (ScheduledTrigger-11987-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
   [junit4]   2> 3239173 DEBUG (simCloudManagerPool-11986-thread-3) [    ] 
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 3239173 DEBUG (simCloudManagerPool-11986-thread-3) [    ] 
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 
10
   [junit4]   2> 3239173 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 
10
   [junit4]   2> 3239173 INFO  (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
   [junit4]   2> 3239173 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 - Initial 
livenodes: [127.0.0.1:10021_solr, 127.0.0.1:10020_solr]
   [junit4]   2> 3239173 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 instantiated 
with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, 
class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$ThrottlingTesterAction}],
 enabled=true}
   [junit4]   2> 3239174 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 10, lastZnodeVersion 10
   [junit4]   2> 3239174 DEBUG (ScheduledTrigger-11987-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 3239174 DEBUG (ScheduledTrigger-11987-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
   [junit4]   2> 3239180 DEBUG (ScheduledTrigger-11987-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3239200 DEBUG (ScheduledTrigger-11987-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3239201 DEBUG (ScheduledTrigger-11987-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 3239201 DEBUG (ScheduledTrigger-11987-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3239201 DEBUG (ScheduledTrigger-11987-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10023_solr at time 
2479968915485789
   [junit4]   2> 3239201 DEBUG (ScheduledTrigger-11987-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 firing 
registered processor for nodes: [127.0.0.1:10023_solr] added at times 
[2479968915485789], now=2479968917164989
   [junit4]   2> 3239201 DEBUG (ScheduledTrigger-11987-thread-3) [    ] 
o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"8cf84c242f45dT71cpd1fni40admcqk3y2085dp",
   [junit4]   2>   "source":"node_added_trigger1",
   [junit4]   2>   "eventTime":2479968915485789,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[2479968915485789],
   [junit4]   2>     "nodeNames":["127.0.0.1:10023_solr"]}}
   [junit4]   2> 3239201 DEBUG (ScheduledTrigger-11987-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 3239202 DEBUG (ScheduledTrigger-11987-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3239202 DEBUG (ScheduledTrigger-11987-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10023_solr at time 
2479968929883589
   [junit4]   2> 3239202 DEBUG (ScheduledTrigger-11987-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 firing 
registered processor for nodes: [127.0.0.1:10023_solr] added at times 
[2479968929883589], now=2479968931215039
   [junit4]   2> 3239202 DEBUG (ScheduledTrigger-11987-thread-4) [    ] 
o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"8cf84c31ea5c5T71cpd1fni40admcqk3y2085dq",
   [junit4]   2>   "source":"node_added_trigger2",
   [junit4]   2>   "eventTime":2479968929883589,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[2479968929883589],
   [junit4]   2>     "nodeNames":["127.0.0.1:10023_solr"]}}
   [junit4]   2> 3239205 DEBUG (ScheduledTrigger-11987-thread-3) [    ] 
o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, 
node_added_trigger2, node_added_trigger1]
   [junit4]   2> 3239206 DEBUG (ScheduledTrigger-11987-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 - removing 
marker path: /autoscaling/nodeAdded/127.0.0.1:10023_solr
   [junit4]   2> 3239206 DEBUG (AutoscalingActionExecutor-11988-thread-1) [    
] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"8cf84c242f45dT71cpd1fni40admcqk3y2085dp",
   [junit4]   2>   "source":"node_added_trigger1",
   [junit4]   2>   "eventTime":2479968915485789,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[2479968915485789],
   [junit4]   2>     "_enqueue_time_":2479969084828039,
   [junit4]   2>     "nodeNames":["127.0.0.1:10023_solr"]}}
   [junit4]   2> 3239206 INFO  (AutoscalingActionExecutor-11988-thread-1) [    
] o.a.s.c.a.s.TestTriggerIntegration action executed from node_added_trigger1
   [junit4]   2> 3239208 DEBUG (AutoscalingActionExecutor-11988-thread-1) [    
] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
   [junit4]   2> 3239208 DEBUG (AutoscalingActionExecutor-11988-thread-1) [    
] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger2 after 100ms
   [junit4]   2> 3239208 DEBUG (AutoscalingActionExecutor-11988-thread-1) [    
] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger1 after 100ms
   [junit4]   2> 3239208 DEBUG (AutoscalingActionExecutor-11988-thread-1) [    
] o.a.s.c.a.ScheduledTriggers -- processing took 71 ms for event 
id=8cf84c242f45dT71cpd1fni40admcqk3y2085dp
   [junit4]   2> 3239308 DEBUG (ScheduledTrigger-11987-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3239308 DEBUG (ScheduledTrigger-11987-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 3239308 DEBUG (ScheduledTrigger-11987-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3239308 DEBUG (ScheduledTrigger-11987-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 3239308 DEBUG (ScheduledTrigger-11987-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3239308 DEBUG (ScheduledTrigger-11987-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 firing 
registered processor for nodes: [127.0.0.1:10023_solr] added at times 
[2479968929883589], now=2479974246723839
   [junit4]   2> 3239308 DEBUG (ScheduledTrigger-11987-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"8cf84c31ea5c5T71cpd1fni40admcqk3y2085du",
   [junit4]   2>   "source":"node_added_trigger2",
   [junit4]   2>   "eventTime":2479968929883589,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[2479968929883589],
   [junit4]   2>     "nodeNames":["127.0.0.1:10023_solr"]}}
   [junit4]   2> 3239308 DEBUG (ScheduledTrigger-11987-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, 
node_added_trigger2, node_added_trigger1]
   [junit4]   2> 3239309 DEBUG (ScheduledTrigger-11987-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 - removing 
marker path: /autoscaling/nodeAdded/127.0.0.1:10023_solr
   [junit4]   2> 3239309 DEBUG (AutoscalingActionExecutor-11988-thread-1) [    
] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"8cf84c31ea5c5T71cpd1fni40admcqk3y2085du",
   [junit4]   2>   "source":"node_added_trigger2",
   [junit4]   2>   "eventTime":2479968929883589,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[2479968929883589],
   [junit4]   2>     "_enqueue_time_":2479974261884939,
   [junit4]   2>     "nodeNames":["127.0.0.1:10023_solr"]}}
   [junit4]   2> 3239309 INFO  (AutoscalingActionExecutor-11988-thread-1) [    
] o.a.s.c.a.s.TestTriggerIntegration last action at 2479969176680839 time = 
2479974314424839
   [junit4]   2> 3239309 INFO  (AutoscalingActionExecutor-11988-thread-1) [    
] o.a.s.c.a.s.TestTriggerIntegration action executed from node_added_trigger2
   [junit4]   2> 3239310 DEBUG (AutoscalingActionExecutor-11988-thread-1) [    
] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
   [junit4]   2> 3239310 DEBUG (AutoscalingActionExecutor-11988-thread-1) [    
] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger2 after 100ms
   [junit4]   2> 3239310 DEBUG (AutoscalingActionExecutor-11988-thread-1) [    
] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger1 after 100ms
   [junit4]   2> 3239311 DEBUG (AutoscalingActionExecutor-11988-thread-1) [    
] o.a.s.c.a.ScheduledTriggers -- processing took 79 ms for event 
id=8cf84c31ea5c5T71cpd1fni40admcqk3y2085du
   [junit4]   2> 3239311 DEBUG (simCloudManagerPool-11986-thread-8) [    ] 
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 3239311 DEBUG (simCloudManagerPool-11986-thread-8) [    ] 
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 
11
   [junit4]   2> 3239311 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 
11
   [junit4]   2> 3239311 INFO  (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
   [junit4]   2> 3239311 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger node_lost_trigger1 - Initial 
livenodes: [127.0.0.1:10021_solr, 127.0.0.1:10020_solr, 127.0.0.1:10023_solr]
   [junit4]   2> 3239311 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 11, lastZnodeVersion 11
   [junit4]   2> 3239311 DEBUG (ScheduledTrigger-11987-thread-4) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with 
currently live nodes: 3
   [junit4]   2> 3239312 DEBUG (simCloudManagerPool-11986-thread-10) [    ] 
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 3239312 DEBUG (simCloudManagerPool-11986-thread-10) [    ] 
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 
12
   [junit4]   2> 3239313 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 
12
   [junit4]   2> 3239313 INFO  (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
   [junit4]   2> 3239313 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger node_lost_trigger2 - Initial 
livenodes: [127.0.0.1:10021_solr, 127.0.0.1:10020_solr, 127.0.0.1:10023_solr]
   [junit4]   2> 3239313 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 12, lastZnodeVersion 12
   [junit4]   2> 3239313 DEBUG (ScheduledTrigger-11987-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger2 with 
currently live nodes: 2
   [junit4]   2> 3239313 DEBUG (ScheduledTrigger-11987-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10023_solr
   [junit4]   2> 3239313 DEBUG (ScheduledTrigger-11987-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost 
nodes: [127.0.0.1:10023_solr]
   [junit4]   2> 3239313 DEBUG (ScheduledTrigger-11987-thread-2) [    ] 
o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"8cf860ffc4707T71cpd1fni40admcqk3y2085dx",
   [junit4]   2>   "source":"node_lost_trigger2",
   [junit4]   2>   "eventTime":2479974514444039,
   [junit4]   2>   "eventType":"NODELOST",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[2479974514444039],
   [junit4]   2>     "nodeNames":["127.0.0.1:10023_solr"]}}
   [junit4]   2> 3239313 DEBUG (ScheduledTrigger-11987-thread-2) [    ] 
o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_lost_trigger2, 
.auto_add_replicas, node_added_trigger2, node_lost_trigger1, 
node_added_trigger1]
   [junit4]   2> 3239314 DEBUG (AutoscalingActionExecutor-11988-thread-1) [    
] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"8cf860ffc4707T71cpd1fni40admcqk3y2085dx",
   [junit4]   2>   "source":"node_lost_trigger2",
   [junit4]   2>   "eventTime":2479974514444039,
   [junit4]   2>   "eventType":"NODELOST",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[2479974514444039],
   [junit4]   2>     "_enqueue_time_":2479974521221589,
   [junit4]   2>     "nodeNames":["127.0.0.1:10023_solr"]}}
   [junit4]   2> 3239314 INFO  (AutoscalingActionExecutor-11988-thread-1) [    
] o.a.s.c.a.s.TestTriggerIntegration action executed from node_lost_trigger2
   [junit4]   2> 3239315 DEBUG (AutoscalingActionExecutor-11988-thread-1) [    
] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger2 after 100ms
   [junit4]   2> 3239315 DEBUG (AutoscalingActionExecutor-11988-thread-1) [    
] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
   [junit4]   2> 3239315 DEBUG (AutoscalingActionExecutor-11988-thread-1) [    
] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger2 after 100ms
   [junit4]   2> 3239315 DEBUG (AutoscalingActionExecutor-11988-thread-1) [    
] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger1 after 100ms
   [junit4]   2> 3239315 DEBUG (AutoscalingActionExecutor-11988-thread-1) [    
] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger1 after 100ms
   [junit4]   2> 3239315 DEBUG (AutoscalingActionExecutor-11988-thread-1) [    
] o.a.s.c.a.ScheduledTriggers -- processing took 51 ms for event 
id=8cf860ffc4707T71cpd1fni40admcqk3y2085dx
   [junit4]   2> 3239415 DEBUG (ScheduledTrigger-11987-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2
   [junit4]   2> 3239416 DEBUG (ScheduledTrigger-11987-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 3239416 DEBUG (ScheduledTrigger-11987-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
   [junit4]   2> 3239416 DEBUG (ScheduledTrigger-11987-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger2 with 
currently live nodes: 2
   [junit4]   2> 3239417 DEBUG (ScheduledTrigger-11987-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 3239417 DEBUG (ScheduledTrigger-11987-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
   [junit4]   2> 3239416 DEBUG (ScheduledTrigger-11987-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10023_solr
   [junit4]   2> 3239418 DEBUG (ScheduledTrigger-11987-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with 
currently live nodes: 2
   [junit4]   2> 3239418 DEBUG (ScheduledTrigger-11987-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10023_solr
   [junit4]   2> 3239418 DEBUG (ScheduledTrigger-11987-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost 
nodes: [127.0.0.1:10023_solr]
   [junit4]   2> 3239418 DEBUG (ScheduledTrigger-11987-thread-2) [    ] 
o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"8cf8747a21aafT71cpd1fni40admcqk3y2085e0",
   [junit4]   2>   "source":"node_lost_trigger1",
   [junit4]   2>   "eventTime":2479979743025839,
   [junit4]   2>   "eventType":"NODELOST",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[2479979743025839],
   [junit4]   2>     "nodeNames":["127.0.0.1:10023_solr"]}}
   [junit4]   2> 3239418 DEBUG (ScheduledTrigger-11987-thread-2) [    ] 
o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_lost_trigger2, 
.auto_add_replicas, node_added_trigger2, node_lost_trigger1, 
node_added_trigger1]
   [junit4]   2> 3239419 DEBUG (AutoscalingActionExecutor-11988-thread-1) [    
] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"8cf8747a21aafT71cpd1fni40admcqk3y2085e0",
   [junit4]   2>   "source":"node_lost_trigger1",
   [junit4]   2>   "eventTime":2479979743025839,
   [junit4]   2>   "eventType":"NODELOST",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[2479979743025839],
   [junit4]   2>     "_enqueue_time_":2479979754974189,
   [junit4]   2>     "nodeNames":["127.0.0.1:10023_solr"]}}
   [junit4]   2> 3239419 INFO  (AutoscalingActionExecutor-11988-thread-1) [    
] o.a.s.c.a.s.TestTriggerIntegration last action at 2479974568763489 time = 
2479979810239539
   [junit4]   2> 3239419 INFO  (AutoscalingActionExecutor-11988-thread-1) [    
] o.a.s.c.a.s.TestTriggerIntegration action executed from node_lost_trigger1
   [junit4]   2> 3239420 INFO  
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [   
 ] o.a.s.c.a.s.SimSolrCloudTestCase 
   [junit4]   2> 
   [junit4]   2> 3239420 INFO  
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [   
 ] o.a.s.c.a.s.SimSolrCloudTestCase 
#############################################
   [junit4]   2> 3239420 INFO  
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [   
 ] o.a.s.c.a.s.SimSolrCloudTestCase ############ FINAL CLUSTER STATS 
############
   [junit4]   2> 3239420 INFO  
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [   
 ] o.a.s.c.a.s.SimSolrCloudTestCase 
#############################################
   [junit4]   2> 
   [junit4]   2> 3239420 INFO  
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [   
 ] o.a.s.c.a.s.SimSolrCloudTestCase ## Live nodes:               2
   [junit4]   2> 3239420 INFO  
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [   
 ] o.a.s.c.a.s.SimSolrCloudTestCase ## Empty nodes:      2
   [junit4]   2> 3239420 INFO  
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [   
 ] o.a.s.c.a.s.SimSolrCloudTestCase ## Dead nodes:               1
   [junit4]   2> 3239420 INFO  
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [   
 ] o.a.s.c.a.s.SimSolrCloudTestCase ##           127.0.0.1:10023_solr
   [junit4]   2> 3239420 INFO  
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [   
 ] o.a.s.c.a.s.SimSolrCloudTestCase ## Collections:      []
   [junit4]   2> 3239420 INFO  
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [   
 ] o.a.s.c.a.s.SimSolrCloudTestCase ## Max replicas per node:    0
   [junit4]   2> 3239420 INFO  
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [   
 ] o.a.s.c.a.s.SimSolrCloudTestCase ## Min replicas per node:    0
   [junit4]   2> 3239420 INFO  
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [   
 ] o.a.s.c.a.s.SimSolrCloudTestCase ## Total replicas:           0
   [junit4]   2> 3239420 INFO  
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [   
 ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Final Solr op counts ##########
   [junit4]   2> 3239420 DEBUG (AutoscalingActionExecutor-11988-thread-1) [    
] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger2 after 100ms
   [junit4]   2> 3239420 INFO  
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [   
 ] o.a.s.c.a.s.SimSolrCloudTestCase ##           - autoscaling        5
   [junit4]   2> 3239420 DEBUG (AutoscalingActionExecutor-11988-thread-1) [    
] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
   [junit4]   2> 3239420 INFO  
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [   
 ] o.a.s.c.a.s.SimSolrCloudTestCase ##           - update             9
   [junit4]   2> 3239420 DEBUG (AutoscalingActionExecutor-11988-thread-1) [    
] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger2 after 100ms
   [junit4]   2> 3239420 INFO  
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [   
 ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Autoscaling event counts 
###########
   [junit4]   2> 3239420 DEBUG (AutoscalingActionExecutor-11988-thread-1) [    
] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger1 after 100ms
   [junit4]   2> 3239420 DEBUG (AutoscalingActionExecutor-11988-thread-1) [    
] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger1 after 100ms
   [junit4]   2> 3239420 INFO  
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [   
 ] o.a.s.c.a.s.SimSolrCloudTestCase ## * Trigger: node_added_trigger1
   [junit4]   2> 3239420 DEBUG (AutoscalingActionExecutor-11988-thread-1) [    
] o.a.s.c.a.ScheduledTriggers -- processing took 64 ms for event 
id=8cf8747a21aafT71cpd1fni40admcqk3y2085e0
   [junit4]   2> 3239420 INFO  
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [   
 ] o.a.s.c.a.s.SimSolrCloudTestCase ##           - STARTED         1
   [junit4]   2> 3239420 INFO  
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [   
 ] o.a.s.c.a.s.SimSolrCloudTestCase ##           - SUCCEEDED       1
   [junit4]   2> 3239420 INFO  
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [   
 ] o.a.s.c.a.s.SimSolrCloudTestCase ## * Trigger: node_added_trigger2
   [junit4]   2> 3239420 INFO  
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [   
 ] o.a.s.c.a.s.SimSolrCloudTestCase ##           - IGNORED         1
   [junit4]   2> 3239420 INFO  
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [   
 ] o.a.s.c.a.s.SimSolrCloudTestCase ##           - STARTED         1
   [junit4]   2> 3239420 INFO  
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [   
 ] o.a.s.c.a.s.SimSolrCloudTestCase ##           - SUCCEEDED       1
   [junit4]   2> 3239420 INFO  
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [   
 ] o.a.s.c.a.s.SimSolrCloudTestCase ## * Trigger: node_lost_trigger1
   [junit4]   2> 3239420 INFO  
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [   
 ] o.a.s.c.a.s.SimSolrCloudTestCase ##           - STARTED         1
   [junit4]   2> 3239420 INFO  
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [   
 ] o.a.s.c.a.s.SimSolrCloudTestCase ##           - SUCCEEDED       1
   [junit4]   2> 3239420 INFO  
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [   
 ] o.a.s.c.a.s.SimSolrCloudTestCase ## * Trigger: node_lost_trigger2
   [junit4]   2> 3239420 INFO  
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [   
 ] o.a.s.c.a.s.SimSolrCloudTestCase ##           - STARTED         1
   [junit4]   2> 3239420 INFO  
(TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [   
 ] o.a.s.c.a.s.SimSolrCloudTestCase ##           - SUCCEEDED       1
   [junit4]   2> 3239429 DEBUG 
(TEST-TestTriggerIntegration.testEventQueue-seed#[F64294E2F6294EBA]) [    ] 
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 
13
   [junit4]   2> 3239429 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 
13
   [junit4]   2> 3239429 DEBUG 
(TEST-TestTriggerIntegration.testEventQueue-seed#[F64294E2F6294EBA]) [    ] 
o.a.s.c.a.s.SimNodeStateProvider - removing dead node values: 
127.0.0.1:10023_solr
   [junit4]   2> 3239429 INFO  
(TEST-TestTriggerIntegration.testEventQueue-seed#[F64294E2F6294EBA]) [    ] 
o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing 
object cache...
   [junit4]   2> 3239430 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 3239430 DEBUG 
(TEST-TestTriggerIntegration.testEventQueue-seed#[F64294E2F6294EBA]) [    ] 
o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 3239430 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 3239430 DEBUG 
(TEST-TestTriggerIntegration.testEventQueue-seed#[F64294E2F6294EBA]) [    ] 
o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 3239430 DEBUG 
(TEST-TestTriggerIntegration.testEventQueue-seed#[F64294E2F6294EBA]) [    ] 
o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 3239430 WARN  (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Interrupted
   [junit4]   2> java.lang.InterruptedException: null
   [junit4]   2>        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1220)
 ~[?:1.8.0_162]
   [junit4]   2>        at 
java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:335)
 ~[?:1.8.0_162]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:169)
 [java/:?]
   [junit4]   2>        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_162]
   [junit4]   2> 3239431 DEBUG 
(TEST-TestTriggerIntegration.testEventQueue-seed#[F64294E2F6294EBA]) [    ] 
o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool 
executor
   [junit4]   2> 3239431 DEBUG 
(TEST-TestTriggerIntegration.testEventQueue-seed#[F64294E2F6294EBA]) [    ] 
o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 3239431 DEBUG 
(TEST-TestTriggerIntegration.testEventQueue-seed#[F64294E2F6294EBA]) [    ] 
o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 3239438 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and 
.scheduled_maintenance triggers
   [junit4]   2> 3239438 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode 
version 14
   [junit4]   2> 3239439 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 14, lastZnodeVersion -1
   [junit4]   2> 3239439 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 
14
   [junit4]   2> 3239439 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial 
livenodes: [127.0.0.1:10021_solr, 127.0.0.1:10020_solr]
   [junit4]   2> 3239439 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 3239439 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 14, lastZnodeVersion 14
   [junit4]   2> 3239440 DEBUG (ScheduledTrigger-11990-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2
   [junit4]   2> 3239460 DEBUG (ScheduledTrigger-11990-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2
   [junit4]   2> 3239481 DEBUG (ScheduledTrigger-11990-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2
   [junit4]   2> 3239502 DEBUG (ScheduledTrigger-11990-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2
   [junit4]   2> 3239522 DEBUG (ScheduledTrigger-11990-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 2
   [junit4]   2> 3239537 DEBUG (simCloudManagerPool-11989-thread-1) [    ] 
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 3239538 DEBUG (simCloudManagerPool-11989-thread-1) [    ] 
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 
15
   [junit4]   2> 3239538 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 
15
   [junit4]   2> 3239540 INFO  (simCloudManagerPool-11989-thread-2) [    ] 
o.a.s.c.a.s.TestTriggerIntegration TestEventQueueAction instantiated
   [junit4]   2> 3239540 DEBUG (simCloudManagerPool-11989-thread-2) [    ] 
o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
   [junit4]   2> 3239540 DEBUG (simCloudManagerPool-11989-thread-2) [    ] 
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 3239541 DEBUG (simCloudManagerPool-11989-thread-2) [    ] 
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 
16
   [junit4]   2> 3239541 INFO  (simCloudManagerPool-11989-thread-2) [    ] 
o.a.s.c.a.s.TestTriggerIntegration TestEventQueueAction instantiated
   [junit4]   2> 3239541 DEBUG (simCloudManagerPool-11989-thread-2) [    ] 
o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
   [junit4]   2> 3239543 DEBUG (ScheduledTrigger-11990-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3239543 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 3239543 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread   -- deleted 
/autoscaling/nodeAdded/127.0.0.1:10024_solr
   [junit4]   2> 3239543 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 16, lastZnodeVersion 15
   [junit4]   2> 3239543 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 
16
   [junit4]   2> 3239543 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 - Initial 
livenodes: [127.0.0.1:10021_solr, 127.0.0.1:10020_solr, 127.0.0.1:10024_solr]
   [junit4]   2> 3239543 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 instantiated 
with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, 
class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestEventQueueAction}],
 enabled=true}
   [junit4]   2> 3239543 DEBUG (Simulated OverseerAutoScalingTriggerThread) [   
 ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 16, lastZnodeVersion 16
   [junit4]   2> 3239543 DEBUG (ScheduledTrigger-11990-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 3239543 DEBUG (ScheduledTrigger-11990-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3239563 DEBUG (ScheduledTrigger-11990-thread-4) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3239565 DEBUG (ScheduledTrigger-11990-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 3239565 DEBUG (ScheduledTrigger-11990-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3239586 DEBUG (ScheduledTrigger-11990-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3239586 DEBUG (ScheduledTrigger-11990-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 3239586 DEBUG (ScheduledTrigger-11990-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3239606 DEBUG (ScheduledTrigger-11990-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3239606 DEBUG (ScheduledTrigger-11990-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 3239606 DEBUG (ScheduledTrigger-11990-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3239627 DEBUG (ScheduledTrigger-11990-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3239627 DEBUG (ScheduledTrigger-11990-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 3239627 DEBUG (ScheduledTrigger-11990-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3239647 DEBUG (ScheduledTrigger-11990-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3239647 DEBUG (ScheduledTrigger-11990-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 3239647 DEBUG (ScheduledTrigger-11990-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3239667 DEBUG (ScheduledTrigger-11990-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3239667 DEBUG (ScheduledTrigger-11990-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 3239667 DEBUG (ScheduledTrigger-11990-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3239687 DEBUG (ScheduledTrigger-11990-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3239687 DEBUG (ScheduledTrigger-11990-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 3239687 DEBUG (ScheduledTrigger-11990-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3239707 DEBUG (ScheduledTrigger-11990-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3239708 DEBUG (ScheduledTrigger-11990-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 3239708 DEBUG (ScheduledTrigger-11990-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3239728 DEBUG (ScheduledTrigger-11990-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3239728 DEBUG (ScheduledTrigger-11990-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 3239728 DEBUG (ScheduledTrigger-11990-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3239748 DEBUG (ScheduledTrigger-11990-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3239748 DEBUG (ScheduledTrigger-11990-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 3239748 DEBUG (ScheduledTrigger-11990-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3239768 DEBUG (ScheduledTrigger-11990-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3239768 DEBUG (ScheduledTrigger-11990-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 3239768 DEBUG (ScheduledTrigger-11990-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3239789 DEBUG (ScheduledTrigger-11990-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3239789 DEBUG (ScheduledTrigger-11990-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 3239789 DEBUG (ScheduledTrigger-11990-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3239809 DEBUG (ScheduledTrigger-11990-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3239809 DEBUG (ScheduledTrigger-11990-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 3239809 DEBUG (ScheduledTrigger-11990-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3239829 DEBUG (ScheduledTrigger-11990-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3239829 DEBUG (ScheduledTrigger-11990-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 3239829 DEBUG (ScheduledTrigger-11990-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3239850 DEBUG (ScheduledTrigger-11990-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3239850 DEBUG (ScheduledTrigger-11990-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 3239850 DEBUG (ScheduledTrigger-11990-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3239870 DEBUG (ScheduledTrigger-11990-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3239870 DEBUG (ScheduledTrigger-11990-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 3239870 DEBUG (ScheduledTrigger-11990-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3239890 DEBUG (ScheduledTrigger-11990-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3239890 DEBUG (ScheduledTrigger-11990-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 3239890 DEBUG (ScheduledTrigger-11990-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3239910 DEBUG (ScheduledTrigger-11990-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3239910 DEBUG (ScheduledTrigger-11990-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 3239910 DEBUG (ScheduledTrigger-11990-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3239930 DEBUG (ScheduledTrigger-11990-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3239930 DEBUG (ScheduledTrigger-11990-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 3239930 DEBUG (ScheduledTrigger-11990-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3239951 DEBUG (ScheduledTrigger-11990-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3239951 DEBUG (ScheduledTrigger-11990-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 3239951 DEBUG (ScheduledTrigger-11990-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3239971 DEBUG (ScheduledTrigger-11990-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3239971 DEBUG (ScheduledTrigger-11990-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 3239971 DEBUG (ScheduledTrigger-11990-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3239991 DEBUG (ScheduledTrigger-11990-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3239991 DEBUG (ScheduledTrigger-11990-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 3239991 DEBUG (ScheduledTrigger-11990-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3240011 DEBUG (ScheduledTrigger-11990-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3240011 DEBUG (ScheduledTrigger-11990-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 3240011 DEBUG (ScheduledTrigger-11990-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3240031 DEBUG (ScheduledTrigger-11990-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3240031 DEBUG (ScheduledTrigger-11990-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 3240031 DEBUG (ScheduledTrigger-11990-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3240052 DEBUG (ScheduledTrigger-11990-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3240052 DEBUG (ScheduledTrigger-11990-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 3240052 DEBUG (ScheduledTrigger-11990-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3240072 DEBUG (ScheduledTrigger-11990-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3240072 DEBUG (ScheduledTrigger-11990-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 3240072 DEBUG (ScheduledTrigger-11990-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3240092 DEBUG (ScheduledTrigger-11990-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3240092 DEBUG (ScheduledTrigger-11990-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 3240092 DEBUG (ScheduledTrigger-11990-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3240112 DEBUG (ScheduledTrigger-11990-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3240112 DEBUG (ScheduledTrigger-11990-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 3240112 DEBUG (ScheduledTrigger-11990-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3240133 DEBUG (ScheduledTrigger-11990-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3240133 DEBUG (ScheduledTrigger-11990-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 3240133 DEBUG (ScheduledTrigger-11990-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3240160 DEBUG (ScheduledTrigger-11990-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3240160 DEBUG (ScheduledTrigger-11990-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 3240160 DEBUG (ScheduledTrigger-11990-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3240180 DEBUG (ScheduledTrigger-11990-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3240180 DEBUG (ScheduledTrigger-11990-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 3240180 DEBUG (ScheduledTrigger-11990-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3240200 DEBUG (ScheduledTrigger-11990-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3240201 DEBUG (ScheduledTrigger-11990-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 3240201 DEBUG (ScheduledTrigger-11990-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3240221 DEBUG (ScheduledTrigger-11990-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3240221 DEBUG (ScheduledTrigger-11990-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 3240221 DEBUG (ScheduledTrigger-11990-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3240241 DEBUG (ScheduledTrigger-11990-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3240241 DEBUG (ScheduledTrigger-11990-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 3240241 DEBUG (ScheduledTrigger-11990-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3240261 DEBUG (ScheduledTrigger-11990-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3240261 DEBUG (ScheduledTrigger-11990-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 3240261 DEBUG (ScheduledTrigger-11990-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3240282 DEBUG (ScheduledTrigger-11990-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3240282 DEBUG (ScheduledTrigger-11990-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 3240282 DEBUG (ScheduledTrigger-11990-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3240306 DEBUG (ScheduledTrigger-11990-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3240306 DEBUG (ScheduledTrigger-11990-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 3240306 DEBUG (ScheduledTrigger-11990-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3240326 DEBUG (ScheduledTrigger-11990-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3240326 DEBUG (ScheduledTrigger-11990-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 3240326 DEBUG (ScheduledTrigger-11990-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3240350 DEBUG (ScheduledTrigger-11990-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 3240350 DEBUG (ScheduledTrigger-11990-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3240350 DEBUG (ScheduledTrigger-11990-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3240370 DEBUG (ScheduledTrigger-11990-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 3240370 DEBUG (ScheduledTrigger-11990-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3240370 DEBUG (ScheduledTrigger-11990-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3240390 DEBUG (ScheduledTrigger-11990-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3240390 DEBUG (ScheduledTrigger-11990-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 3240390 DEBUG (ScheduledTrigger-11990-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3240410 DEBUG (ScheduledTrigger-11990-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3240410 DEBUG (ScheduledTrigger-11990-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 3240410 DEBUG (ScheduledTrigger-11990-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3240431 DEBUG (ScheduledTrigger-11990-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3240431 DEBUG (ScheduledTrigger-11990-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 3240431 DEBUG (ScheduledTrigger-11990-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3240452 DEBUG (ScheduledTrigger-11990-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3240452 DEBUG (ScheduledTrigger-11990-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 3240452 DEBUG (ScheduledTrigger-11990-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3240472 DEBUG (ScheduledTrigger-11990-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3240472 DEBUG (ScheduledTrigger-11990-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 3240473 DEBUG (ScheduledTrigger-11990-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3240493 DEBUG (ScheduledTrigger-11990-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3240493 DEBUG (ScheduledTrigger-11990-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 3240493 DEBUG (ScheduledTrigger-11990-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3240513 DEBUG (ScheduledTrigger-11990-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3240513 DEBUG (ScheduledTrigger-11990-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 3240513 DEBUG (ScheduledTrigger-11990-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3240533 DEBUG (ScheduledTrigger-11990-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3240533 DEBUG (ScheduledTrigger-11990-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 3240533 DEBUG (ScheduledTrigger-11990-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3240553 DEBUG (ScheduledTrigger-11990-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 3
   [junit4]   2> 3240553 DEBUG (ScheduledTrigger-

[...truncated too long message...]

nit4]   2> 15677 DEBUG (ScheduledTrigger-146-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"8959e056ec2b2T71cpd1fni40admcqk3y207rzt",
   [junit4]   2>   "source":"node_added_restore_trigger",
   [junit4]   2>   "eventTime":2416305742201522,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[2416305742201522],
   [junit4]   2>     "nodeNames":["127.0.0.1:10047_solr"]}}
   [junit4]   2> 15677 DEBUG (ScheduledTrigger-146-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, 
node_added_restore_trigger]
   [junit4]   2> 15677 DEBUG (ScheduledTrigger-146-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_restore_trigger - 
removing marker path: /autoscaling/nodeAdded/127.0.0.1:10047_solr
   [junit4]   2> 15678 DEBUG (AutoscalingActionExecutor-147-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"8959e056ec2b2T71cpd1fni40admcqk3y207rzt",
   [junit4]   2>   "source":"node_added_restore_trigger",
   [junit4]   2>   "eventTime":2416305742201522,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[2416305742201522],
   [junit4]   2>     "_enqueue_time_":2416307347319672,
   [junit4]   2>     "nodeNames":["127.0.0.1:10047_solr"]}}
   [junit4]   2> 15678 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[F64294E2F6294EBA])
 [    ] o.a.s.SolrTestCaseJ4 ###Ending testNodeAddedTriggerRestoreState
   [junit4]   2> 15678 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[F64294E2F6294EBA])
 [    ] o.a.s.c.a.s.SimSolrCloudTestCase 
   [junit4]   2> 
   [junit4]   2> 15678 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[F64294E2F6294EBA])
 [    ] o.a.s.c.a.s.SimSolrCloudTestCase 
#############################################
   [junit4]   2> 15678 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[F64294E2F6294EBA])
 [    ] o.a.s.c.a.s.SimSolrCloudTestCase ############ FINAL CLUSTER STATS 
############
   [junit4]   2> 15678 DEBUG (AutoscalingActionExecutor-147-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
   [junit4]   2> 15678 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[F64294E2F6294EBA])
 [    ] o.a.s.c.a.s.SimSolrCloudTestCase 
#############################################
   [junit4]   2> 
   [junit4]   2> 15678 DEBUG (AutoscalingActionExecutor-147-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_restore_trigger after 
100ms
   [junit4]   2> 15678 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[F64294E2F6294EBA])
 [    ] o.a.s.c.a.s.SimSolrCloudTestCase ## Live nodes:              3
   [junit4]   2> 15678 DEBUG (AutoscalingActionExecutor-147-thread-1) [    ] 
o.a.s.c.a.ScheduledTriggers -- processing took 29 ms for event 
id=8959e056ec2b2T71cpd1fni40admcqk3y207rzt
   [junit4]   2> 15678 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[F64294E2F6294EBA])
 [    ] o.a.s.c.a.s.SimSolrCloudTestCase ## Empty nodes:     3
   [junit4]   2> 15678 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[F64294E2F6294EBA])
 [    ] o.a.s.c.a.s.SimSolrCloudTestCase ## Dead nodes:              0
   [junit4]   2> 15678 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[F64294E2F6294EBA])
 [    ] o.a.s.c.a.s.SimSolrCloudTestCase ## Collections:     []
   [junit4]   2> 15678 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[F64294E2F6294EBA])
 [    ] o.a.s.c.a.s.SimSolrCloudTestCase ## Max replicas per node:   0
   [junit4]   2> 15678 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[F64294E2F6294EBA])
 [    ] o.a.s.c.a.s.SimSolrCloudTestCase ## Min replicas per node:   0
   [junit4]   2> 15678 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[F64294E2F6294EBA])
 [    ] o.a.s.c.a.s.SimSolrCloudTestCase ## Total replicas:          0
   [junit4]   2> 15678 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[F64294E2F6294EBA])
 [    ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Final Solr op counts 
##########
   [junit4]   2> 15678 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[F64294E2F6294EBA])
 [    ] o.a.s.c.a.s.SimSolrCloudTestCase ##          - autoscaling        3
   [junit4]   2> 15678 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[F64294E2F6294EBA])
 [    ] o.a.s.c.a.s.SimSolrCloudTestCase ##          - update             2
   [junit4]   2> 15679 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[F64294E2F6294EBA])
 [    ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Autoscaling event counts 
###########
   [junit4]   2> 15679 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[F64294E2F6294EBA])
 [    ] o.a.s.c.a.s.SimSolrCloudTestCase ## * Trigger: 
node_added_restore_trigger
   [junit4]   2> 15679 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[F64294E2F6294EBA])
 [    ] o.a.s.c.a.s.SimSolrCloudTestCase ##          - STARTED         1
   [junit4]   2> 15679 INFO  
(TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[F64294E2F6294EBA])
 [    ] o.a.s.c.a.s.SimSolrCloudTestCase ##          - SUCCEEDED       1
   [junit4] OK      0.15s J1 | 
TestTriggerIntegration.testNodeAddedTriggerRestoreState
   [junit4]   2> 15680 WARN  (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are 
closed, exiting.
   [junit4]   2> 15680 DEBUG 
(SUITE-TestTriggerIntegration-seed#[F64294E2F6294EBA]-worker) [    ] 
o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 15680 DEBUG 
(SUITE-TestTriggerIntegration-seed#[F64294E2F6294EBA]-worker) [    ] 
o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 15680 DEBUG 
(SUITE-TestTriggerIntegration-seed#[F64294E2F6294EBA]-worker) [    ] 
o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 15680 DEBUG 
(SUITE-TestTriggerIntegration-seed#[F64294E2F6294EBA]-worker) [    ] 
o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool 
executor
   [junit4]   2> 15680 DEBUG 
(SUITE-TestTriggerIntegration-seed#[F64294E2F6294EBA]-worker) [    ] 
o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 15680 DEBUG 
(SUITE-TestTriggerIntegration-seed#[F64294E2F6294EBA]-worker) [    ] 
o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> NOTE: leaving temporary files on disk at: 
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.sim.TestTriggerIntegration_F64294E2F6294EBA-001
   [junit4]   2> NOTE: test params are: codec=Lucene70, 
sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@3004de5d),
 locale=es-CO, timezone=America/Halifax
   [junit4]   2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_162 
(64-bit)/cpus=3,threads=1,free=42242040,total=100663296
   [junit4]   2> NOTE: All tests run in this JVM: [TestTriggerIntegration, 
TestTriggerIntegration, TestTriggerIntegration]
   [junit4] Completed [5/5 (2!)] on J1 in 3.58s, 11 tests, 1 failure <<< 
FAILURES!
   [junit4] 
   [junit4] 
   [junit4] Tests with failures [seed: F64294E2F6294EBA]:
   [junit4]   - 
org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testEventQueue
   [junit4]   - 
org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testEventQueue
   [junit4] 
   [junit4] 
   [junit4] JVM J0:     0.87 ..    13.16 =    12.29s
   [junit4] JVM J1:     0.86 ..    16.99 =    16.13s
   [junit4] Execution time total: 17 seconds
   [junit4] Tests summary: 5 suites, 55 tests, 2 failures

BUILD FAILED
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/lucene/common-build.xml:1568:
 The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/lucene/common-build.xml:1092:
 There were test failures: 5 suites, 55 tests, 2 failures [seed: 
F64294E2F6294EBA]

Total time: 18 seconds

[repro] Setting last failure code to 256

[repro] Failures:
[repro]   2/5 failed: 
org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration
[repro] Exiting with code 256
+ mv lucene/build lucene/build.repro
+ mv solr/build solr/build.repro
+ mv lucene/build.orig lucene/build
+ mv solr/build.orig solr/build
Archiving artifacts
Setting 
ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
[WARNINGS] Parsing warnings in console log with parser Java Compiler (javac)
Setting 
ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting 
ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
<Git Blamer> Using GitBlamer to create author and commit information for all 
warnings.
<Git Blamer> GIT_COMMIT=5b250b4a403f5b572f667d89acab6d07553b2e65, 
workspace=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris
[WARNINGS] Computing warning deltas based on reference build #1793
Recording test results
Setting 
ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
Setting 
ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting 
ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting 
ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting 
ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Reply via email to