Build: https://jenkins.thetaphi.de/job/Lucene-Solr-8.0-Linux/231/ Java: 32bit/jdk1.8.0_172 -server -XX:+UseSerialGC
1 tests failed. FAILED: org.apache.solr.cloud.autoscaling.sim.TestSimTriggerIntegration.testNodeAddedTriggerRestoreState Error Message: OverseerTriggerThread never caught up to the latest znodeVersion Stack Trace: java.util.concurrent.TimeoutException: OverseerTriggerThread never caught up to the latest znodeVersion at __randomizedtesting.SeedInfo.seed([A27319673644C5A9:2A4E90180C842404]:0) at org.apache.solr.util.TimeOut.waitFor(TimeOut.java:66) at org.apache.solr.cloud.autoscaling.sim.SimSolrCloudTestCase.assertAutoscalingUpdateComplete(SimSolrCloudTestCase.java:71) at org.apache.solr.cloud.autoscaling.sim.TestSimTriggerIntegration.testNodeAddedTriggerRestoreState(TestSimTriggerIntegration.java:456) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938) at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974) at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368) at java.lang.Thread.run(Thread.java:748) Build Log: [...truncated 14573 lines...] [junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestSimTriggerIntegration [junit4] 2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-8.0-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.sim.TestSimTriggerIntegration_A27319673644C5A9-001/init-core-data-001 [junit4] 2> 1685525 INFO (SUITE-TestSimTriggerIntegration-seed#[A27319673644C5A9]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=false [junit4] 2> 1685526 INFO (SUITE-TestSimTriggerIntegration-seed#[A27319673644C5A9]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN) [junit4] 2> 1685526 INFO (SUITE-TestSimTriggerIntegration-seed#[A27319673644C5A9]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom [junit4] 2> 1685529 INFO (TEST-TestSimTriggerIntegration.testNodeLostTriggerRestoreState-seed#[A27319673644C5A9]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testNodeLostTriggerRestoreState [junit4] 2> 1685531 DEBUG (TEST-TestSimTriggerIntegration.testNodeLostTriggerRestoreState-seed#[A27319673644C5A9]) [ ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10248_solr [junit4] 2> 1685531 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers [junit4] 2> 1685531 DEBUG (TEST-TestSimTriggerIntegration.testNodeLostTriggerRestoreState-seed#[A27319673644C5A9]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=0 [junit4] 2> 1685531 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1 [junit4] 2> 1685531 DEBUG (TEST-TestSimTriggerIntegration.testNodeLostTriggerRestoreState-seed#[A27319673644C5A9]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 0 [junit4] 2> 1685531 INFO (TEST-TestSimTriggerIntegration.testNodeLostTriggerRestoreState-seed#[A27319673644C5A9]) [ ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory. [junit4] 2> 1685532 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 0 [junit4] 2> 1685532 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1 [junit4] 2> 1685532 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10249_solr, 127.0.0.1:10248_solr] [junit4] 2> 1685532 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers [junit4] 2> 1685532 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1 [junit4] 2> 1685532 DEBUG (ScheduledTrigger-7756-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 1685532 DEBUG (simCloudManagerPool-7755-thread-2) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration [junit4] 2> 1685532 DEBUG (simCloudManagerPool-7755-thread-2) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2 [junit4] 2> 1685533 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2 [junit4] 2> 1685533 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers [junit4] 2> 1685533 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2 [junit4] 2> 1685534 DEBUG (simCloudManagerPool-7755-thread-3) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration [junit4] 2> 1685534 DEBUG (simCloudManagerPool-7755-thread-3) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3 [junit4] 2> 1685535 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3 [junit4] 2> 1685535 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestSimTriggerIntegration TestTriggerAction init [junit4] 2> 1685535 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger node_lost_restore_trigger - Initial livenodes: [127.0.0.1:10249_solr, 127.0.0.1:10248_solr] [junit4] 2> 1685535 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers [junit4] 2> 1685535 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3 [junit4] 2> 1685535 DEBUG (ScheduledTrigger-7756-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 1685553 DEBUG (ScheduledTrigger-7756-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 2 [junit4] 2> 1685556 DEBUG (ScheduledTrigger-7756-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger with currently live nodes: 3 and last live nodes: 2 [junit4] 2> 1685556 DEBUG (TEST-TestSimTriggerIntegration.testNodeLostTriggerRestoreState-seed#[A27319673644C5A9]) [ ] o.a.s.c.a.s.SimClusterStateProvider -- created marker: /autoscaling/nodeLost/127.0.0.1:10250_solr [junit4] 2> 1685573 DEBUG (ScheduledTrigger-7756-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 3 [junit4] 2> 1685573 DEBUG (ScheduledTrigger-7756-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10250_solr [junit4] 2> 1685576 DEBUG (ScheduledTrigger-7756-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger with currently live nodes: 2 and last live nodes: 3 [junit4] 2> 1685576 DEBUG (ScheduledTrigger-7756-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10250_solr [junit4] 2> 1685578 DEBUG (simCloudManagerPool-7755-thread-4) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration [junit4] 2> 1685579 DEBUG (simCloudManagerPool-7755-thread-4) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4 [junit4] 2> 1685579 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4 [junit4] 2> 1685579 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestSimTriggerIntegration TestTriggerAction init [junit4] 2> 1685579 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger node_lost_restore_trigger - Initial livenodes: [127.0.0.1:10249_solr, 127.0.0.1:10248_solr] [junit4] 2> 1685579 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers [junit4] 2> 1685579 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4 [junit4] 2> 1685579 DEBUG (ScheduledTrigger-7756-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 1685593 DEBUG (ScheduledTrigger-7756-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 1685600 DEBUG (ScheduledTrigger-7756-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 1685600 DEBUG (ScheduledTrigger-7756-thread-4) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:10250_solr] [junit4] 2> 1685600 DEBUG (ScheduledTrigger-7756-thread-4) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: { [junit4] 2> "id":"3ef1793dbd11dT9d2mneax2y2o7qtzdnfujzeb2", [junit4] 2> "source":"node_lost_restore_trigger", [junit4] 2> "eventTime":1107309474074909, [junit4] 2> "eventType":"NODELOST", [junit4] 2> "properties":{ [junit4] 2> "eventTimes":[1107309474074909], [junit4] 2> "preferredOperation":"movereplica", [junit4] 2> "nodeNames":["127.0.0.1:10250_solr"]}} [junit4] 2> 1685600 DEBUG (ScheduledTrigger-7756-thread-4) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_lost_restore_trigger] [junit4] 2> 1685600 DEBUG (ScheduledTrigger-7756-thread-4) [ ] o.a.s.c.a.SystemLogListener Collection .system missing, skip sending event { [junit4] 2> "id":"3ef1793dbd11dT9d2mneax2y2o7qtzdnfujzeb2", [junit4] 2> "source":"node_lost_restore_trigger", [junit4] 2> "eventTime":1107309474074909, [junit4] 2> "eventType":"NODELOST", [junit4] 2> "properties":{ [junit4] 2> "eventTimes":[1107309474074909], [junit4] 2> "preferredOperation":"movereplica", [junit4] 2> "_enqueue_time_":1107310676705559, [junit4] 2> "nodeNames":["127.0.0.1:10250_solr"]}} [junit4] 2> 1685600 DEBUG (AutoscalingActionExecutor-7757-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for { [junit4] 2> "id":"3ef1793dbd11dT9d2mneax2y2o7qtzdnfujzeb2", [junit4] 2> "source":"node_lost_restore_trigger", [junit4] 2> "eventTime":1107309474074909, [junit4] 2> "eventType":"NODELOST", [junit4] 2> "properties":{ [junit4] 2> "eventTimes":[1107309474074909], [junit4] 2> "preferredOperation":"movereplica", [junit4] 2> "_enqueue_time_":1107310676705559, [junit4] 2> "nodeNames":["127.0.0.1:10250_solr"]}} [junit4] 2> 1685601 DEBUG (AutoscalingActionExecutor-7757-thread-1) [ ] o.a.s.c.a.SystemLogListener Collection .system missing, skip sending event { [junit4] 2> "id":"3ef1793dbd11dT9d2mneax2y2o7qtzdnfujzeb2", [junit4] 2> "source":"node_lost_restore_trigger", [junit4] 2> "eventTime":1107309474074909, [junit4] 2> "eventType":"NODELOST", [junit4] 2> "properties":{ [junit4] 2> "eventTimes":[1107309474074909], [junit4] 2> "preferredOperation":"movereplica", [junit4] 2> "_enqueue_time_":1107310676705559, [junit4] 2> "nodeNames":["127.0.0.1:10250_solr"]}} [junit4] 2> 1685601 DEBUG (AutoscalingActionExecutor-7757-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms [junit4] 2> 1685601 DEBUG (AutoscalingActionExecutor-7757-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_restore_trigger after 100ms [junit4] 2> 1685601 DEBUG (AutoscalingActionExecutor-7757-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 16 ms for event id=3ef1793dbd11dT9d2mneax2y2o7qtzdnfujzeb2 [junit4] 2> 1685601 DEBUG (TEST-TestSimTriggerIntegration.testNodeLostTriggerRestoreState-seed#[A27319673644C5A9]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now [junit4] 2> 1685601 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting. [junit4] 2> 1685601 DEBUG (TEST-TestSimTriggerIntegration.testNodeLostTriggerRestoreState-seed#[A27319673644C5A9]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now [junit4] 2> 1685602 DEBUG (TEST-TestSimTriggerIntegration.testNodeLostTriggerRestoreState-seed#[A27319673644C5A9]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor [junit4] 2> 1685602 DEBUG (TEST-TestSimTriggerIntegration.testNodeLostTriggerRestoreState-seed#[A27319673644C5A9]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor [junit4] 2> 1685602 DEBUG (TEST-TestSimTriggerIntegration.testNodeLostTriggerRestoreState-seed#[A27319673644C5A9]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely [junit4] 2> 1685602 DEBUG (TEST-TestSimTriggerIntegration.testNodeLostTriggerRestoreState-seed#[A27319673644C5A9]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly [junit4] 2> 1685603 INFO (TEST-TestSimTriggerIntegration.testNodeLostTriggerRestoreState-seed#[A27319673644C5A9]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testNodeLostTriggerRestoreState [junit4] 2> 1685606 INFO (TEST-TestSimTriggerIntegration.testNodeAddedTrigger-seed#[A27319673644C5A9]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testNodeAddedTrigger [junit4] 2> 1685608 DEBUG (TEST-TestSimTriggerIntegration.testNodeAddedTrigger-seed#[A27319673644C5A9]) [ ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10251_solr [junit4] 2> 1685609 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers [junit4] 2> 1685609 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1 [junit4] 2> 1685609 DEBUG (TEST-TestSimTriggerIntegration.testNodeAddedTrigger-seed#[A27319673644C5A9]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=0 [junit4] 2> 1685609 DEBUG (TEST-TestSimTriggerIntegration.testNodeAddedTrigger-seed#[A27319673644C5A9]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 0 [junit4] 2> 1685609 INFO (TEST-TestSimTriggerIntegration.testNodeAddedTrigger-seed#[A27319673644C5A9]) [ ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory. [junit4] 2> 1685609 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 0 [junit4] 2> 1685609 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1 [junit4] 2> 1685609 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10251_solr, 127.0.0.1:10252_solr] [junit4] 2> 1685609 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers [junit4] 2> 1685609 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1 [junit4] 2> 1685609 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 1685610 DEBUG (simCloudManagerPool-7761-thread-2) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration [junit4] 2> 1685610 DEBUG (simCloudManagerPool-7761-thread-2) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2 [junit4] 2> 1685610 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2 [junit4] 2> 1685611 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers [junit4] 2> 1685611 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2 [junit4] 2> 1685611 DEBUG (simCloudManagerPool-7761-thread-3) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration [junit4] 2> 1685611 DEBUG (simCloudManagerPool-7761-thread-3) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3 [junit4] 2> 1685611 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3 [junit4] 2> 1685611 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestSimTriggerIntegration TestTriggerAction init [junit4] 2> 1685611 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - Initial livenodes: [127.0.0.1:10251_solr, 127.0.0.1:10252_solr] [junit4] 2> 1685611 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestSimTriggerIntegration$TestTriggerAction}], enabled=true} [junit4] 2> 1685611 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers [junit4] 2> 1685611 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3 [junit4] 2> 1685611 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1685611 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2 [junit4] 2> 1685616 DEBUG (TEST-TestSimTriggerIntegration.testNodeAddedTrigger-seed#[A27319673644C5A9]) [ ] o.a.s.c.a.s.SimClusterStateProvider -- creating marker: /autoscaling/nodeAdded/127.0.0.1:10253_solr [junit4] 2> 1685630 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 2 [junit4] 2> 1685632 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1685632 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1685632 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10253_solr at time 1107312270682859 [junit4] 2> 1685651 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1685652 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1685652 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1685652 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger firing registered processor for nodes: [127.0.0.1:10253_solr] added at times [1107312270682859], now=1107313278324009 [junit4] 2> 1685652 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: { [junit4] 2> "id":"3ef183a8caaebT9d2mneax2y2o7qtzdnfujzeb4", [junit4] 2> "source":"node_added_trigger", [junit4] 2> "eventTime":1107312270682859, [junit4] 2> "eventType":"NODEADDED", [junit4] 2> "properties":{ [junit4] 2> "eventTimes":[1107312270682859], [junit4] 2> "preferredOperation":"movereplica", [junit4] 2> "nodeNames":["127.0.0.1:10253_solr"]}} [junit4] 2> 1685653 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_trigger] [junit4] 2> 1685653 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.SystemLogListener Collection .system missing, skip sending event { [junit4] 2> "id":"3ef183a8caaebT9d2mneax2y2o7qtzdnfujzeb4", [junit4] 2> "source":"node_added_trigger", [junit4] 2> "eventTime":1107312270682859, [junit4] 2> "eventType":"NODEADDED", [junit4] 2> "properties":{ [junit4] 2> "eventTimes":[1107312270682859], [junit4] 2> "preferredOperation":"movereplica", [junit4] 2> "_enqueue_time_":1107313304101809, [junit4] 2> "nodeNames":["127.0.0.1:10253_solr"]}} [junit4] 2> 1685653 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Removing new node from tracking: 127.0.0.1:10253_solr [junit4] 2> 1685653 DEBUG (AutoscalingActionExecutor-7763-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for { [junit4] 2> "id":"3ef183a8caaebT9d2mneax2y2o7qtzdnfujzeb4", [junit4] 2> "source":"node_added_trigger", [junit4] 2> "eventTime":1107312270682859, [junit4] 2> "eventType":"NODEADDED", [junit4] 2> "properties":{ [junit4] 2> "eventTimes":[1107312270682859], [junit4] 2> "preferredOperation":"movereplica", [junit4] 2> "_enqueue_time_":1107313304101809, [junit4] 2> "nodeNames":["127.0.0.1:10253_solr"]}} [junit4] 2> 1685653 DEBUG (AutoscalingActionExecutor-7763-thread-1) [ ] o.a.s.c.a.SystemLogListener Collection .system missing, skip sending event { [junit4] 2> "id":"3ef183a8caaebT9d2mneax2y2o7qtzdnfujzeb4", [junit4] 2> "source":"node_added_trigger", [junit4] 2> "eventTime":1107312270682859, [junit4] 2> "eventType":"NODEADDED", [junit4] 2> "properties":{ [junit4] 2> "eventTimes":[1107312270682859], [junit4] 2> "preferredOperation":"movereplica", [junit4] 2> "_enqueue_time_":1107313304101809, [junit4] 2> "nodeNames":["127.0.0.1:10253_solr"]}} [junit4] 2> 1685653 DEBUG (AutoscalingActionExecutor-7763-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms [junit4] 2> 1685653 DEBUG (AutoscalingActionExecutor-7763-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger after 100ms [junit4] 2> 1685653 DEBUG (AutoscalingActionExecutor-7763-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 8 ms for event id=3ef183a8caaebT9d2mneax2y2o7qtzdnfujzeb4 [junit4] 2> 1685654 DEBUG (simCloudManagerPool-7761-thread-4) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration [junit4] 2> 1685654 DEBUG (simCloudManagerPool-7761-thread-4) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4 [junit4] 2> 1685654 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4 [junit4] 2> 1685654 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers [junit4] 2> 1685654 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4 [junit4] 2> 1685753 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1685753 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1685753 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1685774 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1685774 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1685774 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1685794 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1685794 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1685794 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1685814 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1685814 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1685814 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1685834 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1685834 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1685834 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1685854 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1685854 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1685854 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1685874 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1685874 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1685874 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1685895 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1685895 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1685895 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1685915 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1685915 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1685915 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1685935 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1685935 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1685935 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1685955 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1685955 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1685955 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1685975 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1685975 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1685975 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1685995 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1685995 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1685995 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1686015 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1686015 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1686016 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1686036 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1686036 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1686036 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1686056 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1686056 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1686056 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1686076 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1686076 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1686076 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1686096 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1686096 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1686096 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1686116 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1686116 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1686116 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1686136 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1686136 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1686137 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1686157 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1686157 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1686157 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1686177 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1686177 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1686177 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1686197 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1686197 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1686197 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1686217 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1686217 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1686217 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1686237 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1686237 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1686237 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1686257 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1686257 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1686257 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1686278 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1686278 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1686278 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1686298 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1686298 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1686298 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1686318 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1686318 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1686318 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1686338 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1686338 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1686338 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1686358 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1686358 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1686358 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1686378 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1686378 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1686378 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1686398 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1686399 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1686399 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1686419 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1686419 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1686419 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1686439 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1686439 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1686439 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1686459 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1686459 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1686459 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1686479 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1686479 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1686479 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1686499 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1686499 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1686499 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1686519 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1686519 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1686519 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1686540 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1686540 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1686540 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1686560 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1686560 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1686560 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1686580 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1686580 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1686580 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1686600 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1686600 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1686600 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1686620 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1686620 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1686620 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1686640 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1686640 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1686640 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1686661 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1686661 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1686661 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1686681 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1686681 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1686681 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1686701 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1686701 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1686701 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1686721 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1686721 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1686721 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1686741 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1686741 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1686741 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1686761 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1686761 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1686761 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1686782 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1686782 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1686782 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1686802 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1686802 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1686802 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1686822 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1686822 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1686822 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1686842 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1686842 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1686842 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1686862 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1686862 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1686863 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1686883 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1686883 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1686883 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1686903 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1686903 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1686903 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1686923 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1686923 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1686923 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1686943 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1686943 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1686943 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1686963 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1686963 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1686963 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1686984 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1686984 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1686984 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1687004 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1687004 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1687004 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1687024 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1687024 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1687024 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1687044 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1687044 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1687044 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1687064 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1687064 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1687064 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1687084 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1687084 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1687084 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1687105 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1687105 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1687105 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1687125 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1687125 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1687125 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1687145 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1687145 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1687145 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1687165 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1687165 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1687165 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1687185 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1687185 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1687185 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1687205 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1687205 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1687205 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1687226 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1687226 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1687226 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1687246 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1687246 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1687246 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1687266 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1687266 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1687266 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1687286 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1687286 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1687286 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1687306 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1687306 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1687306 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1687327 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1687327 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1687327 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1687347 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1687347 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1687347 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1687367 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1687367 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1687367 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1687387 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1687387 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1687387 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1687407 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1687407 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1687407 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1687427 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1687427 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1687427 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1687448 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1687448 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1687448 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1687468 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1687468 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1687468 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1687488 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1687488 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1687488 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1687508 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1687508 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1687508 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1687528 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1687528 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1687528 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1687548 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1687548 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1687548 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1687569 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1687569 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1687569 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1687589 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1687589 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1687589 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1687609 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1687609 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1687609 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1687629 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1687629 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1687629 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1687649 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1687649 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1687649 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1687669 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1687670 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1687670 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1687690 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1687690 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1687690 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1687710 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1687710 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1687710 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1687730 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1687730 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1687730 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1687750 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1687750 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1687750 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1687770 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1687770 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1687770 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1687791 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1687791 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1687791 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1687811 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1687811 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1687811 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1687831 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1687831 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1687831 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1687851 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1687851 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1687851 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1687871 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1687871 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1687871 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1687891 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1687891 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1687892 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1687912 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1687912 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1687912 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1687932 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1687932 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1687932 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1687952 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1687952 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1687952 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1687972 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1687972 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1687972 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1687992 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1687992 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1687992 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1688013 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1688013 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1688013 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1688033 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1688033 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1688033 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1688053 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1688053 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1688053 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1688073 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1688073 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1688073 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1688093 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1688093 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1688093 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1688113 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1688113 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1688113 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1688134 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1688134 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1688134 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1688154 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1688154 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1688154 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1688174 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1688174 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1688174 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1688194 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1688194 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1688194 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1688214 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1688214 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1688214 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1688234 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1688234 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1688234 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1688255 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1688255 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1688255 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1688275 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1688275 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1688275 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1688295 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1688295 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1688295 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1688315 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1688315 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1688315 DEBUG (ScheduledTrigger-7762-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1688335 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1688335 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1688335 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1688355 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1688355 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1688355 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1688376 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1688376 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1688376 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1688396 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1688396 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1688396 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1688416 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1688416 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1688416 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1688436 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1688436 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1688436 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1688456 DEBUG (ScheduledTrigger-7762-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1688456 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1688456 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1688476 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 1688477 DEBUG (ScheduledTrigger-7762-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 1688477 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 1688497 DEBUG (ScheduledTrigger-7762-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_ad [...truncated too long message...] 5268059], [junit4] 2> "preferredOperation":"movereplica", [junit4] 2> "_enqueue_time_":1107803684712659, [junit4] 2> "nodeNames":["127.0.0.1:10282_solr"]}} [junit4] 2> 1695461 DEBUG (ScheduledTrigger-7830-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Removing new node from tracking: 127.0.0.1:10282_solr [junit4] 2> 1695461 DEBUG (AutoscalingActionExecutor-7831-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for { [junit4] 2> "id":"3ef8842ca2a5bT9d2mneax2y2o7qtzdnfujzebu", [junit4] 2> "source":"node_added_trigger", [junit4] 2> "eventTime":1107793445268059, [junit4] 2> "eventType":"NODEADDED", [junit4] 2> "properties":{ [junit4] 2> "eventTimes":[1107793445268059], [junit4] 2> "preferredOperation":"movereplica", [junit4] 2> "_enqueue_time_":1107803684712659, [junit4] 2> "nodeNames":["127.0.0.1:10282_solr"]}} [junit4] 2> 1695461 DEBUG (AutoscalingActionExecutor-7831-thread-1) [ ] o.a.s.c.a.SystemLogListener Collection .system missing, skip sending event { [junit4] 2> "id":"3ef8842ca2a5bT9d2mneax2y2o7qtzdnfujzebu", [junit4] 2> "source":"node_added_trigger", [junit4] 2> "eventTime":1107793445268059, [junit4] 2> "eventType":"NODEADDED", [junit4] 2> "properties":{ [junit4] 2> "eventTimes":[1107793445268059], [junit4] 2> "preferredOperation":"movereplica", [junit4] 2> "_enqueue_time_":1107803684712659, [junit4] 2> "nodeNames":["127.0.0.1:10282_solr"]}} [junit4] 2> 1695461 DEBUG (AutoscalingActionExecutor-7831-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms [junit4] 2> 1695461 DEBUG (AutoscalingActionExecutor-7831-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger after 100ms [junit4] 2> 1695461 DEBUG (AutoscalingActionExecutor-7831-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 10 ms for event id=3ef8842ca2a5bT9d2mneax2y2o7qtzdnfujzebu [junit4] 2> 1695461 DEBUG (TEST-TestSimTriggerIntegration.testEventFromRestoredState-seed#[A27319673644C5A9]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now [junit4] 2> 1695461 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting. [junit4] 2> 1695462 DEBUG (TEST-TestSimTriggerIntegration.testEventFromRestoredState-seed#[A27319673644C5A9]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now [junit4] 2> 1695462 DEBUG (TEST-TestSimTriggerIntegration.testEventFromRestoredState-seed#[A27319673644C5A9]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor [junit4] 2> 1695462 DEBUG (TEST-TestSimTriggerIntegration.testEventFromRestoredState-seed#[A27319673644C5A9]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor [junit4] 2> 1695462 DEBUG (TEST-TestSimTriggerIntegration.testEventFromRestoredState-seed#[A27319673644C5A9]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely [junit4] 2> 1695462 DEBUG (TEST-TestSimTriggerIntegration.testEventFromRestoredState-seed#[A27319673644C5A9]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly [junit4] 2> 1695462 INFO (TEST-TestSimTriggerIntegration.testEventFromRestoredState-seed#[A27319673644C5A9]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testEventFromRestoredState [junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-8.0-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.sim.TestSimTriggerIntegration_A27319673644C5A9-001 [junit4] 2> NOTE: test params are: codec=Lucene80, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@1e32143), locale=es-CU, timezone=Europe/Riga [junit4] 2> NOTE: Linux 4.18.0-15-generic i386/Oracle Corporation 1.8.0_172 (32-bit)/cpus=8,threads=1,free=249449144,total=518979584 [junit4] 2> NOTE: All tests run in this JVM: [JsonLoaderTest, HdfsBasicDistributedZkTest, TestUtilizeNode, TestCoreBackup, SuggestComponentContextFilterQueryTest, ReturnFieldsTest, TestReplicaProperties, CacheHeaderTest, TestFieldCacheReopen, TestNumericTokenStream, IndexSchemaTest, TestConfigsApi, TestAnalyzedSuggestions, SpatialFilterTest, TestNonDefinedSimilarityFactory, TestLuceneMatchVersion, ShufflingReplicaListTransformerTest, MetricsHistoryHandlerTest, CollectionTooManyReplicasTest, AdminHandlersProxyTest, TaggerTest, TestIntervalFaceting, TestNestedUpdateProcessor, PKIAuthenticationIntegrationTest, TermVectorComponentTest, TestHashPartitioner, ConjunctionSolrSpellCheckerTest, RegexBoostProcessorTest, SegmentsInfoRequestHandlerTest, ClassificationUpdateProcessorFactoryTest, TestManagedSchemaThreadSafety, TestMinMaxOnMultiValuedField, TestCollectionsAPIViaSolrCloudCluster, DistributedFacetExistsSmallTest, CheckHdfsIndexTest, RecoveryAfterSoftCommitTest, ZkStateWriterTest, TestTrieFacet, SpellPossibilityIteratorTest, TestJsonFacetRefinement, LeaderElectionTest, BasicDistributedZk2Test, RulesTest, SolrIndexConfigTest, TestRandomFaceting, TestUnInvertedFieldException, TestTlogReplica, TestConfigSetsAPI, TestDistribIDF, TestUtils, TestBinaryResponseWriter, TestEmbeddedSolrServerConstructors, BigEndianAscendingWordSerializerTest, TestSQLHandler, SaslZkACLProviderTest, TestSubQueryTransformerCrossCore, TestSolrCloudSnapshots, CursorMarkTest, CdcrReplicationHandlerTest, ZkFailoverTest, TestCustomSort, TestXmlQParserPlugin, TestLegacyNumericRangeQueryBuilder, AddBlockUpdateTest, TestMacros, OutOfBoxZkACLAndCredentialsProvidersTest, DocValuesMissingTest, TestSearchPerf, ConnectionReuseTest, TestCloudRecovery2, FullHLLTest, TestGraphTermsQParserPlugin, TestFieldCollectionResource, TestHighlightDedupGrouping, RootFieldTest, NoCacheHeaderTest, TestPHPSerializedResponseWriter, ChaosMonkeyNothingIsSafeTest, TestInfoStreamLogging, DistanceFunctionTest, TestOverriddenPrefixQueryForCustomFieldType, TestTolerantSearch, CloudExitableDirectoryReaderTest, ConcurrentCreateRoutedAliasTest, TestExtendedDismaxParser, HdfsTlogReplayBufferedWhileIndexingTest, TestFreeTextSuggestions, SolrCoreCheckLockOnStartupTest, TestShortCircuitedRequests, TestSimLargeCluster, QueryEqualityTest, TestSchemaSimilarityResource, TestDocTermOrds, TestBadConfig, SolrRrdBackendFactoryTest, BlockJoinFacetRandomTest, BooleanFieldTest, UUIDUpdateProcessorFallbackTest, RequestLoggingTest, PreAnalyzedFieldManagedSchemaCloudTest, TestAtomicUpdateErrorCases, FastVectorHighlighterTest, CreateCollectionCleanupTest, PhrasesIdentificationComponentTest, AnalyticsMergeStrategyTest, SpatialHeatmapFacetsTest, UpdateLogTest, SolrGraphiteReporterTest, CSVRequestHandlerTest, ConcurrentDeleteAndCreateCollectionTest, CoreMergeIndexesAdminHandlerTest, ReplaceNodeNoTargetTest, SyncSliceTest, SparseHLLTest, TestManagedSynonymGraphFilterFactory, TokenizerChainTest, TestSchemaManager, PeerSyncWithIndexFingerprintCachingTest, ExitableDirectoryReaderTest, OrderedExecutorTest, ChaosMonkeySafeLeaderWithPullReplicasTest, DistributedFacetPivotSmallAdvancedTest, OverseerModifyCollectionTest, TestSolrCloudWithDelegationTokens, ResponseLogComponentTest, ProtectedTermFilterFactoryTest, TestFieldCacheVsDocValues, TestLockTree, TestComplexPhraseLeadingWildcard, HttpPartitionTest, ConfigureRecoveryStrategyTest, MaxSizeAutoCommitTest, TestSolrXml, NodeAddedTriggerIntegrationTest, TestCursorMarkWithoutUniqueKey, DeleteLastCustomShardedReplicaTest, AssignBackwardCompatibilityTest, ClusterStateUpdateTest, CollectionPropsTest, DeleteShardTest, DeleteStatusTest, DistribDocExpirationUpdateProcessorTest, LeaderFailureAfterFreshStartTest, MissingSegmentRecoveryTest, MoveReplicaHDFSFailoverTest, MoveReplicaHDFSTest, MoveReplicaTest, MultiSolrCloudTestCaseTest, OverseerTaskQueueTest, RestartWhileUpdatingTest, RollingRestartTest, SliceStateTest, SolrCloudExampleTest, TestCloudDeleteByQuery, TestCloudPivotFacet, TestDistribDocBasedVersion, TestPrepRecovery, TestRandomRequestDistribution, TestRequestForwarding, TestSSLRandomization, TestStressCloudBlindAtomicUpdates, ZkShardTermsTest, SimpleCollectionCreateDeleteTest, AutoAddReplicasPlanActionTest, AutoScalingHandlerTest, ComputePlanActionTest, IndexSizeTriggerTest, NodeLostTriggerIntegrationTest, NodeLostTriggerTest, ScheduledMaintenanceTriggerTest, TestPolicyCloud, TestSimComputePlanAction, TestSimDistribStateManager, TestSimDistributedQueue, TestSimNodeLostTrigger, TestSimTriggerIntegration] [junit4] Completed [547/840 (1!)] on J1 in 10.01s, 11 tests, 1 error <<< FAILURES! [...truncated 46217 lines...] [repro] Jenkins log URL: https://jenkins.thetaphi.de/job/Lucene-Solr-8.0-Linux/231/consoleText [repro] Revision: 0376bc0052a53480ecb2edea7dfe58298bda5deb [repro] Ant options: "-Dargs=-server -XX:+UseSerialGC" [repro] Repro line: ant test -Dtestcase=TestSimTriggerIntegration -Dtests.method=testNodeAddedTriggerRestoreState -Dtests.seed=A27319673644C5A9 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=es-CU -Dtests.timezone=Europe/Riga -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1 [repro] ant clean [...truncated 6 lines...] [repro] Test suites by module: [repro] solr/core [repro] TestSimTriggerIntegration [repro] ant compile-test [...truncated 3572 lines...] [repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=5 -Dtests.class="*.TestSimTriggerIntegration" -Dtests.showOutput=onerror "-Dargs=-server -XX:+UseSerialGC" -Dtests.seed=A27319673644C5A9 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=es-CU -Dtests.timezone=Europe/Riga -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1 [...truncated 120 lines...] [repro] Failures: [repro] 0/5 failed: org.apache.solr.cloud.autoscaling.sim.TestSimTriggerIntegration [repro] Exiting with code 0 [...truncated 43 lines...]
--------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org