Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/3582/ Java: 64bit/jdk-12-ea+shipilev-fastdebug -XX:+UseCompressedOops -XX:+UseParallelGC
1 tests failed. FAILED: org.apache.solr.cloud.autoscaling.sim.TestSimTriggerIntegration.testSearchRate Error Message: The trigger did not start in time Stack Trace: java.lang.AssertionError: The trigger did not start in time at __randomizedtesting.SeedInfo.seed([A13AF2A7B45074BC:FC72EC2E7B96D2F3]:0) at org.junit.Assert.fail(Assert.java:88) at org.junit.Assert.assertTrue(Assert.java:41) at org.apache.solr.cloud.autoscaling.sim.TestSimTriggerIntegration.testSearchRate(TestSimTriggerIntegration.java:1370) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:567) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938) at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974) at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368) at java.base/java.lang.Thread.run(Thread.java:835) Build Log: [...truncated 14559 lines...] [junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestSimTriggerIntegration [junit4] 2> 2177452 INFO (SUITE-TestSimTriggerIntegration-seed#[A13AF2A7B45074BC]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom [junit4] 2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.sim.TestSimTriggerIntegration_A13AF2A7B45074BC-001/init-core-data-001 [junit4] 2> 2177453 INFO (SUITE-TestSimTriggerIntegration-seed#[A13AF2A7B45074BC]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true [junit4] 2> 2177454 INFO (SUITE-TestSimTriggerIntegration-seed#[A13AF2A7B45074BC]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason="", value=0.0/0.0, ssl=0.0/0.0, clientAuth=0.0/0.0) [junit4] 2> 2177456 INFO (TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[A13AF2A7B45074BC]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testNodeMarkersRegistration [junit4] 2> 2177459 DEBUG (TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10000_solr [junit4] 2> 2177459 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers [junit4] 2> 2177460 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1 [junit4] 2> 2177460 DEBUG (TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=0 [junit4] 2> 2177460 DEBUG (TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 0 [junit4] 2> 2177460 INFO (TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[A13AF2A7B45074BC]) [ ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory. [junit4] 2> 2177460 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 0 [junit4] 2> 2177460 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1 [junit4] 2> 2177460 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10000_solr] [junit4] 2> 2177460 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers [junit4] 2> 2177460 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1 [junit4] 2> 2177461 DEBUG (ScheduledTrigger-9894-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1 and last live nodes: 1 [junit4] 2> 2177484 DEBUG (ScheduledTrigger-9894-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 1 [junit4] 2> 2177506 DEBUG (ScheduledTrigger-9894-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2177526 DEBUG (ScheduledTrigger-9894-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2177547 DEBUG (ScheduledTrigger-9894-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2177567 DEBUG (ScheduledTrigger-9894-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2177587 DEBUG (ScheduledTrigger-9894-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2177607 DEBUG (ScheduledTrigger-9894-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2177616 DEBUG (simCloudManagerPool-9893-thread-2) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration [junit4] 2> 2177617 DEBUG (simCloudManagerPool-9893-thread-2) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2 [junit4] 2> 2177617 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2 [junit4] 2> 2177617 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers [junit4] 2> 2177617 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2 [junit4] 2> 2177618 INFO (TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.s.TestSimTriggerIntegration ====== KILL OVERSEER 1 [junit4] 2> 2177618 INFO (TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache... [junit4] 2> 2177618 DEBUG (TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now [junit4] 2> 2177618 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting. [junit4] 2> 2177618 DEBUG (TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now [junit4] 2> 2177619 DEBUG (TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor [junit4] 2> 2177619 DEBUG (TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor [junit4] 2> 2177619 DEBUG (TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely [junit4] 2> 2177619 DEBUG (TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly [junit4] 2> 2177619 INFO (TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.s.SimCloudManager = killing node 127.0.0.1:10000_solr [junit4] 2> 2177619 DEBUG (TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.s.SimClusterStateProvider -- created marker: /autoscaling/nodeLost/127.0.0.1:10000_solr [junit4] 2> 2177619 DEBUG (TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10001_solr [junit4] 2> 2177620 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2 [junit4] 2> 2177620 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 0 [junit4] 2> 2177620 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2 [junit4] 2> 2177620 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10002_solr] [junit4] 2> 2177620 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger Adding lost node from marker path: 127.0.0.1:10000_solr [junit4] 2> 2177620 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers [junit4] 2> 2177620 DEBUG (ScheduledTrigger-9899-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2177620 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2 [junit4] 2> 2177621 DEBUG (ScheduledTrigger-9899-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10000_solr [junit4] 2> 2177641 DEBUG (ScheduledTrigger-9899-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2177661 DEBUG (ScheduledTrigger-9899-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2177681 DEBUG (ScheduledTrigger-9899-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2177701 DEBUG (ScheduledTrigger-9899-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2177722 DEBUG (ScheduledTrigger-9899-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2177742 DEBUG (ScheduledTrigger-9899-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2177760 INFO (TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.s.TestSimTriggerIntegration ====== ADD TRIGGERS [junit4] 2> 2177762 DEBUG (ScheduledTrigger-9899-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2177779 INFO (simCloudManagerPool-9898-thread-1) [ ] o.a.s.c.a.s.TestSimTriggerIntegration TestEventMarkerAction init [junit4] 2> 2177780 DEBUG (simCloudManagerPool-9898-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration [junit4] 2> 2177780 DEBUG (simCloudManagerPool-9898-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3 [junit4] 2> 2177780 INFO (simCloudManagerPool-9898-thread-1) [ ] o.a.s.c.a.s.TestSimTriggerIntegration TestEventMarkerAction init [junit4] 2> 2177780 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3 [junit4] 2> 2177781 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10002_solr] [junit4] 2> 2177781 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestSimTriggerIntegration$TestEventMarkerAction}], enabled=true} [junit4] 2> 2177781 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers [junit4] 2> 2177781 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3 [junit4] 2> 2177781 DEBUG (ScheduledTrigger-9899-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 2177781 DEBUG (ScheduledTrigger-9899-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2 [junit4] 2> 2177781 INFO (simCloudManagerPool-9898-thread-2) [ ] o.a.s.c.a.s.TestSimTriggerIntegration TestEventMarkerAction init [junit4] 2> 2177781 DEBUG (simCloudManagerPool-9898-thread-2) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration [junit4] 2> 2177781 DEBUG (simCloudManagerPool-9898-thread-2) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4 [junit4] 2> 2177782 INFO (simCloudManagerPool-9898-thread-2) [ ] o.a.s.c.a.s.TestSimTriggerIntegration TestEventMarkerAction init [junit4] 2> 2177782 INFO (simCloudManagerPool-9898-thread-2) [ ] o.a.s.c.a.s.TestSimTriggerIntegration TestEventMarkerAction init [junit4] 2> 2177782 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4 [junit4] 2> 2177782 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger node_lost_trigger - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10002_solr] [junit4] 2> 2177782 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers [junit4] 2> 2177782 DEBUG (ScheduledTrigger-9899-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2177782 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4 [junit4] 2> 2177782 DEBUG (ScheduledTrigger-9899-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2177787 INFO (TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.s.TestSimTriggerIntegration ====== ADD NODE 1 [junit4] 2> 2177787 DEBUG (TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.s.SimClusterStateProvider -- creating marker: /autoscaling/nodeAdded/127.0.0.1:10003_solr [junit4] 2> 2177787 INFO (TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.s.TestSimTriggerIntegration ====== KILL OVERSEER 2 [junit4] 2> 2177787 INFO (TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache... [junit4] 2> 2177787 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting. [junit4] 2> 2177788 DEBUG (TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now [junit4] 2> 2177788 DEBUG (TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now [junit4] 2> 2177788 DEBUG (TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor [junit4] 2> 2177788 DEBUG (TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor [junit4] 2> 2177788 DEBUG (TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely [junit4] 2> 2177788 DEBUG (TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly [junit4] 2> 2177788 INFO (TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.s.SimCloudManager = killing node 127.0.0.1:10001_solr [junit4] 2> 2177789 DEBUG (TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.s.SimClusterStateProvider -- created marker: /autoscaling/nodeLost/127.0.0.1:10001_solr [junit4] 2> 2177789 DEBUG (TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10002_solr [junit4] 2> 2177789 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4 [junit4] 2> 2177789 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestSimTriggerIntegration TestEventMarkerAction init [junit4] 2> 2177789 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestSimTriggerIntegration TestEventMarkerAction init [junit4] 2> 2177789 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 0 [junit4] 2> 2177790 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4 [junit4] 2> 2177790 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger node_lost_trigger - Initial livenodes: [127.0.0.1:10002_solr, 127.0.0.1:10003_solr] [junit4] 2> 2177790 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger Adding lost node from marker path: 127.0.0.1:10001_solr [junit4] 2> 2177790 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - Initial livenodes: [127.0.0.1:10002_solr, 127.0.0.1:10003_solr] [junit4] 2> 2177790 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestSimTriggerIntegration$TestEventMarkerAction}], enabled=true} [junit4] 2> 2177790 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger Adding node from marker path: 127.0.0.1:10003_solr [junit4] 2> 2177790 DEBUG (ScheduledTrigger-9902-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2177790 DEBUG (ScheduledTrigger-9902-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10001_solr [junit4] 2> 2177790 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10002_solr, 127.0.0.1:10003_solr] [junit4] 2> 2177790 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger Adding lost node from marker path: 127.0.0.1:10001_solr [junit4] 2> 2177790 DEBUG (ScheduledTrigger-9902-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2177790 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers [junit4] 2> 2177791 DEBUG (ScheduledTrigger-9902-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10001_solr [junit4] 2> 2177791 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4 [junit4] 2> 2177790 DEBUG (ScheduledTrigger-9902-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 2177791 DEBUG (ScheduledTrigger-9902-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2 [junit4] 2> 2177791 DEBUG (ScheduledTrigger-9902-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10003_solr at time 719134872615285 [junit4] 2> 2177811 DEBUG (ScheduledTrigger-9902-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2177811 DEBUG (ScheduledTrigger-9902-thread-3) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:10001_solr] [junit4] 2> 2177811 DEBUG (ScheduledTrigger-9902-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2177811 DEBUG (ScheduledTrigger-9902-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 2177812 DEBUG (ScheduledTrigger-9902-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2 [junit4] 2> 2177812 DEBUG (ScheduledTrigger-9902-thread-3) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: { [junit4] 2> "id":"28e0ca1a3a07dTarpu8w4c1qhsr5p4grq3a1wp4", [junit4] 2> "source":"node_lost_trigger", [junit4] 2> "eventTime":719134856028285, [junit4] 2> "eventType":"NODELOST", [junit4] 2> "properties":{ [junit4] 2> "eventTimes":[719134856028285], [junit4] 2> "preferredOperation":"movereplica", [junit4] 2> "nodeNames":["127.0.0.1:10001_solr"]}} [junit4] 2> 2177812 DEBUG (ScheduledTrigger-9902-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger firing registered processor for nodes: [127.0.0.1:10003_solr] added at times [719134872615285], now=719135920447785 [junit4] 2> 2177812 DEBUG (ScheduledTrigger-9902-thread-3) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_lost_trigger, node_added_trigger] [junit4] 2> 2177812 DEBUG (ScheduledTrigger-9902-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -------- Cooldown period - rejecting event: { [junit4] 2> "id":"28e0ca2a0b975Tarpu8w4c1qhsr5p4grq3a1wp5", [junit4] 2> "source":"node_added_trigger", [junit4] 2> "eventTime":719134872615285, [junit4] 2> "eventType":"NODEADDED", [junit4] 2> "properties":{ [junit4] 2> "eventTimes":[719134872615285], [junit4] 2> "preferredOperation":"movereplica", [junit4] 2> "nodeNames":["127.0.0.1:10003_solr"]}} [junit4] 2> 2177812 DEBUG (ScheduledTrigger-9902-thread-3) [ ] o.a.s.c.a.SystemLogListener Collection .system missing, skip sending event { [junit4] 2> "id":"28e0ca1a3a07dTarpu8w4c1qhsr5p4grq3a1wp4", [junit4] 2> "source":"node_lost_trigger", [junit4] 2> "eventTime":719134856028285, [junit4] 2> "eventType":"NODELOST", [junit4] 2> "properties":{ [junit4] 2> "eventTimes":[719134856028285], [junit4] 2> "preferredOperation":"movereplica", [junit4] 2> "_enqueue_time_":719135932383435, [junit4] 2> "nodeNames":["127.0.0.1:10001_solr"]}} [junit4] 2> 2177812 DEBUG (ScheduledTrigger-9902-thread-1) [ ] o.a.s.c.a.SystemLogListener Collection .system missing, skip sending event { [junit4] 2> "id":"28e0ca2a0b975Tarpu8w4c1qhsr5p4grq3a1wp5", [junit4] 2> "source":"node_added_trigger", [junit4] 2> "eventTime":719134872615285, [junit4] 2> "eventType":"NODEADDED", [junit4] 2> "properties":{ [junit4] 2> "eventTimes":[719134872615285], [junit4] 2> "cooldown":true, [junit4] 2> "preferredOperation":"movereplica", [junit4] 2> "nodeNames":["127.0.0.1:10003_solr"]}} [junit4] 2> 2177812 DEBUG (ScheduledTrigger-9902-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Processor returned false for [127.0.0.1:10003_solr]! [junit4] 2> 2177812 DEBUG (AutoscalingActionExecutor-9903-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for { [junit4] 2> "id":"28e0ca1a3a07dTarpu8w4c1qhsr5p4grq3a1wp4", [junit4] 2> "source":"node_lost_trigger", [junit4] 2> "eventTime":719134856028285, [junit4] 2> "eventType":"NODELOST", [junit4] 2> "properties":{ [junit4] 2> "eventTimes":[719134856028285], [junit4] 2> "preferredOperation":"movereplica", [junit4] 2> "_enqueue_time_":719135932383435, [junit4] 2> "nodeNames":["127.0.0.1:10001_solr"]}} [junit4] 2> 2177813 DEBUG (AutoscalingActionExecutor-9903-thread-1) [ ] o.a.s.c.a.SystemLogListener Collection .system missing, skip sending event { [junit4] 2> "id":"28e0ca1a3a07dTarpu8w4c1qhsr5p4grq3a1wp4", [junit4] 2> "source":"node_lost_trigger", [junit4] 2> "eventTime":719134856028285, [junit4] 2> "eventType":"NODELOST", [junit4] 2> "properties":{ [junit4] 2> "eventTimes":[719134856028285], [junit4] 2> "preferredOperation":"movereplica", [junit4] 2> "_enqueue_time_":719135932383435, [junit4] 2> "nodeNames":["127.0.0.1:10001_solr"]}} [junit4] 2> 2177813 DEBUG (AutoscalingActionExecutor-9903-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger after 100ms [junit4] 2> 2177813 DEBUG (AutoscalingActionExecutor-9903-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms [junit4] 2> 2177813 DEBUG (AutoscalingActionExecutor-9903-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger after 100ms [junit4] 2> 2177813 DEBUG (AutoscalingActionExecutor-9903-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 30 ms for event id=28e0ca1a3a07dTarpu8w4c1qhsr5p4grq3a1wp4 [junit4] 2> 2177913 DEBUG (ScheduledTrigger-9902-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2177913 DEBUG (ScheduledTrigger-9902-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2177913 DEBUG (ScheduledTrigger-9902-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger [junit4] 2> 2177913 DEBUG (ScheduledTrigger-9902-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2 [junit4] 2> 2177913 DEBUG (ScheduledTrigger-9902-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger firing registered processor for nodes: [127.0.0.1:10003_solr] added at times [719134872615285], now=719140998940185 [junit4] 2> 2177913 DEBUG (ScheduledTrigger-9902-thread-1) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: { [junit4] 2> "id":"28e0ca2a0b975Tarpu8w4c1qhsr5p4grq3a1wp6", [junit4] 2> "source":"node_added_trigger", [junit4] 2> "eventTime":719134872615285, [junit4] 2> "eventType":"NODEADDED", [junit4] 2> "properties":{ [junit4] 2> "eventTimes":[719134872615285], [junit4] 2> "preferredOperation":"movereplica", [junit4] 2> "nodeNames":["127.0.0.1:10003_solr"]}} [junit4] 2> 2177913 DEBUG (ScheduledTrigger-9902-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_lost_trigger, node_added_trigger] [junit4] 2> 2177914 DEBUG (ScheduledTrigger-9902-thread-1) [ ] o.a.s.c.a.SystemLogListener Collection .system missing, skip sending event { [junit4] 2> "id":"28e0ca2a0b975Tarpu8w4c1qhsr5p4grq3a1wp6", [junit4] 2> "source":"node_added_trigger", [junit4] 2> "eventTime":719134872615285, [junit4] 2> "eventType":"NODEADDED", [junit4] 2> "properties":{ [junit4] 2> "eventTimes":[719134872615285], [junit4] 2> "preferredOperation":"movereplica", [junit4] 2> "_enqueue_time_":719141011976035, [junit4] 2> "nodeNames":["127.0.0.1:10003_solr"]}} [junit4] 2> 2177914 DEBUG (ScheduledTrigger-9902-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Removing new node from tracking: 127.0.0.1:10003_solr [junit4] 2> 2177914 DEBUG (AutoscalingActionExecutor-9903-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for { [junit4] 2> "id":"28e0ca2a0b975Tarpu8w4c1qhsr5p4grq3a1wp6", [junit4] 2> "source":"node_added_trigger", [junit4] 2> "eventTime":719134872615285, [junit4] 2> "eventType":"NODEADDED", [junit4] 2> "properties":{ [junit4] 2> "eventTimes":[719134872615285], [junit4] 2> "preferredOperation":"movereplica", [junit4] 2> "_enqueue_time_":719141011976035, [junit4] 2> "nodeNames":["127.0.0.1:10003_solr"]}} [junit4] 2> 2177914 DEBUG (AutoscalingActionExecutor-9903-thread-1) [ ] o.a.s.c.a.SystemLogListener Collection .system missing, skip sending event { [junit4] 2> "id":"28e0ca2a0b975Tarpu8w4c1qhsr5p4grq3a1wp6", [junit4] 2> "source":"node_added_trigger", [junit4] 2> "eventTime":719134872615285, [junit4] 2> "eventType":"NODEADDED", [junit4] 2> "properties":{ [junit4] 2> "eventTimes":[719134872615285], [junit4] 2> "preferredOperation":"movereplica", [junit4] 2> "_enqueue_time_":719141011976035, [junit4] 2> "nodeNames":["127.0.0.1:10003_solr"]}} [junit4] 2> 2177914 DEBUG (AutoscalingActionExecutor-9903-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger after 100ms [junit4] 2> 2177914 DEBUG (AutoscalingActionExecutor-9903-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms [junit4] 2> 2177914 DEBUG (AutoscalingActionExecutor-9903-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger after 100ms [junit4] 2> 2177914 DEBUG (AutoscalingActionExecutor-9903-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 18 ms for event id=28e0ca2a0b975Tarpu8w4c1qhsr5p4grq3a1wp6 [junit4] 2> 2177933 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting. [junit4] 2> 2177933 DEBUG (TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now [junit4] 2> 2177934 DEBUG (TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now [junit4] 2> 2177934 DEBUG (TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor [junit4] 2> 2177934 DEBUG (TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor [junit4] 2> 2177934 DEBUG (TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely [junit4] 2> 2177934 DEBUG (TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly [junit4] 2> 2177935 INFO (TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[A13AF2A7B45074BC]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testNodeMarkersRegistration [junit4] IGNOR/A 0.00s J0 | TestSimTriggerIntegration.testListeners [junit4] > Assumption #1: 'badapple' test group is disabled (@BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028")) [junit4] 2> 2177937 INFO (TEST-TestSimTriggerIntegration.testSearchRate-seed#[A13AF2A7B45074BC]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testSearchRate [junit4] 2> 2177940 DEBUG (TEST-TestSimTriggerIntegration.testSearchRate-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10004_solr [junit4] 2> 2177940 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers [junit4] 2> 2177940 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1 [junit4] 2> 2177941 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 0 [junit4] 2> 2177941 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1 [junit4] 2> 2177941 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10004_solr] [junit4] 2> 2177941 DEBUG (TEST-TestSimTriggerIntegration.testSearchRate-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=0 [junit4] 2> 2177941 DEBUG (TEST-TestSimTriggerIntegration.testSearchRate-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 0 [junit4] 2> 2177941 DEBUG (ScheduledTrigger-9906-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1 and last live nodes: 1 [junit4] 2> 2177941 INFO (TEST-TestSimTriggerIntegration.testSearchRate-seed#[A13AF2A7B45074BC]) [ ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory. [junit4] 2> 2177941 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers [junit4] 2> 2177942 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1 [junit4] 2> 2177947 DEBUG (simCloudManagerPool-9905-thread-2) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration [junit4] 2> 2177947 DEBUG (simCloudManagerPool-9905-thread-2) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2 [junit4] 2> 2177948 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2 [junit4] 2> 2177948 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers [junit4] 2> 2177948 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2 [junit4] 2> 2177972 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 1 [junit4] 2> 2177972 DEBUG (simCloudManagerPool-9905-thread-3) [ ] o.a.s.c.a.s.SimClusterStateProvider -- simCreateCollection collection1, currentVersion=1 [junit4] 2> 2177973 DEBUG (simCloudManagerPool-9905-thread-3) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=1 [junit4] 2> 2177973 DEBUG (simCloudManagerPool-9905-thread-3) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 1 [junit4] 2> 2177992 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2178013 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2178033 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2178053 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2178073 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2178093 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2178113 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2178134 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2178154 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2178164 DEBUG (simCloudManagerPool-9905-thread-3) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=2 [junit4] 2> 2178164 DEBUG (simCloudManagerPool-9905-thread-3) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 2 [junit4] 2> 2178174 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2178194 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2178214 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2178235 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2178255 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2178275 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2178295 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2178315 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2178336 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2178356 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2178376 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2178396 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2178414 DEBUG (simCloudManagerPool-9905-thread-4) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=3 [junit4] 2> 2178414 DEBUG (simCloudManagerPool-9905-thread-4) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 3 [junit4] 2> 2178416 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2178436 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2178437 DEBUG (simCloudManagerPool-9905-thread-3) [ ] o.a.s.c.a.s.SimClusterStateProvider -- finished createCollection collection1, currentVersion=4 [junit4] 2> 2178447 DEBUG (simCloudManagerPool-9905-thread-5) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=4 [junit4] 2> 2178466 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2178466 DEBUG (simCloudManagerPool-9905-thread-5) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 4 [junit4] 2> 2178486 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2178506 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2178526 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2178546 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2178567 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2178587 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2178613 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2178633 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2178633 INFO (TEST-TestSimTriggerIntegration.testSearchRate-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.CloudTestUtils -- shards without leaders: [shard1] [junit4] 2> 2178636 INFO (TEST-TestSimTriggerIntegration.testSearchRate-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.CloudTestUtils -- shards without leaders: [shard1] [junit4] 2> 2178638 INFO (TEST-TestSimTriggerIntegration.testSearchRate-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.CloudTestUtils -- shards without leaders: [shard1] [junit4] 2> 2178640 INFO (TEST-TestSimTriggerIntegration.testSearchRate-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.CloudTestUtils -- shards without leaders: [shard1] [junit4] 2> 2178642 INFO (TEST-TestSimTriggerIntegration.testSearchRate-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.CloudTestUtils -- shards without leaders: [shard1] [junit4] 2> 2178644 INFO (TEST-TestSimTriggerIntegration.testSearchRate-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.CloudTestUtils -- shards without leaders: [shard1] [junit4] 2> 2178646 INFO (TEST-TestSimTriggerIntegration.testSearchRate-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.CloudTestUtils -- shards without leaders: [shard1] [junit4] 2> 2178649 INFO (TEST-TestSimTriggerIntegration.testSearchRate-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.CloudTestUtils -- shards without leaders: [shard1] [junit4] 2> 2178651 INFO (TEST-TestSimTriggerIntegration.testSearchRate-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.CloudTestUtils -- shards without leaders: [shard1] [junit4] 2> 2178653 INFO (TEST-TestSimTriggerIntegration.testSearchRate-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.CloudTestUtils -- shards without leaders: [shard1] [junit4] 2> 2178653 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2178655 INFO (TEST-TestSimTriggerIntegration.testSearchRate-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.CloudTestUtils -- shards without leaders: [shard1] [junit4] 2> 2178657 INFO (TEST-TestSimTriggerIntegration.testSearchRate-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.CloudTestUtils -- shards without leaders: [shard1] [junit4] 2> 2178659 INFO (TEST-TestSimTriggerIntegration.testSearchRate-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.CloudTestUtils -- shards without leaders: [shard1] [junit4] 2> 2178662 INFO (TEST-TestSimTriggerIntegration.testSearchRate-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.CloudTestUtils -- shards without leaders: [shard1] [junit4] 2> 2178664 INFO (TEST-TestSimTriggerIntegration.testSearchRate-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.CloudTestUtils -- shards without leaders: [shard1] [junit4] 2> 2178666 INFO (TEST-TestSimTriggerIntegration.testSearchRate-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.CloudTestUtils -- shards without leaders: [shard1] [junit4] 2> 2178668 INFO (TEST-TestSimTriggerIntegration.testSearchRate-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.CloudTestUtils -- shards without leaders: [shard1] [junit4] 2> 2178670 INFO (TEST-TestSimTriggerIntegration.testSearchRate-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.CloudTestUtils -- shards without leaders: [shard1] [junit4] 2> 2178672 INFO (TEST-TestSimTriggerIntegration.testSearchRate-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.CloudTestUtils -- shards without leaders: [shard1] [junit4] 2> 2178673 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2178675 INFO (TEST-TestSimTriggerIntegration.testSearchRate-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.CloudTestUtils -- shards without leaders: [shard1] [junit4] 2> 2178677 INFO (TEST-TestSimTriggerIntegration.testSearchRate-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.CloudTestUtils -- shards without leaders: [shard1] [junit4] 2> 2178679 INFO (TEST-TestSimTriggerIntegration.testSearchRate-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.CloudTestUtils -- shards without leaders: [shard1] [junit4] 2> 2178681 INFO (TEST-TestSimTriggerIntegration.testSearchRate-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.CloudTestUtils -- shards without leaders: [shard1] [junit4] 2> 2178683 INFO (TEST-TestSimTriggerIntegration.testSearchRate-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.CloudTestUtils -- shards without leaders: [shard1] [junit4] 2> 2178685 INFO (TEST-TestSimTriggerIntegration.testSearchRate-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.CloudTestUtils -- shards without leaders: [shard1] [junit4] 2> 2178687 INFO (TEST-TestSimTriggerIntegration.testSearchRate-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.CloudTestUtils -- shards without leaders: [shard1] [junit4] 2> 2178690 INFO (TEST-TestSimTriggerIntegration.testSearchRate-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.CloudTestUtils -- shards without leaders: [shard1] [junit4] 2> 2178692 INFO (TEST-TestSimTriggerIntegration.testSearchRate-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.CloudTestUtils -- shards without leaders: [shard1] [junit4] 2> 2178693 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2178694 INFO (TEST-TestSimTriggerIntegration.testSearchRate-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.CloudTestUtils -- shards without leaders: [shard1] [junit4] 2> 2178696 INFO (TEST-TestSimTriggerIntegration.testSearchRate-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.CloudTestUtils -- shards without leaders: [shard1] [junit4] 2> 2178698 INFO (TEST-TestSimTriggerIntegration.testSearchRate-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.CloudTestUtils -- shards without leaders: [shard1] [junit4] 2> 2178700 INFO (TEST-TestSimTriggerIntegration.testSearchRate-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.CloudTestUtils -- shards without leaders: [shard1] [junit4] 2> 2178703 INFO (TEST-TestSimTriggerIntegration.testSearchRate-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.CloudTestUtils -- shards without leaders: [shard1] [junit4] 2> 2178705 INFO (TEST-TestSimTriggerIntegration.testSearchRate-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.CloudTestUtils -- shards without leaders: [shard1] [junit4] 2> 2178707 INFO (TEST-TestSimTriggerIntegration.testSearchRate-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.CloudTestUtils -- shards without leaders: [shard1] [junit4] 2> 2178709 INFO (TEST-TestSimTriggerIntegration.testSearchRate-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.CloudTestUtils -- shards without leaders: [shard1] [junit4] 2> 2178711 INFO (TEST-TestSimTriggerIntegration.testSearchRate-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.CloudTestUtils -- shards without leaders: [shard1] [junit4] 2> 2178713 INFO (TEST-TestSimTriggerIntegration.testSearchRate-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.CloudTestUtils -- shards without leaders: [shard1] [junit4] 2> 2178713 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2178716 INFO (TEST-TestSimTriggerIntegration.testSearchRate-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.CloudTestUtils -- shards without leaders: [shard1] [junit4] 2> 2178718 INFO (TEST-TestSimTriggerIntegration.testSearchRate-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.CloudTestUtils -- shards without leaders: [shard1] [junit4] 2> 2178720 INFO (TEST-TestSimTriggerIntegration.testSearchRate-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.CloudTestUtils -- shards without leaders: [shard1] [junit4] 2> 2178722 INFO (TEST-TestSimTriggerIntegration.testSearchRate-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.CloudTestUtils -- shards without leaders: [shard1] [junit4] 2> 2178724 INFO (TEST-TestSimTriggerIntegration.testSearchRate-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.CloudTestUtils -- shards without leaders: [shard1] [junit4] 2> 2178726 INFO (TEST-TestSimTriggerIntegration.testSearchRate-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.CloudTestUtils -- shards without leaders: [shard1] [junit4] 2> 2178729 INFO (TEST-TestSimTriggerIntegration.testSearchRate-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.CloudTestUtils -- shards without leaders: [shard1] [junit4] 2> 2178731 INFO (TEST-TestSimTriggerIntegration.testSearchRate-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.CloudTestUtils -- shards without leaders: [shard1] [junit4] 2> 2178613 DEBUG (simCloudManagerPool-9905-thread-6) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for collection1 / shard1 (currentVersion=5): {"core_node1":{ [junit4] 2> "core":"collection1_shard1_replica_n1", [junit4] 2> "shard":"shard1", [junit4] 2> "collection":"collection1", [junit4] 2> "node_name":"127.0.0.1:10004_solr", [junit4] 2> "type":"NRT", [junit4] 2> "leader":"true", [junit4] 2> "SEARCHER.searcher.maxDoc":0, [junit4] 2> "SEARCHER.searcher.deletedDocs":0, [junit4] 2> "INDEX.sizeInBytes":10240, [junit4] 2> "state":"active", [junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6, [junit4] 2> "SEARCHER.searcher.numDocs":0}} [junit4] 2> 2178733 DEBUG (TEST-TestSimTriggerIntegration.testSearchRate-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=5 [junit4] 2> 2178733 DEBUG (TEST-TestSimTriggerIntegration.testSearchRate-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 5 [junit4] 2> 2178734 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2178754 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2178774 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2178794 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2178814 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2178835 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2178855 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2178875 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2178883 DEBUG (simCloudManagerPool-9905-thread-8) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration [junit4] 2> 2178884 DEBUG (simCloudManagerPool-9905-thread-8) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3 [junit4] 2> 2178884 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3 [junit4] 2> 2178885 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers [junit4] 2> 2178885 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3 [junit4] 2> 2178895 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2178915 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2178931 DEBUG (simCloudManagerPool-9905-thread-9) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration [junit4] 2> 2178931 DEBUG (simCloudManagerPool-9905-thread-9) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4 [junit4] 2> 2178931 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4 [junit4] 2> 2178932 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers [junit4] 2> 2178932 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4 [junit4] 2> 2178936 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2178956 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2178976 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2178996 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2179016 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2179037 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2179057 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2179077 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2179097 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2179117 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2179138 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2179158 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2179178 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2179180 DEBUG (ScheduledTrigger-9906-thread-2) [ ] o.a.s.c.a.SearchRateTrigger ### rates for node 127.0.0.1:10004_solr [junit4] 2> 2179198 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2179199 DEBUG (ScheduledTrigger-9906-thread-2) [ ] o.a.s.c.a.SearchRateTrigger ### metrics:solr.core.collection1.shard1.replica_n1:QUERY./select.requestTimes:1minRate 250 [junit4] 2> 2179218 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2179239 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2179259 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2179279 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2179280 DEBUG (ScheduledTrigger-9906-thread-2) [ ] o.a.s.c.a.SearchRateTrigger ### rates for node 127.0.0.1:10005_solr [junit4] 2> 2179280 DEBUG (ScheduledTrigger-9906-thread-2) [ ] o.a.s.c.a.SearchRateTrigger ### metrics:solr.core.collection1.shard1.replica_n2:QUERY./select.requestTimes:1minRate 250 [junit4] 2> 2179298 DEBUG (ScheduledTrigger-9906-thread-2) [ ] o.a.s.c.a.SearchRateTrigger ## Collection: collection1 [junit4] 2> 2179299 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2179319 DEBUG (ScheduledTrigger-9906-thread-2) [ ] o.a.s.c.a.SearchRateTrigger ## - shard1 [junit4] 2> 2179319 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2179340 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2179340 DEBUG (ScheduledTrigger-9906-thread-2) [ ] o.a.s.c.a.SearchRateTrigger ## collection1_shard1_replica_n1 250.0 [junit4] 2> 2179340 DEBUG (ScheduledTrigger-9906-thread-2) [ ] o.a.s.c.a.SearchRateTrigger ## collection1_shard1_replica_n2 250.0 [junit4] 2> 2179360 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2179380 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2179400 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2179420 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2179440 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2179461 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2179481 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2179501 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2179521 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2179541 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2179562 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2179582 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2179602 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2179622 DEBUG (ScheduledTrigger-9906-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2179626 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting. [junit4] 2> 2179631 DEBUG (TEST-TestSimTriggerIntegration.testSearchRate-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now [junit4] 2> 2179631 DEBUG (TEST-TestSimTriggerIntegration.testSearchRate-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now [junit4] 2> 2179632 DEBUG (TEST-TestSimTriggerIntegration.testSearchRate-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor [junit4] 2> 2179632 DEBUG (TEST-TestSimTriggerIntegration.testSearchRate-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor [junit4] 2> 2180071 DEBUG (ScheduledTrigger-9906-thread-2) [ ] o.a.s.c.a.SearchRateTrigger -- collection1.shard1: totalShardRate=500.0, shardRate=250.0 [junit4] 2> 2180147 DEBUG (TEST-TestSimTriggerIntegration.testSearchRate-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely [junit4] 2> 2180148 DEBUG (TEST-TestSimTriggerIntegration.testSearchRate-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly [junit4] 2> 2180149 INFO (TEST-TestSimTriggerIntegration.testSearchRate-seed#[A13AF2A7B45074BC]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testSearchRate [junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestSimTriggerIntegration -Dtests.method=testSearchRate -Dtests.seed=A13AF2A7B45074BC -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=et -Dtests.timezone=Pacific/Funafuti -Dtests.asserts=true -Dtests.file.encoding=UTF-8 [junit4] FAILURE 2.26s J0 | TestSimTriggerIntegration.testSearchRate <<< [junit4] > Throwable #1: java.lang.AssertionError: The trigger did not start in time [junit4] > at __randomizedtesting.SeedInfo.seed([A13AF2A7B45074BC:FC72EC2E7B96D2F3]:0) [junit4] > at org.apache.solr.cloud.autoscaling.sim.TestSimTriggerIntegration.testSearchRate(TestSimTriggerIntegration.java:1370) [junit4] > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [junit4] > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) [junit4] > at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [junit4] > at java.base/java.lang.reflect.Method.invoke(Method.java:567) [junit4] > at java.base/java.lang.Thread.run(Thread.java:835) [junit4] 2> 2180193 INFO (TEST-TestSimTriggerIntegration.testCooldown-seed#[A13AF2A7B45074BC]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testCooldown [junit4] 2> 2180196 DEBUG (TEST-TestSimTriggerIntegration.testCooldown-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10006_solr [junit4] 2> 2180197 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers [junit4] 2> 2180197 DEBUG (TEST-TestSimTriggerIntegration.testCooldown-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=0 [junit4] 2> 2180197 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1 [junit4] 2> 2180197 DEBUG (TEST-TestSimTriggerIntegration.testCooldown-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 0 [junit4] 2> 2180197 INFO (TEST-TestSimTriggerIntegration.testCooldown-seed#[A13AF2A7B45074BC]) [ ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory. [junit4] 2> 2180197 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 0 [junit4] 2> 2180197 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1 [junit4] 2> 2180198 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10006_solr, 127.0.0.1:10007_solr] [junit4] 2> 2180198 DEBUG (ScheduledTrigger-9912-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2180199 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers [junit4] 2> 2180199 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1 [junit4] 2> 2180199 DEBUG (simCloudManagerPool-9911-thread-2) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration [junit4] 2> 2180200 DEBUG (simCloudManagerPool-9911-thread-2) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2 [junit4] 2> 2180200 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2 [junit4] 2> 2180201 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers [junit4] 2> 2180201 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2 [junit4] 2> 2180222 DEBUG (ScheduledTrigger-9912-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2 [junit4] 2> 2180223 DEBUG (simCloudManagerPool-9911-thread-3) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration [junit4] 2> 2180223 DEBUG (simCloudManagerPool-9911-thread-3) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3 [junit4] 2> 2180223 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3 [junit4] 2> 2180224 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestSimTriggerIntegration TestTriggerAction init [junit4] 2> 2180224 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_cooldown_trigger - Initial livenodes: [127.0.0.1:10006_solr, 127.0.0.1:10007_solr] [junit4] 2> 2180224 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_cooldown_trigger instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestSimTriggerIntegration$TestTriggerAction}], enabled=true} [junit4] 2> 2180224 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers [junit4] 2> 2180224 DEBUG (ScheduledTrigger-9912-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_cooldown_trigger [junit4] 2> 2180224 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3 [junit4] 2> 2180224 DEBUG (ScheduledTrigger-9912-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2 [junit4] 2> 2180224 DEBUG (simCloudManagerPool-9911-thread-4) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration [junit4] 2> 2180224 DEBUG (simCloudManagerPool-9911-thread-4) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4 [junit4] 2> 2180225 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4 [junit4] 2> 2180225 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers [junit4] 2> 2180225 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4 [junit4] 2> 2180230 DEBUG (TEST-TestSimTriggerIntegration.testCooldown-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.s.SimClusterStateProvider -- creating marker: /autoscaling/nodeAdded/127.0.0.1:10008_solr [junit4] 2> 2180242 DEBUG (ScheduledTrigger-9912-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 2 [junit4] 2> 2180244 DEBUG (ScheduledTrigger-9912-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_cooldown_trigger [junit4] 2> 2180244 DEBUG (ScheduledTrigger-9912-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 2180244 DEBUG (ScheduledTrigger-9912-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10008_solr at time 719257564039235 [junit4] 2> 2180262 DEBUG (ScheduledTrigger-9912-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3 [junit4] 2> 2180265 DEBUG (ScheduledTrigger-9912-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_cooldown_trigger [junit4] 2> 2180265 DEBUG (ScheduledTrigger-9912-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 2180265 DEBUG (ScheduledTrigger-9912-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_cooldown_trigger firing registered processor for nodes: [127.0.0.1:10008_solr] added at times [719257564039235], now=719258579927185 [junit4] 2> 2180265 DEBUG (ScheduledTrigger-9912-thread-1) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: { [junit4] 2> "id":"28e29339b4843Tarpu8w4c1qhsr5p4grq3a1wp9", [junit4] 2> "source":"node_added_cooldown_trigger", [junit4] 2> "eventTime":719257564039235, [junit4] 2> "eventType":"NODEADDED", [junit4] 2> "properties":{ [junit4] 2> "eventTimes":[719257564039235], [junit4] 2> "preferredOperation":"movereplica", [junit4] 2> "nodeNames":["127.0.0.1:10008_solr"]}} [junit4] 2> 2180265 DEBUG (ScheduledTrigger-9912-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_cooldown_trigger] [junit4] 2> 2180265 DEBUG (ScheduledTrigger-9912-thread-1) [ ] o.a.s.c.a.SystemLogListener Collection .system missing, skip sending event { [junit4] 2> "id":"28e29339b4843Tarpu8w4c1qhsr5p4grq3a1wp9", [junit4] 2> "source":"node_added_cooldown_trigger", [junit4] 2> "eventTime":719257564039235, [junit4] 2> "eventType":"NODEADDED", [junit4] 2> "properties":{ [junit4] 2> "eventTimes":[719257564039235], [junit4] 2> "preferredOperation":"movereplica", [junit4] 2> "_enqueue_time_":719258598554585, [junit4] 2> "nodeNames":["127.0.0.1:10008_solr"]}} [junit4] 2> 2180265 DEBUG (ScheduledTrigger-9912-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Removing new node from tracking: 127.0.0.1:10008_solr [junit4] 2> 2180266 DEBUG (AutoscalingActionExecutor-9913-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for { [junit4] 2> "id":"28e29339b4843Tarpu8w4c1qhsr5p4grq3a1wp9", [junit4] 2> "source":"node_added_cooldown_trigger", [junit4] 2> "eventTime":719257564039235, [junit4] 2> "eventType":"NODEADDED", [junit4] 2> "properties":{ [junit4] 2> "eventTimes":[719257564039235], [junit4] 2> "preferredOperation":"movereplica", [junit4] 2> "_enqueue_time_":719258598554585, [junit4] 2> "nodeNames":["127.0.0.1:10008_solr"]}} [junit4] 2> 2180266 DEBUG (AutoscalingActionExecutor-9913-thread-1) [ ] o.a.s.c.a.SystemLogListener Collection .system missing, skip sending event { [junit4] 2> "id":"28e29339b4843Tarpu8w4c1qhsr5p4grq3a1wp9", [junit4] 2> "source":"node_added_cooldown_trigger", [junit4] 2> "eventTime":719257564039235, [junit4] 2> "eventType":"NODEADDED", [junit4] 2> "properties":{ [junit4] 2> "eventTimes":[719257564039235], [junit4] 2> "preferredOperation":"movereplica", [junit4] 2> "_enqueue_time_":719258598554585, [junit4] 2> "nodeNames":["127.0.0.1:10008_solr"]}} [junit4] 2> 2180286 DEBUG (AutoscalingActionExecutor-9913-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_cooldown_trigger after 100ms [junit4] 2> 2180286 DEBUG (AutoscalingActionExecutor-9913-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms [junit4] 2> 2180286 DEBUG (AutoscalingActionExecutor-9913-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 1008 ms for event id=28e29339b4843Tarpu8w4c1qhsr5p4grq3a1wp9 [junit4] 2> 2180286 DEBUG (TEST-TestSimTriggerIntegration.testCooldown-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.s.SimClusterStateProvider -- creating marker: /autoscaling/nodeAdded/127.0.0.1:10009_solr [junit4] 2> 2180386 DEBUG (ScheduledTrigger-9912-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_cooldown_trigger [junit4] 2> 2180386 DEBUG (ScheduledTrigger-9912-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4 [junit4] 2> 2180386 DEBUG (ScheduledTrigger-9912-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4 and last live nodes: 3 [junit4] 2> 2180386 DEBUG (ScheduledTrigger-9912-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10009_solr at time 719264638885035 [junit4] 2> 2180406 DEBUG (ScheduledTrigger-9912-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_cooldown_trigger [junit4] 2> 2180406 DEBUG (ScheduledTrigger-9912-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4 and last live nodes: 4 [junit4] 2> 2180406 DEBUG (ScheduledTrigger-9912-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4 [junit4] 2> 2180406 DEBUG (ScheduledTrigger-9912-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_cooldown_trigger firing registered processor for nodes: [127.0.0.1:10009_solr] added at times [719264638885035], now=719265656403185 [junit4] 2> 2180406 DEBUG (ScheduledTrigger-9912-thread-3) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: { [junit4] 2> "id":"28e2ad94cdcabTarpu8w4c1qhsr5p4grq3a1wpa", [junit4] 2> "source":"node_added_cooldown_trigger", [junit4] 2> "eventTime":719264638885035, [junit4] 2> "eventType":"NODEADDED", [junit4] 2> "properties":{ [junit4] 2> "eventTimes":[719264638885035], [junit4] 2> "preferredOperation":"movereplica", [junit4] 2> "nodeNames":["127.0.0.1:10009_solr"]}} [junit4] 2> 2180407 DEBUG (ScheduledTrigger-9912-thread-3) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_cooldown_trigger] [junit4] 2> 2180407 DEBUG (ScheduledTrigger-9912-thread-3) [ ] o.a.s.c.a.SystemLogListener Collection .system missing, skip sending event { [junit4] 2> "id":"28e2ad94cdcabTarpu8w4c1qhsr5p4grq3a1wpa", [junit4] 2> "source":"node_added_cooldown_trigger", [junit4] 2> "eventTime":719264638885035, [junit4] 2> "eventType":"NODEADDED", [junit4] 2> "properties":{ [junit4] 2> "eventTimes":[719264638885035], [junit4] 2> "preferredOperation":"movereplica", [junit4] 2> "_enqueue_time_":719265669034235, [junit4] 2> "nodeNames":["127.0.0.1:10009_solr"]}} [junit4] 2> 2180407 DEBUG (ScheduledTrigger-9912-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Removing new node from tracking: 127.0.0.1:10009_solr [junit4] 2> 2180407 DEBUG (AutoscalingActionExecutor-9913-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for { [junit4] 2> "id":"28e2ad94cdcabTarpu8w4c1qhsr5p4grq3a1wpa", [junit4] 2> "source":"node_added_cooldown_trigger", [junit4] 2> "eventTime":719264638885035, [junit4] 2> "eventType":"NODEADDED", [junit4] 2> "properties":{ [junit4] 2> "eventTimes":[719264638885035], [junit4] 2> "preferredOperation":"movereplica", [junit4] 2> "_enqueue_time_":719265669034235, [junit4] 2> "nodeNames":["127.0.0.1:10009_solr"]}} [junit4] 2> 2180407 DEBUG (AutoscalingActionExecutor-9913-thread-1) [ ] o.a.s.c.a.SystemLogListener Collection .system missing, skip sending event { [junit4] 2> "id":"28e2ad94cdcabTarpu8w4c1qhsr5p4grq3a1wpa", [junit4] 2> "source":"node_added_cooldown_trigger", [junit4] 2> "eventTime":719264638885035, [junit4] 2> "eventType":"NODEADDED", [junit4] 2> "properties":{ [junit4] 2> "eventTimes":[719264638885035], [junit4] 2> "preferredOperation":"movereplica", [junit4] 2> "_enqueue_time_":719265669034235, [junit4] 2> "nodeNames":["127.0.0.1:10009_solr"]}} [junit4] 2> 2180407 DEBUG (AutoscalingActionExecutor-9913-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_cooldown_trigger after 100ms [junit4] 2> 2180407 DEBUG (AutoscalingActionExecutor-9913-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms [junit4] 2> 2180407 DEBUG (AutoscalingActionExecutor-9913-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 14 ms for event id=28e2ad94cdcabTarpu8w4c1qhsr5p4grq3a1wpa [junit4] 2> 2180408 DEBUG (TEST-TestSimTriggerIntegration.testCooldown-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now [junit4] 2> 2180408 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting. [junit4] 2> 2180408 DEBUG (TEST-TestSimTriggerIntegration.testCooldown-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now [junit4] 2> 2180408 DEBUG (TEST-TestSimTriggerIntegration.testCooldown-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor [junit4] 2> 2180408 DEBUG (TEST-TestSimTriggerIntegration.testCooldown-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor [junit4] 2> 2180408 DEBUG (TEST-TestSimTriggerIntegration.testCooldown-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely [junit4] 2> 2180408 DEBUG (TEST-TestSimTriggerIntegration.testCooldown-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly [junit4] 2> 2180409 INFO (TEST-TestSimTriggerIntegration.testCooldown-seed#[A13AF2A7B45074BC]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testCooldown [junit4] 2> 2180411 INFO (TEST-TestSimTriggerIntegration.testEventQueue-seed#[A13AF2A7B45074BC]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testEventQueue [junit4] 2> 2180414 DEBUG (TEST-TestSimTriggerIntegration.testEventQueue-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10010_solr [junit4] 2> 2180414 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers [junit4] 2> 2180414 DEBUG (TEST-TestSimTriggerIntegration.testEventQueue-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=0 [junit4] 2> 2180414 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1 [junit4] 2> 2180414 DEBUG (TEST-TestSimTriggerIntegration.testEventQueue-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 0 [junit4] 2> 2180414 INFO (TEST-TestSimTriggerIntegration.testEventQueue-seed#[A13AF2A7B45074BC]) [ ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics hi [...truncated too long message...] "preferredOperation":"movereplica", [junit4] 2> "_enqueue_time_":719345483505435, [junit4] 2> "nodeNames":["127.0.0.1:10029_solr"]}} [junit4] 2> 2182006 DEBUG (AutoscalingActionExecutor-9961-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for { [junit4] 2> "id":"28e3d2ca1b6b7Tarpu8w4c1qhsr5p4grq3a1wpq", [junit4] 2> "source":"node_lost_trigger", [junit4] 2> "eventTime":719343346366135, [junit4] 2> "eventType":"NODELOST", [junit4] 2> "properties":{ [junit4] 2> "eventTimes":[719343346366135], [junit4] 2> "preferredOperation":"movereplica", [junit4] 2> "_enqueue_time_":719345483505435, [junit4] 2> "nodeNames":["127.0.0.1:10029_solr"]}} [junit4] 2> 2182006 DEBUG (AutoscalingActionExecutor-9961-thread-1) [ ] o.a.s.c.a.SystemLogListener Collection .system missing, skip sending event { [junit4] 2> "id":"28e3d2ca1b6b7Tarpu8w4c1qhsr5p4grq3a1wpq", [junit4] 2> "source":"node_lost_trigger", [junit4] 2> "eventTime":719343346366135, [junit4] 2> "eventType":"NODELOST", [junit4] 2> "properties":{ [junit4] 2> "eventTimes":[719343346366135], [junit4] 2> "preferredOperation":"movereplica", [junit4] 2> "_enqueue_time_":719345483505435, [junit4] 2> "nodeNames":["127.0.0.1:10029_solr"]}} [junit4] 2> 2182006 DEBUG (AutoscalingActionExecutor-9961-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger after 100ms [junit4] 2> 2182006 DEBUG (AutoscalingActionExecutor-9961-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms [junit4] 2> 2182006 DEBUG (AutoscalingActionExecutor-9961-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 23 ms for event id=28e3d2ca1b6b7Tarpu8w4c1qhsr5p4grq3a1wpq [junit4] 2> 2182007 DEBUG (simCloudManagerPool-9959-thread-4) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration [junit4] 2> 2182007 DEBUG (simCloudManagerPool-9959-thread-4) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4 [junit4] 2> 2182008 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4 [junit4] 2> 2182008 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers [junit4] 2> 2182008 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4 [junit4] 2> 2182053 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting. [junit4] 2> 2182053 DEBUG (TEST-TestSimTriggerIntegration.testNodeLostTrigger-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now [junit4] 2> 2182054 DEBUG (TEST-TestSimTriggerIntegration.testNodeLostTrigger-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now [junit4] 2> 2182054 DEBUG (TEST-TestSimTriggerIntegration.testNodeLostTrigger-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor [junit4] 2> 2182054 DEBUG (TEST-TestSimTriggerIntegration.testNodeLostTrigger-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor [junit4] 2> 2182054 DEBUG (TEST-TestSimTriggerIntegration.testNodeLostTrigger-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely [junit4] 2> 2182054 DEBUG (TEST-TestSimTriggerIntegration.testNodeLostTrigger-seed#[A13AF2A7B45074BC]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly [junit4] 2> 2182055 INFO (TEST-TestSimTriggerIntegration.testNodeLostTrigger-seed#[A13AF2A7B45074BC]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testNodeLostTrigger [junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.sim.TestSimTriggerIntegration_A13AF2A7B45074BC-001 [junit4] 2> NOTE: test params are: codec=FastDecompressionCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST_DECOMPRESSION, chunkSize=21482, maxDocsPerChunk=7, blockSize=149), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST_DECOMPRESSION, chunkSize=21482, blockSize=149)), sim=RandomSimilarity(queryNorm=false): {}, locale=et, timezone=Pacific/Funafuti [junit4] 2> NOTE: Linux 4.18.0-15-generic amd64/Aleksey Shipilev 12-testing (64-bit)/cpus=8,threads=1,free=240275976,total=519045120 [junit4] 2> NOTE: All tests run in this JVM: [TestJsonFacetsStatsParsing, LegacyCloudClusterPropTest, OpenCloseCoreStressTest, TestFieldTypeCollectionResource, CacheHeaderTest, TestSolrConfigHandlerCloud, ChaosMonkeyNothingIsSafeTest, TestLRUStatsCache, TestRangeQuery, TestDistribIDF, TestDirectoryFactory, TestDistributedMap, UniqFieldsUpdateProcessorFactoryTest, TestSubQueryTransformer, TestLFUCache, TestRawResponseWriter, TestGroupingSearch, CoreAdminRequestStatusTest, V2ApiIntegrationTest, TestPayloadScoreQParserPlugin, TestSolr4Spatial2, Tagger2Test, TestConfigSetProperties, FileUtilsTest, TestEmbeddedSolrServerConstructors, TestMaxScoreQueryParser, TestJmxIntegration, RuleEngineTest, TestConfig, SolrGangliaReporterTest, OpenExchangeRatesOrgProviderTest, TestOmitPositions, BitVectorTest, SuggesterTest, NodeMutatorTest, TestHashQParserPlugin, DistribDocExpirationUpdateProcessorTest, TestPullReplicaErrorHandling, BadIndexSchemaTest, TestComplexPhraseQParserPlugin, SpatialRPTFieldTypeTest, TestImpersonationWithHadoopAuth, LeaderFailureAfterFreshStartTest, BlobRepositoryCloudTest, PeerSyncReplicationTest, SolrIndexMetricsTest, TestSimClusterStateProvider, NoCacheHeaderTest, ReplicaListTransformerTest, TestBackupRepositoryFactory, CreateCollectionCleanupTest, DistributedSuggestComponentTest, TestBulkSchemaConcurrent, CollectionReloadTest, TestRuleBasedAuthorizationPlugin, DateRangeFieldTest, CopyFieldTest, TestReplicationHandler, StatsReloadRaceTest, CollectionsAPIDistributedZkTest, HdfsChaosMonkeySafeLeaderTest, DocValuesTest, HdfsCollectionsAPIDistributedZkTest, DeleteShardTest, DistributedFacetPivotLongTailTest, DeleteLastCustomShardedReplicaTest, OverseerCollectionConfigSetProcessorTest, IndexBasedSpellCheckerTest, ParsingFieldUpdateProcessorsTest, TestTolerantUpdateProcessorRandomCloud, TestSimpleQParserPlugin, TestSortByMinMaxFunction, BufferStoreTest, ChaosMonkeySafeLeaderWithPullReplicasTest, UnloadDistributedZkTest, TestRebalanceLeaders, TestAnalyzedSuggestions, OverseerStatusTest, TestManagedResource, InfixSuggestersTest, JavabinLoaderTest, CollectionPropsTest, TestCustomDocTransformer, TestSystemIdResolver, TolerantUpdateProcessorTest, TestSolrQueryResponse, BinaryUpdateRequestHandlerTest, HttpTriggerListenerTest, DistributedFacetPivotWhiteBoxTest, TestSchemalessBufferedUpdates, MultiTermTest, TestCollectionAPIs, TestPivotHelperCode, TestLegacyTerms, TestNRTOpen, LeaderElectionIntegrationTest, TestClassicSimilarityFactory, BaseCdcrDistributedZkTest, DataDrivenBlockJoinTest, TestStandardQParsers, DistributedVersionInfoTest, ZkStateReaderTest, AliasIntegrationTest, FieldMutatingUpdateProcessorTest, HdfsWriteToMultipleCollectionsTest, WrapperMergePolicyFactoryTest, TestManagedSynonymFilterFactory, TestRestoreCore, TestNamedUpdateProcessors, MetricsHandlerTest, MissingSegmentRecoveryTest, TestCloudRecovery, TimeZoneUtilsTest, JvmMetricsTest, GraphQueryTest, ZkControllerTest, TestAddFieldRealTimeGet, TokenizerChainTest, UpdateParamsTest, DirectoryFactoryTest, FieldAnalysisRequestHandlerTest, AtomicUpdateProcessorFactoryTest, StatelessScriptUpdateProcessorFactoryTest, PathHierarchyTokenizerFactoryTest, ComputePlanActionTest, TestCoreDiscovery, TestCollectionAPI, TestSolrJ, SignificantTermsQParserPluginTest, TestLuceneMatchVersion, ReplicationFactorTest, CloudExitableDirectoryReaderTest, QueryParsingTest, TestZkChroot, TestScoreJoinQPNoScore, RemoteQueryErrorTest, TestNoOpRegenerator, TestHashPartitioner, TestLeaderElectionWithEmptyReplica, TestRandomRequestDistribution, TestShortCircuitedRequests, TestSizeLimitedDistributedMap, TestSkipOverseerOperations, TestStressLiveNodes, TestWithCollection, ShardSplitTest, TestSimGenericDistributedQueue, TestSimTriggerIntegration] [junit4] Completed [535/838 (1!)] on J0 in 4.76s, 11 tests, 1 failure, 1 skipped <<< FAILURES! [...truncated 43418 lines...] [repro] Jenkins log URL: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/3582/consoleText [repro] Revision: 4be095e69823451f094152d8e2142605418caf4e [repro] Ant options: "-Dargs=-XX:+UseCompressedOops -XX:+UseParallelGC" [repro] Repro line: ant test -Dtestcase=TestSimTriggerIntegration -Dtests.method=testSearchRate -Dtests.seed=A13AF2A7B45074BC -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=et -Dtests.timezone=Pacific/Funafuti -Dtests.asserts=true -Dtests.file.encoding=UTF-8 [repro] ant clean [...truncated 6 lines...] [repro] Test suites by module: [repro] solr/core [repro] TestSimTriggerIntegration [repro] ant compile-test [...truncated 3588 lines...] [repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=5 -Dtests.class="*.TestSimTriggerIntegration" -Dtests.showOutput=onerror "-Dargs=-XX:+UseCompressedOops -XX:+UseParallelGC" -Dtests.seed=A13AF2A7B45074BC -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=et -Dtests.timezone=Pacific/Funafuti -Dtests.asserts=true -Dtests.file.encoding=UTF-8 [...truncated 125 lines...] [repro] Failures: [repro] 0/5 failed: org.apache.solr.cloud.autoscaling.sim.TestSimTriggerIntegration [repro] Exiting with code 0 [...truncated 43 lines...]
--------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org