Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/1124/
1 tests failed.
FAILED:
org.apache.solr.cloud.autoscaling.sim.TestSimTriggerIntegration.testNodeLostTriggerRestoreState
Error Message:
The trigger did not fire at all
Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
at
__randomizedtesting.SeedInfo.seed([5F0FE7B4669AB7C0:74F032EFFCE2A210]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at
org.apache.solr.cloud.autoscaling.sim.TestSimTriggerIntegration.testNodeLostTriggerRestoreState(TestSimTriggerIntegration.java:340)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 13185 lines...]
[junit4] Suite:
org.apache.solr.cloud.autoscaling.sim.TestSimTriggerIntegration
[junit4] 2> Creating dataDir:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.sim.TestSimTriggerIntegration_5F0FE7B4669AB7C0-001/init-core-data-001
[junit4] IGNOR/A 0.00s J2 | TestSimTriggerIntegration.testEventQueue
[junit4] > Assumption #1: 'awaitsfix' test group is disabled
(@AwaitsFix(bugUrl=https://issues.apache.org/jira/browse/SOLR-12028))
[junit4] IGNOR/A 0.00s J2 | TestSimTriggerIntegration.testCooldown
[junit4] > Assumption #1: 'badapple' test group is disabled
(@BadApple(bugUrl=https://issues.apache.org/jira/browse/SOLR-12028))
[junit4] IGNOR/A 0.00s J2 |
TestSimTriggerIntegration.testNodeMarkersRegistration
[junit4] > Assumption #1: 'badapple' test group is disabled
(@BadApple(bugUrl=https://issues.apache.org/jira/browse/SOLR-12028))
[junit4] IGNOR/A 0.00s J2 | TestSimTriggerIntegration.testNodeAddedTrigger
[junit4] > Assumption #1: 'badapple' test group is disabled
(@BadApple(bugUrl=https://issues.apache.org/jira/browse/SOLR-12028))
[junit4] IGNOR/A 0.00s J2 | TestSimTriggerIntegration.testTriggerThrottling
[junit4] > Assumption #1: 'badapple' test group is disabled
(@BadApple(bugUrl=https://issues.apache.org/jira/browse/SOLR-12028))
[junit4] 2> 1169068 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and
.scheduled_maintenance triggers
[junit4] 2> 1169068 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode
version 0
[junit4] 2> 1169068 DEBUG
(TEST-TestSimTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[5F0FE7B4669AB7C0])
[ ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader:
127.0.0.1:10000_solr
[junit4] 2> 1169068 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion -1
[junit4] 2> 1169068 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 0
[junit4] 2> 1169069 DEBUG
(TEST-TestSimTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[5F0FE7B4669AB7C0])
[ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states,
currentVersion=0
[junit4] 2> 1169071 DEBUG
(TEST-TestSimTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[5F0FE7B4669AB7C0])
[ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 0
[junit4] 2> 1169071 INFO
(TEST-TestSimTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[5F0FE7B4669AB7C0])
[ ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics
history in memory.
[junit4] 2> 1169073 INFO (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial
livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
[junit4] 2> 1169074 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1169074 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion 0
[junit4] 2> 1169074 DEBUG (ScheduledTrigger-2653-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169079 DEBUG (simCloudManagerPool-2652-thread-1) [ ]
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1169079 DEBUG (simCloudManagerPool-2652-thread-1) [ ]
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version
1
[junit4] 2> 1169080 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 1169081 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1169081 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 1169082 DEBUG (simCloudManagerPool-2652-thread-2) [ ]
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1169082 DEBUG (simCloudManagerPool-2652-thread-2) [ ]
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version
2
[junit4] 2> 1169082 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
[junit4] 2> 1169083 INFO (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.s.TestSimTriggerIntegration TestTriggerAction init
[junit4] 2> 1169083 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_restore_trigger -
Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr,
127.0.0.1:10002_solr]
[junit4] 2> 1169083 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_restore_trigger
instantiated with properties: {event=nodeAdded, waitFor=5, actions=[{name=test,
class=org.apache.solr.cloud.autoscaling.sim.TestSimTriggerIntegration$TestTriggerAction}],
enabled=true}
[junit4] 2> 1169083 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.NodeAddedTrigger Adding node from marker path: 127.0.0.1:10002_solr
[junit4] 2> 1169083 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_restore_trigger -
removing marker path: /autoscaling/nodeAdded/127.0.0.1:10002_solr
[junit4] 2> 1169083 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
[junit4] 2> 1169083 DEBUG (ScheduledTrigger-2653-thread-2) [ ]
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_restore_trigger
[junit4] 2> 1169083 DEBUG (ScheduledTrigger-2653-thread-2) [ ]
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 1169095 DEBUG (ScheduledTrigger-2653-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 3 and last live nodes: 2
[junit4] 2> 1169104 DEBUG (ScheduledTrigger-2653-thread-1) [ ]
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_restore_trigger
[junit4] 2> 1169104 DEBUG (ScheduledTrigger-2653-thread-1) [ ]
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 1169114 DEBUG (simCloudManagerPool-2652-thread-3) [ ]
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1169114 DEBUG (simCloudManagerPool-2652-thread-3) [ ]
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version
3
[junit4] 2> 1169114 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
[junit4] 2> 1169114 INFO (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.s.TestSimTriggerIntegration TestTriggerAction init
[junit4] 2> 1169114 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_restore_trigger -
Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr,
127.0.0.1:10002_solr]
[junit4] 2> 1169114 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_restore_trigger
instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test,
class=org.apache.solr.cloud.autoscaling.sim.TestSimTriggerIntegration$TestTriggerAction}],
enabled=true}
[junit4] 2> 1169115 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
[junit4] 2> 1169115 DEBUG (ScheduledTrigger-2653-thread-3) [ ]
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_restore_trigger
[junit4] 2> 1169115 DEBUG (ScheduledTrigger-2653-thread-3) [ ]
o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 1169115 DEBUG (ScheduledTrigger-2653-thread-3) [ ]
o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_restore_trigger firing
registered processor for nodes: [127.0.0.1:10002_solr] added at times
[27066532294902797], now=27066533886797847
[junit4] 2> 1169116 DEBUG (ScheduledTrigger-2653-thread-4) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 3 and last live nodes: 3
[junit4] 2> 1169117 DEBUG (ScheduledTrigger-2653-thread-3) [ ]
o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"6028de22bd440dTcxtwu5iqr5vzilyjacucopgcm",
[junit4] 2> "source":"node_added_restore_trigger",
[junit4] 2> "eventTime":27066532294902797,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[27066532294902797],
[junit4] 2> "preferredOperation":"movereplica",
[junit4] 2> "nodeNames":["127.0.0.1:10002_solr"]}}
[junit4] 2> 1169117 DEBUG (ScheduledTrigger-2653-thread-3) [ ]
o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas,
node_added_restore_trigger]
[junit4] 2> 1169119 DEBUG (simCloudManagerPool-2652-thread-4) [ ]
o.a.s.c.a.s.SimClusterStateProvider -- simCreateCollection .system,
currentVersion=1
[junit4] 2> 1169119 DEBUG (simCloudManagerPool-2652-thread-4) [ ]
o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states,
currentVersion=1
[junit4] 2> 1169119 DEBUG (simCloudManagerPool-2652-thread-4) [ ]
o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 1
[junit4] 2> 1169132 DEBUG (simCloudManagerPool-2652-thread-4) [ ]
o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states,
currentVersion=2
[junit4] 2> 1169135 DEBUG (simCloudManagerPool-2652-thread-4) [ ]
o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 2
[junit4] 2> 1169150 DEBUG (simCloudManagerPool-2652-thread-5) [ ]
o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states,
currentVersion=3
[junit4] 2> 1169151 DEBUG (simCloudManagerPool-2652-thread-5) [ ]
o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 3
[junit4] 2> 1169163 DEBUG (simCloudManagerPool-2652-thread-6) [ ]
o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states,
currentVersion=4
[junit4] 2> 1169163 DEBUG (simCloudManagerPool-2652-thread-6) [ ]
o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 4
[junit4] 2> 1169174 DEBUG (simCloudManagerPool-2652-thread-7) [ ]
o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states,
currentVersion=5
[junit4] 2> 1169174 DEBUG (simCloudManagerPool-2652-thread-7) [ ]
o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 5
[junit4] 2> 1169176 DEBUG (simCloudManagerPool-2652-thread-8) [ ]
o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for .system / shard1
(currentVersion=6): {"core_node1":{
[junit4] 2> "core":".system_shard1_replica_n1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":".system",
[junit4] 2> "node_name":"127.0.0.1:10002_solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "state":"active",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0}}
[junit4] 2> 1169178 DEBUG (simCloudManagerPool-2652-thread-4) [ ]
o.a.s.c.a.s.SimClusterStateProvider -- finished createCollection .system,
currentVersion=6
[junit4] 2> 1169182 DEBUG (simCloudManagerPool-2652-thread-4) [ ]
o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states,
currentVersion=6
[junit4] 2> 1169182 DEBUG (simCloudManagerPool-2652-thread-4) [ ]
o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 6
[junit4] 2> 1169188 DEBUG (ScheduledTrigger-2653-thread-3) [ ]
o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_restore_trigger -
removing marker path: /autoscaling/nodeAdded/127.0.0.1:10002_solr
[junit4] 2> 1169192 DEBUG (AutoscalingActionExecutor-2654-thread-1) [ ]
o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"6028de22bd440dTcxtwu5iqr5vzilyjacucopgcm",
[junit4] 2> "source":"node_added_restore_trigger",
[junit4] 2> "eventTime":27066532294902797,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[27066532294902797],
[junit4] 2> "preferredOperation":"movereplica",
[junit4] 2> "_enqueue_time_":27066534034847997,
[junit4] 2> "nodeNames":["127.0.0.1:10002_solr"]}}
[junit4] 2> 1169194 INFO (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are
closed, exiting.
[junit4] 2> 1169194 DEBUG
(TEST-TestSimTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[5F0FE7B4669AB7C0])
[ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool
executor now
[junit4] 2> 1169195 DEBUG
(TEST-TestSimTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[5F0FE7B4669AB7C0])
[ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 1169195 DEBUG
(TEST-TestSimTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[5F0FE7B4669AB7C0])
[ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 1169195 WARN (AutoscalingActionExecutor-2654-thread-1) [ ]
o.a.s.c.a.SystemLogListener Exception sending event to collection .system
[junit4] 2> java.io.IOException: java.lang.InterruptedException
[junit4] 2> at
org.apache.solr.cloud.autoscaling.sim.SimCloudManager.request(SimCloudManager.java:654)
~[test/:?]
[junit4] 2> at
org.apache.solr.cloud.autoscaling.SystemLogListener.onEvent(SystemLogListener.java:118)
~[java/:?]
[junit4] 2> at
org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerListeners.fireListeners(ScheduledTriggers.java:802)
~[java/:?]
[junit4] 2> at
org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerListeners.fireListeners(ScheduledTriggers.java:769)
~[java/:?]
[junit4] 2> at
org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:326)
~[java/:?]
[junit4] 2> at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
~[?:1.8.0_191]
[junit4] 2> at
java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_191]
[junit4] 2> at
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
~[java/:?]
[junit4] 2> at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[?:1.8.0_191]
[junit4] 2> at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[?:1.8.0_191]
[junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_191]
[junit4] 2> Caused by: java.lang.InterruptedException
[junit4] 2> at
java.util.concurrent.FutureTask.awaitDone(FutureTask.java:404) ~[?:1.8.0_191]
[junit4] 2> at
java.util.concurrent.FutureTask.get(FutureTask.java:191) ~[?:1.8.0_191]
[junit4] 2> at
org.apache.solr.cloud.autoscaling.sim.SimCloudManager.request(SimCloudManager.java:652)
~[test/:?]
[junit4] 2> ... 10 more
[junit4] 2> 1169196 WARN (AutoscalingActionExecutor-2654-thread-1) [ ]
o.a.s.c.a.ScheduledTriggers Unhandled exception executing actions
[junit4] 2> org.apache.solr.common.AlreadyClosedException:
ScheduledTrigger node_added_restore_trigger has been closed.
[junit4] 2> at
org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerWrapper.dequeue(ScheduledTriggers.java:549)
~[java/:?]
[junit4] 2> at
org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:319)
~[java/:?]
[junit4] 2> at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
~[?:1.8.0_191]
[junit4] 2> at
java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_191]
[junit4] 2> at
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
~[java/:?]
[junit4] 2> at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[?:1.8.0_191]
[junit4] 2> at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[?:1.8.0_191]
[junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_191]
[junit4] 2> 1169197 DEBUG (AutoscalingActionExecutor-2654-thread-1) [ ]
o.a.s.c.a.ScheduledTriggers -- processing took 275 ms for event
id=6028de22bd440dTcxtwu5iqr5vzilyjacucopgcm
[junit4] 2> 1169197 DEBUG
(TEST-TestSimTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[5F0FE7B4669AB7C0])
[ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread
pool executor
[junit4] 2> 1169197 DEBUG
(TEST-TestSimTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[5F0FE7B4669AB7C0])
[ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 1169197 DEBUG
(TEST-TestSimTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[5F0FE7B4669AB7C0])
[ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed
explicitly
[junit4] IGNOR/A 0.00s J2 | TestSimTriggerIntegration.testNodeLostTrigger
[junit4] > Assumption #1: 'awaitsfix' test group is disabled
(@AwaitsFix(bugUrl=https://issues.apache.org/jira/browse/SOLR-12028))
[junit4] 2> 1169206 DEBUG
(TEST-TestSimTriggerIntegration.testNodeLostTriggerRestoreState-seed#[5F0FE7B4669AB7C0])
[ ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader:
127.0.0.1:10003_solr
[junit4] 2> 1169206 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and
.scheduled_maintenance triggers
[junit4] 2> 1169206 DEBUG
(TEST-TestSimTriggerIntegration.testNodeLostTriggerRestoreState-seed#[5F0FE7B4669AB7C0])
[ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states,
currentVersion=0
[junit4] 2> 1169206 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode
version 0
[junit4] 2> 1169206 DEBUG
(TEST-TestSimTriggerIntegration.testNodeLostTriggerRestoreState-seed#[5F0FE7B4669AB7C0])
[ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 0
[junit4] 2> 1169206 INFO
(TEST-TestSimTriggerIntegration.testNodeLostTriggerRestoreState-seed#[5F0FE7B4669AB7C0])
[ ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics
history in memory.
[junit4] 2> 1169206 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion -1
[junit4] 2> 1169207 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 0
[junit4] 2> 1169207 DEBUG (simCloudManagerPool-2658-thread-1) [ ]
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1169207 DEBUG (simCloudManagerPool-2658-thread-1) [ ]
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version
1
[junit4] 2> 1169221 DEBUG (simCloudManagerPool-2658-thread-2) [ ]
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1169221 DEBUG (simCloudManagerPool-2658-thread-2) [ ]
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version
2
[junit4] 2> 1169227 INFO (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial
livenodes: [127.0.0.1:10004_solr, 127.0.0.1:10003_solr]
[junit4] 2> 1169227 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.NodeLostTrigger Adding lost node from marker path:
127.0.0.1:10005_solr
[junit4] 2> 1169228 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1169228 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 0
[junit4] 2> 1169228 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
[junit4] 2> 1169228 DEBUG (ScheduledTrigger-2659-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169228 INFO (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.s.TestSimTriggerIntegration TestTriggerAction init
[junit4] 2> 1169228 INFO (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.NodeLostTrigger NodeLostTrigger node_lost_restore_trigger -
Initial livenodes: [127.0.0.1:10004_solr, 127.0.0.1:10003_solr]
[junit4] 2> 1169229 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1169229 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
[junit4] 2> 1169236 DEBUG (ScheduledTrigger-2659-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169249 DEBUG (ScheduledTrigger-2659-thread-3) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169254 DEBUG (simCloudManagerPool-2658-thread-3) [ ]
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1169254 DEBUG (simCloudManagerPool-2658-thread-3) [ ]
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version
3
[junit4] 2> 1169255 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
[junit4] 2> 1169255 INFO (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.s.TestSimTriggerIntegration TestTriggerAction init
[junit4] 2> 1169255 INFO (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.NodeLostTrigger NodeLostTrigger node_lost_restore_trigger -
Initial livenodes: [127.0.0.1:10004_solr, 127.0.0.1:10003_solr]
[junit4] 2> 1169255 DEBUG (ScheduledTrigger-2659-thread-4) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169264 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1169264 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
[junit4] 2> 1169269 DEBUG (ScheduledTrigger-2659-thread-4) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169276 DEBUG (ScheduledTrigger-2659-thread-3) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169289 DEBUG (ScheduledTrigger-2659-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169296 DEBUG (ScheduledTrigger-2659-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169309 DEBUG (ScheduledTrigger-2659-thread-4) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169316 DEBUG (ScheduledTrigger-2659-thread-3) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169329 DEBUG (ScheduledTrigger-2659-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169337 DEBUG (ScheduledTrigger-2659-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169350 DEBUG (ScheduledTrigger-2659-thread-4) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169357 DEBUG (ScheduledTrigger-2659-thread-3) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169370 DEBUG (ScheduledTrigger-2659-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169377 DEBUG (ScheduledTrigger-2659-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169390 DEBUG (ScheduledTrigger-2659-thread-4) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169397 DEBUG (ScheduledTrigger-2659-thread-3) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169410 DEBUG (ScheduledTrigger-2659-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169418 DEBUG (ScheduledTrigger-2659-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169430 DEBUG (ScheduledTrigger-2659-thread-4) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169438 DEBUG (ScheduledTrigger-2659-thread-3) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169451 DEBUG (ScheduledTrigger-2659-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169458 DEBUG (ScheduledTrigger-2659-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169471 DEBUG (ScheduledTrigger-2659-thread-4) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169478 DEBUG (ScheduledTrigger-2659-thread-3) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169491 DEBUG (ScheduledTrigger-2659-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169498 DEBUG (ScheduledTrigger-2659-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169511 DEBUG (ScheduledTrigger-2659-thread-4) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169519 DEBUG (ScheduledTrigger-2659-thread-3) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169532 DEBUG (ScheduledTrigger-2659-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169539 DEBUG (ScheduledTrigger-2659-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169552 DEBUG (ScheduledTrigger-2659-thread-4) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169559 DEBUG (ScheduledTrigger-2659-thread-3) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169572 DEBUG (ScheduledTrigger-2659-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169579 DEBUG (ScheduledTrigger-2659-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169592 DEBUG (ScheduledTrigger-2659-thread-4) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169599 DEBUG (ScheduledTrigger-2659-thread-3) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169612 DEBUG (ScheduledTrigger-2659-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169620 DEBUG (ScheduledTrigger-2659-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169633 DEBUG (ScheduledTrigger-2659-thread-4) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169640 DEBUG (ScheduledTrigger-2659-thread-3) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169653 DEBUG (ScheduledTrigger-2659-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169660 DEBUG (ScheduledTrigger-2659-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169673 DEBUG (ScheduledTrigger-2659-thread-4) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169680 DEBUG (ScheduledTrigger-2659-thread-3) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169693 DEBUG (ScheduledTrigger-2659-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169700 DEBUG (ScheduledTrigger-2659-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169714 DEBUG (ScheduledTrigger-2659-thread-4) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169721 DEBUG (ScheduledTrigger-2659-thread-3) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169734 DEBUG (ScheduledTrigger-2659-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169741 DEBUG (ScheduledTrigger-2659-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169754 DEBUG (ScheduledTrigger-2659-thread-4) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169761 DEBUG (ScheduledTrigger-2659-thread-3) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169774 DEBUG (ScheduledTrigger-2659-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169781 DEBUG (ScheduledTrigger-2659-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169794 DEBUG (ScheduledTrigger-2659-thread-4) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169801 DEBUG (ScheduledTrigger-2659-thread-3) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169815 DEBUG (ScheduledTrigger-2659-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169822 DEBUG (ScheduledTrigger-2659-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169835 DEBUG (ScheduledTrigger-2659-thread-4) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169842 DEBUG (ScheduledTrigger-2659-thread-3) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169855 DEBUG (ScheduledTrigger-2659-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169862 DEBUG (ScheduledTrigger-2659-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169875 DEBUG (ScheduledTrigger-2659-thread-4) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169882 DEBUG (ScheduledTrigger-2659-thread-3) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169896 DEBUG (ScheduledTrigger-2659-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169902 DEBUG (ScheduledTrigger-2659-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169916 DEBUG (ScheduledTrigger-2659-thread-4) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169923 DEBUG (ScheduledTrigger-2659-thread-3) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169936 DEBUG (ScheduledTrigger-2659-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169943 DEBUG (ScheduledTrigger-2659-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169956 DEBUG (ScheduledTrigger-2659-thread-4) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169963 DEBUG (ScheduledTrigger-2659-thread-3) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169976 DEBUG (ScheduledTrigger-2659-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169983 DEBUG (ScheduledTrigger-2659-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1169997 DEBUG (ScheduledTrigger-2659-thread-4) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170003 DEBUG (ScheduledTrigger-2659-thread-3) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170017 DEBUG (ScheduledTrigger-2659-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170024 DEBUG (ScheduledTrigger-2659-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170037 DEBUG (ScheduledTrigger-2659-thread-4) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170044 DEBUG (ScheduledTrigger-2659-thread-3) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170057 DEBUG (ScheduledTrigger-2659-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170064 DEBUG (ScheduledTrigger-2659-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170077 DEBUG (ScheduledTrigger-2659-thread-4) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170084 DEBUG (ScheduledTrigger-2659-thread-3) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170098 DEBUG (ScheduledTrigger-2659-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170104 DEBUG (ScheduledTrigger-2659-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170118 DEBUG (ScheduledTrigger-2659-thread-4) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170125 DEBUG (ScheduledTrigger-2659-thread-3) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170138 DEBUG (ScheduledTrigger-2659-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170145 DEBUG (ScheduledTrigger-2659-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170158 DEBUG (ScheduledTrigger-2659-thread-4) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170165 DEBUG (ScheduledTrigger-2659-thread-3) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170178 DEBUG (ScheduledTrigger-2659-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170185 DEBUG (ScheduledTrigger-2659-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170199 DEBUG (ScheduledTrigger-2659-thread-4) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170205 DEBUG (ScheduledTrigger-2659-thread-3) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170219 DEBUG (ScheduledTrigger-2659-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170226 DEBUG (ScheduledTrigger-2659-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170239 DEBUG (ScheduledTrigger-2659-thread-4) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170246 DEBUG (ScheduledTrigger-2659-thread-4) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170259 DEBUG (ScheduledTrigger-2659-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170266 DEBUG (ScheduledTrigger-2659-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170280 DEBUG (ScheduledTrigger-2659-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170286 DEBUG (ScheduledTrigger-2659-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170300 DEBUG (ScheduledTrigger-2659-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170306 DEBUG (ScheduledTrigger-2659-thread-3) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170320 DEBUG (ScheduledTrigger-2659-thread-4) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170327 DEBUG (ScheduledTrigger-2659-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170340 DEBUG (ScheduledTrigger-2659-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170347 DEBUG (ScheduledTrigger-2659-thread-3) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170361 DEBUG (ScheduledTrigger-2659-thread-4) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170367 DEBUG (ScheduledTrigger-2659-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170381 DEBUG (ScheduledTrigger-2659-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170387 DEBUG (ScheduledTrigger-2659-thread-3) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170401 DEBUG (ScheduledTrigger-2659-thread-4) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170407 DEBUG (ScheduledTrigger-2659-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170421 DEBUG (ScheduledTrigger-2659-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170428 DEBUG (ScheduledTrigger-2659-thread-3) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170441 DEBUG (ScheduledTrigger-2659-thread-4) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170448 DEBUG (ScheduledTrigger-2659-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170462 DEBUG (ScheduledTrigger-2659-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170468 DEBUG (ScheduledTrigger-2659-thread-3) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170482 DEBUG (ScheduledTrigger-2659-thread-4) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170488 DEBUG (ScheduledTrigger-2659-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170502 DEBUG (ScheduledTrigger-2659-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170508 DEBUG (ScheduledTrigger-2659-thread-3) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170522 DEBUG (ScheduledTrigger-2659-thread-4) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170529 DEBUG (ScheduledTrigger-2659-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170543 DEBUG (ScheduledTrigger-2659-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170549 DEBUG (ScheduledTrigger-2659-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170563 DEBUG (ScheduledTrigger-2659-thread-4) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170569 DEBUG (ScheduledTrigger-2659-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170583 DEBUG (ScheduledTrigger-2659-thread-3) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170589 DEBUG (ScheduledTrigger-2659-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170603 DEBUG (ScheduledTrigger-2659-thread-4) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170610 DEBUG (ScheduledTrigger-2659-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170624 DEBUG (ScheduledTrigger-2659-thread-3) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170630 DEBUG (ScheduledTrigger-2659-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170644 DEBUG (ScheduledTrigger-2659-thread-4) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170650 DEBUG (ScheduledTrigger-2659-thread-4) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170664 DEBUG (ScheduledTrigger-2659-thread-4) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170670 DEBUG (ScheduledTrigger-2659-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170684 DEBUG (ScheduledTrigger-2659-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170690 DEBUG (ScheduledTrigger-2659-thread-3) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170705 DEBUG (ScheduledTrigger-2659-thread-4) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170711 DEBUG (ScheduledTrigger-2659-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170725 DEBUG (ScheduledTrigger-2659-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170731 DEBUG (ScheduledTrigger-2659-thread-3) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170745 DEBUG (ScheduledTrigger-2659-thread-4) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170751 DEBUG (ScheduledTrigger-2659-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170765 DEBUG (ScheduledTrigger-2659-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170771 DEBUG (ScheduledTrigger-2659-thread-3) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170786 DEBUG (ScheduledTrigger-2659-thread-4) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170791 DEBUG (ScheduledTrigger-2659-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170806 DEBUG (ScheduledTrigger-2659-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170812 DEBUG (ScheduledTrigger-2659-thread-3) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170826 DEBUG (ScheduledTrigger-2659-thread-4) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170832 DEBUG (ScheduledTrigger-2659-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170846 DEBUG (ScheduledTrigger-2659-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170852 DEBUG (ScheduledTrigger-2659-thread-3) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170866 DEBUG (ScheduledTrigger-2659-thread-4) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170872 DEBUG (ScheduledTrigger-2659-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170887 DEBUG (ScheduledTrigger-2659-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170893 DEBUG (ScheduledTrigger-2659-thread-3) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170907 DEBUG (ScheduledTrigger-2659-thread-4) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170913 DEBUG (ScheduledTrigger-2659-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170927 DEBUG (ScheduledTrigger-2659-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170933 DEBUG (ScheduledTrigger-2659-thread-3) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170947 DEBUG (ScheduledTrigger-2659-thread-4) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170953 DEBUG (ScheduledTrigger-2659-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170967 DEBUG (ScheduledTrigger-2659-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170973 DEBUG (ScheduledTrigger-2659-thread-3) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170988 DEBUG (ScheduledTrigger-2659-thread-4) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1170994 DEBUG (ScheduledTrigger-2659-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1171008 DEBUG (ScheduledTrigger-2659-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1171014 DEBUG (ScheduledTrigger-2659-thread-3) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1171028 DEBUG (ScheduledTrigger-2659-thread-4) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1171034 DEBUG (ScheduledTrigger-2659-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1171048 DEBUG (ScheduledTrigger-2659-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1171054 DEBUG (ScheduledTrigger-2659-thread-3) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger
with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 1171056 INFO (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are
closed, exiting.
[junit4] 2> 1171057 DEBUG
(TEST-TestSimTriggerIntegration.testNodeLostTriggerRestoreState-seed#[5F0FE7B4669AB7C0])
[ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool
executor now
[junit4] 2> 1171057 DEBUG
(TEST-TestSimTriggerIntegration.testNodeLostTriggerRestoreState-seed#[5F0FE7B4669AB7C0])
[ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 1171057 DEBUG
(TEST-TestSimTriggerIntegration.testNodeLostTriggerRestoreState-seed#[5F0FE7B4669AB7C0])
[ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 1171057 DEBUG
(TEST-TestSimTriggerIntegration.testNodeLostTriggerRestoreState-seed#[5F0FE7B4669AB7C0])
[ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread
pool executor
[junit4] 2> 1171057 DEBUG
(TEST-TestSimTriggerIntegration.testNodeLostTriggerRestoreState-seed#[5F0FE7B4669AB7C0])
[ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 1171057 DEBUG
(TEST-TestSimTriggerIntegration.testNodeLostTriggerRestoreState-seed#[5F0FE7B4669AB7C0])
[ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed
explicitly
[junit4] 2> NOTE: reproduce with: ant test
-Dtestcase=TestSimTriggerIntegration
-Dtests.method=testNodeLostTriggerRestoreState -Dtests.seed=5F0FE7B4669AB7C0
-Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=ar-EG
-Dtests.timezone=Pacific/Chatham -Dtests.asserts=true
-Dtests.file.encoding=UTF-8
[junit4] FAILURE 1.86s J2 |
TestSimTriggerIntegration.testNodeLostTriggerRestoreState <<<
[junit4] > Throwable #1: java.lang.AssertionError: The trigger did not
fire at all
[junit4] > at
__randomizedtesting.SeedInfo.seed([5F0FE7B4669AB7C0:74F032EFFCE2A210]:0)
[junit4] > at
org.apache.solr.cloud.autoscaling.sim.TestSimTriggerIntegration.testNodeLostTriggerRestoreState(TestSimTriggerIntegration.java:340)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] IGNOR/A 0.00s J2 |
TestSimTriggerIntegration.testEventFromRestoredState
[junit4] > Assumption #1: 'badapple' test group is disabled
(@BadApple(bugUrl=https://issues.apache.org/jira/browse/SOLR-12028))
[junit4] IGNOR/A 0.00s J2 | TestSimTriggerIntegration.testSearchRate
[junit4] > Assumption #1: 'awaitsfix' test group is disabled
(@AwaitsFix(bugUrl=https://issues.apache.org/jira/browse/SOLR-12028))
[junit4] IGNOR/A 0.00s J2 | TestSimTriggerIntegration.testListeners
[junit4] > Assumption #1: 'badapple' test group is disabled
(@BadApple(bugUrl=https://issues.apache.org/jira/browse/SOLR-12028))
[junit4] 2> NOTE: leaving temporary files on disk at:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.sim.TestSimTriggerIntegration_5F0FE7B4669AB7C0-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {},
docValues:{}, maxPointsInLeafNode=1730, maxMBSortInHeap=6.837119827969705,
sim=RandomSimilarity(queryNorm=false): {}, locale=ar-EG,
timezone=Pacific/Chatham
[junit4] 2> NOTE: Linux 4.4.0-112-generic amd64/Oracle Corporation
1.8.0_191 (64-bit)/cpus=4,threads=1,free=150864408,total=493355008
[junit4] 2> NOTE: All tests run in this JVM:
[AtomicUpdateProcessorFactoryTest, SliceStateTest, TestSimpleTextCodec,
ResponseLogComponentTest, StatsReloadRaceTest,
TestSolrCloudWithHadoopAuthPlugin, TestCharFilters, TestDynamicURP,
DistributedIntervalFacetingTest, TestCloudPseudoReturnFields,
TestCoreContainer, ScheduledTriggerIntegrationTest, TestXmlQParserPlugin,
TestFieldCacheSort, HDFSCollectionsAPITest, TestStressUserVersions,
TestBlendedInfixSuggestions, SolrIndexSplitterTest, BasicAuthIntegrationTest,
RegexBytesRefFilterTest, QueryEqualityTest, TestSolrConfigHandlerCloud,
ChangedSchemaMergeTest, SolrCmdDistributorTest, BinaryUpdateRequestHandlerTest,
RequiredFieldsTest, ExplicitHLLTest, TestStressInPlaceUpdates,
DirectSolrConnectionTest, ByteBuffersDirectoryFactoryTest,
TestEmbeddedSolrServerAdminHandler, TestHdfsCloudBackupRestore, CursorMarkTest,
UninvertDocValuesMergePolicyTest, LoggingHandlerTest, TestCustomSort,
HdfsChaosMonkeyNothingIsSafeTest, TestManagedSchemaThreadSafety,
TestInitQParser, TestSchemaVersionResource, HttpPartitionTest,
TestRandomRequestDistribution, TestConfigSetsAPIZkFailure, ConvertedLegacyTest,
BlockCacheTest, SolrJmxReporterCloudTest,
PeerSyncWithLeaderAndIndexFingerprintCachingTest, TestReRankQParserPlugin,
NodeAddedTriggerTest, TestWordDelimiterFilterFactory,
SolrMetricsIntegrationTest, TestLMDirichletSimilarityFactory,
TestSha256AuthenticationProvider, SimplePostToolTest, ResourceLoaderTest,
TestRequestStatusCollectionAPI, TestDistributedGrouping,
CollectionsAPIAsyncDistributedZkTest, HdfsSyncSliceTest,
SpellCheckComponentTest, StatelessScriptUpdateProcessorFactoryTest,
CoreAdminRequestStatusTest, TestRecoveryHdfs, TestQueryTypes,
HdfsDirectoryTest, RandomizedTaggerTest, TestJsonFacetRefinement,
TestSimTriggerIntegration]
[junit4] Completed [177/836 (1!)] on J2 in 2.04s, 11 tests, 1 failure, 9
skipped <<< FAILURES!
[...truncated 48418 lines...]
-ecj-javadoc-lint-tests:
[mkdir] Created dir: /tmp/ecj962099505
[ecj-lint] Compiling 935 source files to /tmp/ecj962099505
[ecj-lint] invalid Class-Path header in manifest of jar file:
/home/jenkins/.ivy2/cache/org.restlet.jee/org.restlet/jars/org.restlet-2.3.0.jar
[ecj-lint] invalid Class-Path header in manifest of jar file:
/home/jenkins/.ivy2/cache/org.restlet.jee/org.restlet.ext.servlet/jars/org.restlet.ext.servlet-2.3.0.jar
[ecj-lint] ----------
[ecj-lint] 1. WARNING in
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test/org/apache/solr/analysis/TokenizerChainTest.java
(at line 37)
[ecj-lint] TokenizerChain tokenizerChain = new TokenizerChain(
[ecj-lint] ^^^^^^^^^^^^^^
[ecj-lint] Resource leak: 'tokenizerChain' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 2. WARNING in
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test/org/apache/solr/cloud/ZkNodePropsTest.java
(at line 48)
[ecj-lint] new JavaBinCodec().marshal(zkProps.getProperties(), baos);
[ecj-lint] ^^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: '<unassigned Closeable value>' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 3. WARNING in
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test/org/apache/solr/cloud/autoscaling/sim/SimCloudManager.java
(at line 712)
[ecj-lint] throw new UnsupportedOperationException("must add at least 1
node first");
[ecj-lint]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: 'queryRequest' is not closed at this location
[ecj-lint] ----------
[ecj-lint] 4. WARNING in
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test/org/apache/solr/cloud/autoscaling/sim/SimCloudManager.java
(at line 718)
[ecj-lint] throw new UnsupportedOperationException("must add at least 1
node first");
[ecj-lint]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: 'queryRequest' is not closed at this location
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 5. WARNING in
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test/org/apache/solr/response/TestBinaryResponseWriter.java
(at line 75)
[ecj-lint] new JavaBinCodec(new BinaryResponseWriter.Resolver(null,
null)).marshal(nl, baos);
[ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: '<unassigned Closeable value>' is never closed
[ecj-lint] ----------
[ecj-lint] 6. WARNING in
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test/org/apache/solr/response/TestBinaryResponseWriter.java
(at line 77)
[ecj-lint] nl = (NamedList) new JavaBinCodec().unmarshal(new
ByteArrayInputStream(byteBuffer.array(), 0, byteBuffer.limit()));
[ecj-lint] ^^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: '<unassigned Closeable value>' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 7. ERROR in
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test/org/apache/solr/security/BasicAuthIntegrationTest.java
(at line 72)
[ecj-lint] import static java.nio.charset.StandardCharsets.UTF_8;
[ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] The import java.nio.charset.StandardCharsets.UTF_8 is never used
[ecj-lint] ----------
[ecj-lint] 8. ERROR in
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test/org/apache/solr/security/BasicAuthIntegrationTest.java
(at line 73)
[ecj-lint] import static java.util.Collections.singletonMap;
[ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] The import java.util.Collections.singletonMap is never used
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 9. WARNING in
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test/org/apache/solr/update/processor/DistributedUpdateProcessorTest.java
(at line 39)
[ecj-lint] DistributedUpdateProcessor processor = new
DistributedUpdateProcessor(
[ecj-lint] ^^^^^^^^^
[ecj-lint] Resource leak: 'processor' is never closed
[ecj-lint] ----------
[ecj-lint] 9 problems (2 errors, 7 warnings)
BUILD FAILED
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/build.xml:633: The
following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/build.xml:101: The
following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build.xml:680:
The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/common-build.xml:2099:
The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/common-build.xml:2132:
Compile failed; see the compiler error output for details.
Total time: 187 minutes 2 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
