Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/4367/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseG1GC
1 tests failed.
FAILED:
org.apache.solr.cloud.autoscaling.sim.TestComputePlanAction.testNodeAdded
Error Message:
Stack Trace:
java.util.ConcurrentModificationException
at
__randomizedtesting.SeedInfo.seed([CE91D734C7FAA56D:AB52814365590D6E]:0)
at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:901)
at java.util.ArrayList$Itr.next(ArrayList.java:851)
at
org.apache.solr.cloud.autoscaling.sim.SimSolrCloudTestCase.tearDown(SimSolrCloudTestCase.java:141)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:992)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 11591 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestComputePlanAction
[junit4] 2> 60976 INFO
(SUITE-TestComputePlanAction-seed#[CE91D734C7FAA56D]-worker) [ ]
o.a.s.SolrTestCaseJ4 SecureRandom sanity checks:
test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir:
/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.sim.TestComputePlanAction_CE91D734C7FAA56D-001/init-core-data-001
[junit4] 2> 60977 WARN
(SUITE-TestComputePlanAction-seed#[CE91D734C7FAA56D]-worker) [ ]
o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=22 numCloses=22
[junit4] 2> 60978 INFO
(SUITE-TestComputePlanAction-seed#[CE91D734C7FAA56D]-worker) [ ]
o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true)
w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 60981 INFO
(SUITE-TestComputePlanAction-seed#[CE91D734C7FAA56D]-worker) [ ]
o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via:
@org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
w/ MAC_OS_X supressed clientAuth
[junit4] 2> 61001 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 61001 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode
version 0
[junit4] 2> 61001 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10000_solr]
[junit4] 2> 61001 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion -1
[junit4] 2> 61001 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 0
[junit4] 2> 61006 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 61006 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion 0
[junit4] 2> 61007 INFO
(TEST-TestComputePlanAction.testNodeAdded-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.SolrTestCaseJ4 ###Starting testNodeAdded
[junit4] 2> 61007 DEBUG
(TEST-TestComputePlanAction.testNodeAdded-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version
1
[junit4] 2> 61008 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 61008 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 61009 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 61010 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 61010 WARN (ScheduledTrigger-145-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger NodeLostTrigger ran but was already closed
[junit4] 2> 61010 INFO
(TEST-TestComputePlanAction.testNodeAdded-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing
object cache...
[junit4] 2> 61011 WARN (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are
closed, exiting.
[junit4] 2> 61011 DEBUG
(TEST-TestComputePlanAction.testNodeAdded-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 61012 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 61012 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode
version 2
[junit4] 2> 61012 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10000_solr]
[junit4] 2> 61012 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion -1
[junit4] 2> 61012 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
[junit4] 2> 61012 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 61012 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
[junit4] 2> 61012 DEBUG (ScheduledTrigger-148-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 1
[junit4] 2> 61043 DEBUG (ScheduledTrigger-148-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 1
[junit4] 2> 61074 DEBUG (ScheduledTrigger-148-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 1
[junit4] 2> 61098 DEBUG (ScheduledTrigger-148-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 1
[junit4] 2> 61122 DEBUG (ScheduledTrigger-148-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 1
[junit4] 2> 61144 DEBUG (ScheduledTrigger-148-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 1
[junit4] 2> 61174 DEBUG (ScheduledTrigger-148-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 1
[junit4] 2> 61204 DEBUG (ScheduledTrigger-148-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 1
[junit4] 2> 61212 DEBUG (simCloudManagerPool-147-thread-1) [ ]
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 61213 DEBUG (simCloudManagerPool-147-thread-1) [ ]
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version
3
[junit4] 2> 61213 DEBUG (simCloudManagerPool-147-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10000_solr]
[junit4] 2> 61213 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
[junit4] 2> 61213 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 61213 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
[junit4] 2> 61230 DEBUG (simCloudManagerPool-147-thread-2) [ ]
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 61231 DEBUG (simCloudManagerPool-147-thread-2) [ ]
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version
4
[junit4] 2> 61231 DEBUG (simCloudManagerPool-147-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10000_solr]
[junit4] 2> 61231 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
[junit4] 2> 61231 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 61231 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
[junit4] 2> 61236 DEBUG (ScheduledTrigger-148-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 1
[junit4] 2> 61259 DEBUG (ScheduledTrigger-148-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 1
[junit4] 2> 61279 DEBUG (ScheduledTrigger-148-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 1
[junit4] 2> 61306 DEBUG (ScheduledTrigger-148-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 1
[junit4] 2> 61332 DEBUG (ScheduledTrigger-148-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 1
[junit4] 2> 61353 DEBUG (ScheduledTrigger-148-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 1
[junit4] 2> 61377 DEBUG (ScheduledTrigger-148-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 1
[junit4] 2> 61404 DEBUG (ScheduledTrigger-148-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 1
[junit4] 2> 61405 DEBUG (simCloudManagerPool-147-thread-3) [ ]
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 61405 DEBUG (simCloudManagerPool-147-thread-3) [ ]
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version
5
[junit4] 2> 61405 DEBUG (simCloudManagerPool-147-thread-3) [ ]
o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10000_solr]
[junit4] 2> 61413 DEBUG (simCloudManagerPool-147-thread-3) [ ]
o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:10000_solr]
[junit4] 2> 61413 DEBUG (simCloudManagerPool-147-thread-3) [ ]
o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated
with properties: {event=nodeAdded, waitFor=1, actions=[{name=compute_plan,
class=solr.ComputePlanAction}, {name=test,
class=org.apache.solr.cloud.autoscaling.sim.TestComputePlanAction$AssertingTriggerAction}],
enabled=true}
[junit4] 2> 61413 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
[junit4] 2> 61413 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
[junit4] 2> 61413 DEBUG (ScheduledTrigger-148-thread-2) [ ]
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 61416 DEBUG (ScheduledTrigger-148-thread-2) [ ]
o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
[junit4] 2> 61417 DEBUG (simCloudManagerPool-147-thread-4) [ ]
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 61417 DEBUG (simCloudManagerPool-147-thread-4) [ ]
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version
6
[junit4] 2> 61417 DEBUG (simCloudManagerPool-147-thread-4) [ ]
o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10000_solr]
[junit4] 2> 61417 DEBUG (simCloudManagerPool-147-thread-4) [ ]
o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:10000_solr]
[junit4] 2> 61417 DEBUG (simCloudManagerPool-147-thread-4) [ ]
o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated
with properties: {event=nodeAdded, waitFor=1, actions=[{name=compute_plan,
class=solr.ComputePlanAction}, {name=test,
class=org.apache.solr.cloud.autoscaling.sim.TestComputePlanAction$AssertingTriggerAction}],
enabled=true}
[junit4] 2> 61417 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
[junit4] 2> 61417 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion 6
[junit4] 2> 61427 DEBUG (ScheduledTrigger-148-thread-4) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 1
[junit4] 2> 61429 DEBUG (simCloudManagerPool-147-thread-5) [ ]
o.a.s.c.o.ClusterStateMutator building a new cName: testNodeAdded
[junit4] 2> 61438 DEBUG (ScheduledTrigger-148-thread-4) [ ]
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 61438 DEBUG (ScheduledTrigger-148-thread-4) [ ]
o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
[junit4] 2> 61440 WARN (simCloudManagerPool-147-thread-5) [ ]
o.a.s.c.CreateCollectionCmd Specified number of replicas of 2 on collection
testNodeAdded is higher than the number of Solr instances currently live or
live and part of your createNodeSet(1). It's unusual to run two replica of the
same slice on the same Solr-instance.
[junit4] 2> 61450 DEBUG (ScheduledTrigger-148-thread-3) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 1
[junit4] 2> 61458 DEBUG (simCloudManagerPool-147-thread-5) [ ]
o.a.s.c.s.c.a.PolicyHelper Creating a new session
[junit4] 2> 61459 DEBUG (ScheduledTrigger-148-thread-2) [ ]
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 61459 DEBUG (ScheduledTrigger-148-thread-2) [ ]
o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
[junit4] 2> 61463 DEBUG (simCloudManagerPool-147-thread-5) [ ]
o.a.s.c.s.c.a.PolicyHelper New session created
[junit4] 2> 61482 DEBUG (ScheduledTrigger-148-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 1
[junit4] 2> 61483 DEBUG (ScheduledTrigger-148-thread-1) [ ]
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 61483 DEBUG (ScheduledTrigger-148-thread-1) [ ]
o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
[junit4] 2> 61485 INFO (simCloudManagerPool-147-thread-5) [ ]
o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 126410947
sessionWrapper.createTime 126409845752035, this.sessionWrapper.createTime
126409845752035
[junit4] 2> 61485 DEBUG (simCloudManagerPool-147-thread-5) [ ]
o.a.s.c.s.c.a.PolicyHelper session set to NULL
[junit4] 2> 61507 DEBUG (ScheduledTrigger-148-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 1
[junit4] 2> 61508 DEBUG (ScheduledTrigger-148-thread-1) [ ]
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 61508 DEBUG (ScheduledTrigger-148-thread-1) [ ]
o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
[junit4] 2> 61530 DEBUG (ScheduledTrigger-148-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 1
[junit4] 2> 61531 DEBUG (ScheduledTrigger-148-thread-1) [ ]
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 61531 DEBUG (ScheduledTrigger-148-thread-1) [ ]
o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
[junit4] 2> 61531 DEBUG (simCloudManagerPool-147-thread-8) [ ]
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 61531 DEBUG (simCloudManagerPool-147-thread-8) [ ]
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version
7
[junit4] 2> 61531 DEBUG (simCloudManagerPool-147-thread-8) [ ]
o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10000_solr]
[junit4] 2> 61532 DEBUG (simCloudManagerPool-147-thread-8) [ ]
o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:10000_solr]
[junit4] 2> 61532 DEBUG (simCloudManagerPool-147-thread-8) [ ]
o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated
with properties: {event=nodeAdded, waitFor=1, actions=[{name=compute_plan,
class=solr.ComputePlanAction}, {name=test,
class=org.apache.solr.cloud.autoscaling.sim.TestComputePlanAction$AssertingTriggerAction}],
enabled=true}
[junit4] 2> 61532 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 7
[junit4] 2> 61532 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion 7
[junit4] 2> 61560 DEBUG (ScheduledTrigger-148-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2
[junit4] 2> 61560 DEBUG (ScheduledTrigger-148-thread-4) [ ]
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 61560 DEBUG (ScheduledTrigger-148-thread-4) [ ]
o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 61560 DEBUG (ScheduledTrigger-148-thread-4) [ ]
o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10001_solr at time
126414701000835
[junit4] 2> 61581 DEBUG (ScheduledTrigger-148-thread-3) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2
[junit4] 2> 61581 DEBUG (ScheduledTrigger-148-thread-2) [ ]
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 61581 DEBUG (ScheduledTrigger-148-thread-2) [ ]
o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 61581 DEBUG (ScheduledTrigger-148-thread-2) [ ]
o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger firing
registered processor for nodes: [127.0.0.1:10001_solr] added at times
[126414701000835], now=126415739147685
[junit4] 2> 61585 DEBUG (ScheduledTrigger-148-thread-2) [ ]
o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"72f93758f883T7h5cqnja7ziypqi95z6l2x4sh",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":126414701000835,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[126414701000835],
[junit4] 2> "nodeNames":["127.0.0.1:10001_solr"]}}
[junit4] 2> 61605 DEBUG (AutoscalingActionExecutor-149-thread-1) [ ]
o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"72f93758f883T7h5cqnja7ziypqi95z6l2x4sh",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":126414701000835,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[126414701000835],
[junit4] 2> "_enqueue_time_":126415939022785,
[junit4] 2> "nodeNames":["127.0.0.1:10001_solr"]}}
[junit4] 2> 61611 DEBUG (AutoscalingActionExecutor-149-thread-1) [ ]
o.a.s.c.a.ComputePlanAction -- processing event: {
[junit4] 2> "id":"72f93758f883T7h5cqnja7ziypqi95z6l2x4sh",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":126414701000835,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[126414701000835],
[junit4] 2> "_enqueue_time_":126415939022785,
[junit4] 2> "nodeNames":["127.0.0.1:10001_solr"]}} with context
properties: {BEFORE_ACTION=[compute_plan]}
[junit4] 2> 61611 DEBUG (AutoscalingActionExecutor-149-thread-1) [ ]
o.a.s.c.s.c.a.PolicyHelper Creating a new session
[junit4] 2> 61612 DEBUG (AutoscalingActionExecutor-149-thread-1) [ ]
o.a.s.c.s.c.a.PolicyHelper New session created
[junit4] 2> 61634 INFO (AutoscalingActionExecutor-149-thread-1) [ ]
o.a.s.c.a.ComputePlanAction Computed Plan:
action=MOVEREPLICA&collection=testNodeAdded&targetNode=127.0.0.1:10001_solr&inPlaceMove=true&replica=core_node1
[junit4] 2> 61637 INFO (AutoscalingActionExecutor-149-thread-1) [ ]
o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 126418552
sessionWrapper.createTime 126417280400985, this.sessionWrapper.createTime
126417280400985
[junit4] 2> 61637 DEBUG (AutoscalingActionExecutor-149-thread-1) [ ]
o.a.s.c.s.c.a.PolicyHelper session set to NULL
[junit4] 2> 61656 INFO
(TEST-TestComputePlanAction.testNodeAdded-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.c.a.s.TestComputePlanAction Node values: {
[junit4] 2> "127.0.0.1:10001_solr":{
[junit4] 2> "sysprop.java.vendor":"Oracle Corporation",
[junit4] 2> "cores":0,
[junit4] 2> "host":"127.0.0.1",
[junit4] 2> "freedisk":1000,
[junit4] 2> "sysprop.java.version":"1.8.0_144",
[junit4] 2> "ip_1":"127",
[junit4] 2> "ip_3":"0",
[junit4] 2> "ip_2":"0",
[junit4] 2> "ip_4":"1",
[junit4] 2> "node":"127.0.0.1:10001_solr",
[junit4] 2> "metrics:solr.jvm:buffers.direct.Count":0,
[junit4] 2> "port":10001,
[junit4] 2> "heapUsage":123450000,
[junit4] 2> "sysLoadAvg":1.0,
[junit4] 2>
"metrics:solr.node:ADMIN./admin/authorization.clientErrors:count":0},
[junit4] 2> "127.0.0.1:10000_solr":{
[junit4] 2> "sysprop.java.vendor":"Oracle Corporation",
[junit4] 2> "cores":2,
[junit4] 2> "host":"127.0.0.1",
[junit4] 2> "freedisk":980,
[junit4] 2> "sysprop.java.version":"1.8.0_144",
[junit4] 2> "ip_1":"127",
[junit4] 2> "ip_3":"0",
[junit4] 2> "ip_2":"0",
[junit4] 2> "ip_4":"1",
[junit4] 2> "node":"127.0.0.1:10000_solr",
[junit4] 2> "metrics:solr.jvm:buffers.direct.Count":0,
[junit4] 2> "port":10000,
[junit4] 2> "heapUsage":123450000,
[junit4] 2> "sysLoadAvg":1.0,
[junit4] 2>
"metrics:solr.node:ADMIN./admin/authorization.clientErrors:count":0}}
[junit4] 2> 61656 INFO
(TEST-TestComputePlanAction.testNodeAdded-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.c.a.s.TestComputePlanAction Live nodes: [127.0.0.1:10001_solr,
127.0.0.1:10000_solr], collection state:
DocCollection(testNodeAdded//clusterstate.json/1)={
[junit4] 2> "replicationFactor":"2",
[junit4] 2> "pullReplicas":"0",
[junit4] 2> "router":{"name":"compositeId"},
[junit4] 2> "maxShardsPerNode":"1",
[junit4] 2> "autoAddReplicas":"false",
[junit4] 2> "nrtReplicas":"2",
[junit4] 2> "tlogReplicas":"0",
[junit4] 2> "autoCreated":"true",
[junit4] 2> "shards":{"shard1":{
[junit4] 2> "replicas":{
[junit4] 2> "core_node1":{
[junit4] 2> "node_name":"127.0.0.1:10000_solr",
[junit4] 2> "core":"testNodeAdded_shard1_replica_n1",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInBytes":123450000},
[junit4] 2> "core_node2":{
[junit4] 2> "node_name":"127.0.0.1:10000_solr",
[junit4] 2> "core":"testNodeAdded_shard1_replica_n2",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInBytes":123450000}},
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active"}}}
[junit4] 2> 61657 INFO
(TEST-TestComputePlanAction.testNodeAdded-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.c.a.s.TestComputePlanAction -------------_ FINAL STATE --------------
[junit4] 2> 61657 INFO
(TEST-TestComputePlanAction.testNodeAdded-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.c.a.s.TestComputePlanAction * Node values: {
[junit4] 2> "127.0.0.1:10001_solr":{
[junit4] 2> "sysprop.java.vendor":"Oracle Corporation",
[junit4] 2> "cores":0,
[junit4] 2> "host":"127.0.0.1",
[junit4] 2> "freedisk":1000,
[junit4] 2> "sysprop.java.version":"1.8.0_144",
[junit4] 2> "ip_1":"127",
[junit4] 2> "ip_3":"0",
[junit4] 2> "ip_2":"0",
[junit4] 2> "ip_4":"1",
[junit4] 2> "node":"127.0.0.1:10001_solr",
[junit4] 2> "metrics:solr.jvm:buffers.direct.Count":0,
[junit4] 2> "port":10001,
[junit4] 2> "heapUsage":123450000,
[junit4] 2> "sysLoadAvg":1.0,
[junit4] 2>
"metrics:solr.node:ADMIN./admin/authorization.clientErrors:count":0},
[junit4] 2> "127.0.0.1:10000_solr":{
[junit4] 2> "sysprop.java.vendor":"Oracle Corporation",
[junit4] 2> "cores":2,
[junit4] 2> "host":"127.0.0.1",
[junit4] 2> "freedisk":980,
[junit4] 2> "sysprop.java.version":"1.8.0_144",
[junit4] 2> "ip_1":"127",
[junit4] 2> "ip_3":"0",
[junit4] 2> "ip_2":"0",
[junit4] 2> "ip_4":"1",
[junit4] 2> "node":"127.0.0.1:10000_solr",
[junit4] 2> "metrics:solr.jvm:buffers.direct.Count":0,
[junit4] 2> "port":10000,
[junit4] 2> "heapUsage":123450000,
[junit4] 2> "sysLoadAvg":1.0,
[junit4] 2>
"metrics:solr.node:ADMIN./admin/authorization.clientErrors:count":0}}
[junit4] 2> 61657 INFO
(TEST-TestComputePlanAction.testNodeAdded-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.c.a.s.TestComputePlanAction * Live nodes: [127.0.0.1:10001_solr,
127.0.0.1:10000_solr]
[junit4] 2> 61658 INFO
(TEST-TestComputePlanAction.testNodeAdded-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.c.a.s.TestComputePlanAction * Collection testNodeAdded state:
DocCollection(testNodeAdded//clusterstate.json/1)={
[junit4] 2> "replicationFactor":"2",
[junit4] 2> "pullReplicas":"0",
[junit4] 2> "router":{"name":"compositeId"},
[junit4] 2> "maxShardsPerNode":"1",
[junit4] 2> "autoAddReplicas":"false",
[junit4] 2> "nrtReplicas":"2",
[junit4] 2> "tlogReplicas":"0",
[junit4] 2> "autoCreated":"true",
[junit4] 2> "shards":{"shard1":{
[junit4] 2> "replicas":{
[junit4] 2> "core_node1":{
[junit4] 2> "node_name":"127.0.0.1:10000_solr",
[junit4] 2> "core":"testNodeAdded_shard1_replica_n1",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInBytes":123450000},
[junit4] 2> "core_node2":{
[junit4] 2> "node_name":"127.0.0.1:10000_solr",
[junit4] 2> "core":"testNodeAdded_shard1_replica_n2",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInBytes":123450000}},
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active"}}}
[junit4] 2> 61659 INFO
(TEST-TestComputePlanAction.testNodeAdded-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.SolrTestCaseJ4 ###Ending testNodeAdded
[junit4] 2> 61659 INFO
(TEST-TestComputePlanAction.testNodeAdded-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.c.a.s.SimSolrCloudTestCase
[junit4] 2>
[junit4] 2> 61659 INFO
(TEST-TestComputePlanAction.testNodeAdded-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2> 61660 INFO
(TEST-TestComputePlanAction.testNodeAdded-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.c.a.s.SimSolrCloudTestCase ############ FINAL CLUSTER STATS ############
[junit4] 2> 61660 INFO
(TEST-TestComputePlanAction.testNodeAdded-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2>
[junit4] 2> 61660 INFO
(TEST-TestComputePlanAction.testNodeAdded-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.c.a.s.SimSolrCloudTestCase ## Live nodes: 2
[junit4] 2> 61665 INFO
(TEST-TestComputePlanAction.testNodeAdded-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.c.a.s.SimSolrCloudTestCase ## Empty nodes: 1
[junit4] 2> 61665 INFO
(TEST-TestComputePlanAction.testNodeAdded-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.c.a.s.SimSolrCloudTestCase ## Dead nodes: 0
[junit4] 2> 61669 INFO
(TEST-TestComputePlanAction.testNodeAdded-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.c.a.s.SimSolrCloudTestCase ## Collections: [testNodeAdded]
[junit4] 2> 61669 INFO
(TEST-TestComputePlanAction.testNodeAdded-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.c.a.s.SimSolrCloudTestCase ## Max replicas per node: 2
[junit4] 2> 61669 INFO
(TEST-TestComputePlanAction.testNodeAdded-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.c.a.s.SimSolrCloudTestCase ## Min replicas per node: 0
[junit4] 2> 61669 INFO
(TEST-TestComputePlanAction.testNodeAdded-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.c.a.s.SimSolrCloudTestCase ## Total replicas: 2
[junit4] 2> 61671 INFO
(TEST-TestComputePlanAction.testNodeAdded-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.c.a.s.SimSolrCloudTestCase ## * testNodeAdded 2
[junit4] 2> 61672 INFO
(TEST-TestComputePlanAction.testNodeAdded-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.c.a.s.SimSolrCloudTestCase ## - active 2
[junit4] 2> 61672 INFO
(TEST-TestComputePlanAction.testNodeAdded-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.c.a.s.SimSolrCloudTestCase ######### Final Solr op counts ##########
[junit4] 2> 61676 INFO
(TEST-TestComputePlanAction.testNodeAdded-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.c.a.s.SimSolrCloudTestCase ## - CREATE 1
[junit4] 2> 61676 INFO
(TEST-TestComputePlanAction.testNodeAdded-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.c.a.s.SimSolrCloudTestCase ## - autoscaling 5
[junit4] 2> 61677 INFO
(TEST-TestComputePlanAction.testNodeAdded-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.c.a.s.SimSolrCloudTestCase ## - update 2
[junit4] 2> 61677 INFO
(TEST-TestComputePlanAction.testNodeAdded-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.c.a.s.SimSolrCloudTestCase ######### Autoscaling event counts ###########
[junit4] 2> 61711 DEBUG (AutoscalingActionExecutor-149-thread-1) [ ]
o.a.s.c.a.ScheduledTriggers -- processing took 5278 ms for event
id=72f93758f883T7h5cqnja7ziypqi95z6l2x4sh
[junit4] 2> 61721 DEBUG (ScheduledTrigger-148-thread-3) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2
[junit4] 2> 61721 DEBUG (ScheduledTrigger-148-thread-2) [ ]
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 61725 DEBUG (ScheduledTrigger-148-thread-2) [ ]
o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> NOTE: reproduce with: ant test
-Dtestcase=TestComputePlanAction -Dtests.method=testNodeAdded
-Dtests.seed=CE91D734C7FAA56D -Dtests.slow=true -Dtests.locale=sq-AL
-Dtests.timezone=America/Indiana/Marengo -Dtests.asserts=true
-Dtests.file.encoding=UTF-8
[junit4] ERROR 0.72s J0 | TestComputePlanAction.testNodeAdded <<<
[junit4] > Throwable #1: java.util.ConcurrentModificationException
[junit4] > at
__randomizedtesting.SeedInfo.seed([CE91D734C7FAA56D:AB52814365590D6E]:0)
[junit4] > at
java.util.ArrayList$Itr.checkForComodification(ArrayList.java:901)
[junit4] > at java.util.ArrayList$Itr.next(ArrayList.java:851)
[junit4] > at
org.apache.solr.cloud.autoscaling.sim.SimSolrCloudTestCase.tearDown(SimSolrCloudTestCase.java:141)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 61757 DEBUG (ScheduledTrigger-148-thread-3) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2
[junit4] 2> 61757 DEBUG (ScheduledTrigger-148-thread-4) [ ]
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 61757 DEBUG (ScheduledTrigger-148-thread-4) [ ]
o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 61767 INFO
(TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[CE91D734C7FAA56D])
[ ] o.a.s.SolrTestCaseJ4 ###Starting testNodeWithMultipleReplicasLost
[junit4] 2> 61767 DEBUG
(TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[CE91D734C7FAA56D])
[ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode
version 8
[junit4] 2> 61767 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 8
[junit4] 2> 61768 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 61768 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 61768 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 8, lastZnodeVersion 8
[junit4] 2> 61769 INFO
(TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[CE91D734C7FAA56D])
[ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and
clearing object cache...
[junit4] 2> 61770 WARN (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Interrupted
[junit4] 2> java.lang.InterruptedException
[junit4] 2> at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014)
[junit4] 2> at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048)
[junit4] 2> at
org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:166)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 61770 DEBUG
(TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[CE91D734C7FAA56D])
[ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed
explicitly
[junit4] 2> 61772 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 61772 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode
version 9
[junit4] 2> 61772 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10000_solr]
[junit4] 2> 61772 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 9, lastZnodeVersion -1
[junit4] 2> 61772 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 9
[junit4] 2> 61772 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 61772 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 9, lastZnodeVersion 9
[junit4] 2> 61773 DEBUG (ScheduledTrigger-151-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 1
[junit4] 2> 61802 DEBUG (ScheduledTrigger-151-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 1
[junit4] 2> 61826 DEBUG (ScheduledTrigger-151-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 1
[junit4] 2> 61854 DEBUG (ScheduledTrigger-151-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 1
[junit4] 2> 61875 DEBUG (ScheduledTrigger-151-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 1
[junit4] 2> 61880 DEBUG (simCloudManagerPool-150-thread-1) [ ]
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 61880 DEBUG (simCloudManagerPool-150-thread-1) [ ]
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version
10
[junit4] 2> 61882 DEBUG (simCloudManagerPool-150-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10000_solr]
[junit4] 2> 61882 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 10
[junit4] 2> 61882 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 61882 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 10, lastZnodeVersion 10
[junit4] 2> 61891 DEBUG (simCloudManagerPool-150-thread-2) [ ]
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 61891 DEBUG (simCloudManagerPool-150-thread-2) [ ]
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version
11
[junit4] 2> 61897 DEBUG (ScheduledTrigger-151-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 1
[junit4] 2> 61897 DEBUG (simCloudManagerPool-150-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10000_solr]
[junit4] 2> 61898 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 11
[junit4] 2> 61898 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 61898 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 11, lastZnodeVersion 11
[junit4] 2> 61918 DEBUG (ScheduledTrigger-151-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 1
[junit4] 2> 61944 DEBUG (ScheduledTrigger-151-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 1
[junit4] 2> 61969 DEBUG (ScheduledTrigger-151-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 1
[junit4] 2> 61990 DEBUG (ScheduledTrigger-151-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 1
[junit4] 2> 62011 DEBUG (ScheduledTrigger-151-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 4
[junit4] 2> 62023 DEBUG (simCloudManagerPool-150-thread-3) [ ]
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 62024 DEBUG (simCloudManagerPool-150-thread-3) [ ]
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version
12
[junit4] 2> 62024 DEBUG (simCloudManagerPool-150-thread-3) [ ]
o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10004_solr,
127.0.0.1:10000_solr, 127.0.0.1:10002_solr, 127.0.0.1:10003_solr]
[junit4] 2> 62024 DEBUG (simCloudManagerPool-150-thread-3) [ ]
o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10004_solr,
127.0.0.1:10000_solr, 127.0.0.1:10002_solr, 127.0.0.1:10003_solr]
[junit4] 2> 62024 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 12
[junit4] 2> 62024 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 62024 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 12, lastZnodeVersion 12
[junit4] 2> 62024 DEBUG (ScheduledTrigger-151-thread-4) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with
currently live nodes: 4
[junit4] 2> 62038 DEBUG (ScheduledTrigger-151-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 4
[junit4] 2> 62030 DEBUG (simCloudManagerPool-150-thread-4) [ ]
o.a.s.c.o.ClusterStateMutator building a new cName:
testNodeWithMultipleReplicasLost
[junit4] 2> 62045 DEBUG (ScheduledTrigger-151-thread-3) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with
currently live nodes: 4
[junit4] 2> 62052 DEBUG (simCloudManagerPool-150-thread-4) [ ]
o.a.s.c.s.c.a.PolicyHelper Creating a new session
[junit4] 2> 62052 DEBUG (simCloudManagerPool-150-thread-4) [ ]
o.a.s.c.s.c.a.PolicyHelper New session created
[junit4] 2> 62056 INFO (simCloudManagerPool-150-thread-4) [ ]
o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 126439513
sessionWrapper.createTime 126439313446635, this.sessionWrapper.createTime
126439313446635
[junit4] 2> 62056 DEBUG (simCloudManagerPool-150-thread-4) [ ]
o.a.s.c.s.c.a.PolicyHelper session set to NULL
[junit4] 2> 62060 DEBUG (ScheduledTrigger-151-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 4
[junit4] 2> 62067 DEBUG (ScheduledTrigger-151-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with
currently live nodes: 4
[junit4] 2> 62081 DEBUG (ScheduledTrigger-151-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 4
[junit4] 2> 62087 DEBUG (ScheduledTrigger-151-thread-3) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with
currently live nodes: 4
[junit4] 2> 62105 DEBUG (ScheduledTrigger-151-thread-4) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 4
[junit4] 2> 62109 DEBUG (ScheduledTrigger-151-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with
currently live nodes: 4
[junit4] 2> 62126 DEBUG (ScheduledTrigger-151-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 4
[junit4] 2> 62129 DEBUG (ScheduledTrigger-151-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with
currently live nodes: 4
[junit4] 2> 62131 DEBUG
(TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[CE91D734C7FAA56D])
[ ] o.a.s.c.a.s.TestComputePlanAction -- cluster state: znodeVersion: 4
[junit4] 2> live nodes:[127.0.0.1:10004_solr, 127.0.0.1:10000_solr,
127.0.0.1:10002_solr, 127.0.0.1:10003_solr]
[junit4] 2>
collections:{testNodeWithMultipleReplicasLost=DocCollection(testNodeWithMultipleReplicasLost//clusterstate.json/3)={
[junit4] 2> "replicationFactor":"3",
[junit4] 2> "pullReplicas":"0",
[junit4] 2> "router":{"name":"compositeId"},
[junit4] 2> "maxShardsPerNode":"1",
[junit4] 2> "autoAddReplicas":"false",
[junit4] 2> "nrtReplicas":"3",
[junit4] 2> "tlogReplicas":"0",
[junit4] 2> "autoCreated":"true",
[junit4] 2> "shards":{
[junit4] 2> "shard2":{
[junit4] 2> "replicas":{
[junit4] 2> "core_node6":{
[junit4] 2> "node_name":"127.0.0.1:10002_solr",
[junit4] 2>
"core":"testNodeWithMultipleReplicasLost_shard2_replica_n6",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInBytes":123450000},
[junit4] 2> "core_node4":{
[junit4] 2> "node_name":"127.0.0.1:10000_solr",
[junit4] 2>
"core":"testNodeWithMultipleReplicasLost_shard2_replica_n4",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInBytes":123450000},
[junit4] 2> "core_node5":{
[junit4] 2> "node_name":"127.0.0.1:10004_solr",
[junit4] 2>
"core":"testNodeWithMultipleReplicasLost_shard2_replica_n5",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInBytes":123450000}},
[junit4] 2> "range":"0-7fffffff",
[junit4] 2> "state":"active"},
[junit4] 2> "shard1":{
[junit4] 2> "replicas":{
[junit4] 2> "core_node1":{
[junit4] 2> "node_name":"127.0.0.1:10003_solr",
[junit4] 2>
"core":"testNodeWithMultipleReplicasLost_shard1_replica_n1",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInBytes":123450000},
[junit4] 2> "core_node2":{
[junit4] 2> "node_name":"127.0.0.1:10002_solr",
[junit4] 2>
"core":"testNodeWithMultipleReplicasLost_shard1_replica_n2",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInBytes":123450000},
[junit4] 2> "core_node3":{
[junit4] 2> "node_name":"127.0.0.1:10004_solr",
[junit4] 2>
"core":"testNodeWithMultipleReplicasLost_shard1_replica_n3",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInBytes":123450000}},
[junit4] 2> "range":"80000000-ffffffff",
[junit4] 2> "state":"active"}}}}
[junit4] 2> 62152 DEBUG (ScheduledTrigger-151-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 3
[junit4] 2> 62153 DEBUG (ScheduledTrigger-151-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10004_solr
[junit4] 2> 62153 DEBUG (ScheduledTrigger-151-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with
currently live nodes: 3
[junit4] 2> 62153 DEBUG (ScheduledTrigger-151-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10004_solr
[junit4] 2> 62179 DEBUG (ScheduledTrigger-151-thread-4) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with
currently live nodes: 3
[junit4] 2> 62179 DEBUG (ScheduledTrigger-151-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 3
[junit4] 2> 62184 DEBUG (ScheduledTrigger-151-thread-4) [ ]
o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost
nodes: [127.0.0.1:10004_solr]
[junit4] 2> 62198 DEBUG (ScheduledTrigger-151-thread-4) [ ]
o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"73001e3e732bT7h5cqnja7ziypqi95z6l2x4sm",
[junit4] 2> "source":"node_lost_trigger",
[junit4] 2> "eventTime":126444344603435,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[126444344603435],
[junit4] 2> "nodeNames":["127.0.0.1:10004_solr"]}}
[junit4] 2> 62208 DEBUG (simCloudManagerPool-150-thread-12) [ ]
o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for
testNodeWithMultipleReplicasLost / shard1: {"core_node1":{
[junit4] 2> "node_name":"127.0.0.1:10003_solr",
[junit4] 2>
"core":"testNodeWithMultipleReplicasLost_shard1_replica_n1",
[junit4] 2> "leader":"true",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInBytes":123450000,
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"testNodeWithMultipleReplicasLost"}}
[junit4] 2> 62209 DEBUG (simCloudManagerPool-150-thread-11) [ ]
o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for
testNodeWithMultipleReplicasLost / shard2: {"core_node6":{
[junit4] 2> "node_name":"127.0.0.1:10002_solr",
[junit4] 2>
"core":"testNodeWithMultipleReplicasLost_shard2_replica_n6",
[junit4] 2> "leader":"true",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInBytes":123450000,
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"testNodeWithMultipleReplicasLost"}}
[junit4] 2> 62216 DEBUG (AutoscalingActionExecutor-152-thread-1) [ ]
o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"73001e3e732bT7h5cqnja7ziypqi95z6l2x4sm",
[junit4] 2> "source":"node_lost_trigger",
[junit4] 2> "eventTime":126444344603435,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[126444344603435],
[junit4] 2> "_enqueue_time_":126447102905835,
[junit4] 2> "nodeNames":["127.0.0.1:10004_solr"]}}
[junit4] 2> 62217 DEBUG (AutoscalingActionExecutor-152-thread-1) [ ]
o.a.s.c.a.ComputePlanAction -- processing event: {
[junit4] 2> "id":"73001e3e732bT7h5cqnja7ziypqi95z6l2x4sm",
[junit4] 2> "source":"node_lost_trigger",
[junit4] 2> "eventTime":126444344603435,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[126444344603435],
[junit4] 2> "_enqueue_time_":126447102905835,
[junit4] 2> "nodeNames":["127.0.0.1:10004_solr"]}} with context
properties: {BEFORE_ACTION=[compute_plan]}
[junit4] 2> 62223 DEBUG (AutoscalingActionExecutor-152-thread-1) [ ]
o.a.s.c.s.c.a.PolicyHelper Creating a new session
[junit4] 2> 62229 DEBUG (AutoscalingActionExecutor-152-thread-1) [ ]
o.a.s.c.s.c.a.PolicyHelper New session created
[junit4] 2> 62231 INFO (AutoscalingActionExecutor-152-thread-1) [ ]
o.a.s.c.a.ComputePlanAction Computed Plan:
action=MOVEREPLICA&collection=testNodeWithMultipleReplicasLost&targetNode=127.0.0.1:10003_solr&inPlaceMove=true&replica=core_node5
[junit4] 2> 62231 INFO (AutoscalingActionExecutor-152-thread-1) [ ]
o.a.s.c.a.ComputePlanAction Computed Plan:
action=MOVEREPLICA&collection=testNodeWithMultipleReplicasLost&targetNode=127.0.0.1:10000_solr&inPlaceMove=true&replica=core_node3
[junit4] 2> 62231 INFO (AutoscalingActionExecutor-152-thread-1) [ ]
o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 126448264
sessionWrapper.createTime 126448125077635, this.sessionWrapper.createTime
126448125077635
[junit4] 2> 62231 DEBUG (AutoscalingActionExecutor-152-thread-1) [ ]
o.a.s.c.s.c.a.PolicyHelper session set to NULL
[junit4] 2> 62240 INFO
(TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[CE91D734C7FAA56D])
[ ] o.a.s.c.a.s.TestComputePlanAction
action=MOVEREPLICA&collection=testNodeWithMultipleReplicasLost&targetNode=127.0.0.1:10003_solr&inPlaceMove=true&replica=core_node5
[junit4] 2> 62240 INFO
(TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[CE91D734C7FAA56D])
[ ] o.a.s.c.a.s.TestComputePlanAction
action=MOVEREPLICA&collection=testNodeWithMultipleReplicasLost&targetNode=127.0.0.1:10000_solr&inPlaceMove=true&replica=core_node3
[junit4] 2> 62241 INFO
(TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[CE91D734C7FAA56D])
[ ] o.a.s.c.a.s.TestComputePlanAction -------------_ FINAL STATE
--------------
[junit4] 2> 62241 INFO
(TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[CE91D734C7FAA56D])
[ ] o.a.s.c.a.s.TestComputePlanAction * Node values: {
[junit4] 2> "127.0.0.1:10004_solr":{
[junit4] 2> "sysprop.java.vendor":"Oracle Corporation",
[junit4] 2> "cores":2,
[junit4] 2> "host":"127.0.0.1",
[junit4] 2> "freedisk":980,
[junit4] 2> "sysprop.java.version":"1.8.0_144",
[junit4] 2> "ip_1":"127",
[junit4] 2> "ip_3":"0",
[junit4] 2> "ip_2":"0",
[junit4] 2> "ip_4":"1",
[junit4] 2> "node":"127.0.0.1:10004_solr",
[junit4] 2> "metrics:solr.jvm:buffers.direct.Count":0,
[junit4] 2> "port":10004,
[junit4] 2> "heapUsage":123450000,
[junit4] 2> "sysLoadAvg":1.0,
[junit4] 2>
"metrics:solr.node:ADMIN./admin/authorization.clientErrors:count":0},
[junit4] 2> "127.0.0.1:10000_solr":{
[junit4] 2> "sysprop.java.vendor":"Oracle Corporation",
[junit4] 2> "cores":1,
[junit4] 2> "host":"127.0.0.1",
[junit4] 2> "freedisk":970,
[junit4] 2> "sysprop.java.version":"1.8.0_144",
[junit4] 2> "ip_1":"127",
[junit4] 2> "ip_3":"0",
[junit4] 2> "ip_2":"0",
[junit4] 2> "ip_4":"1",
[junit4] 2> "node":"127.0.0.1:10000_solr",
[junit4] 2> "metrics:solr.jvm:buffers.direct.Count":0,
[junit4] 2> "port":10000,
[junit4] 2> "heapUsage":123450000,
[junit4] 2> "sysLoadAvg":1.0,
[junit4] 2>
"metrics:solr.node:ADMIN./admin/authorization.clientErrors:count":0},
[junit4] 2> "127.0.0.1:10002_solr":{
[junit4] 2> "sysprop.java.vendor":"Oracle Corporation",
[junit4] 2> "cores":2,
[junit4] 2> "host":"127.0.0.1",
[junit4] 2> "freedisk":980,
[junit4] 2> "sysprop.java.version":"1.8.0_144",
[junit4] 2> "ip_1":"127",
[junit4] 2> "ip_3":"0",
[junit4] 2> "ip_2":"0",
[junit4] 2> "ip_4":"1",
[junit4] 2> "node":"127.0.0.1:10002_solr",
[junit4] 2> "metrics:solr.jvm:buffers.direct.Count":0,
[junit4] 2> "port":10002,
[junit4] 2> "heapUsage":123450000,
[junit4] 2> "sysLoadAvg":1.0,
[junit4] 2>
"metrics:solr.node:ADMIN./admin/authorization.clientErrors:count":0},
[junit4] 2> "127.0.0.1:10003_solr":{
[junit4] 2> "sysprop.java.vendor":"Oracle Corporation",
[junit4] 2> "cores":1,
[junit4] 2> "host":"127.0.0.1",
[junit4] 2> "freedisk":990,
[junit4] 2> "sysprop.java.version":"1.8.0_144",
[junit4] 2> "ip_1":"127",
[junit4] 2> "ip_3":"0",
[junit4] 2> "ip_2":"0",
[junit4] 2> "ip_4":"1",
[junit4] 2> "node":"127.0.0.1:10003_solr",
[junit4] 2> "metrics:solr.jvm:buffers.direct.Count":0,
[junit4] 2> "port":10003,
[junit4] 2> "heapUsage":123450000,
[junit4] 2> "sysLoadAvg":1.0,
[junit4] 2>
"metrics:solr.node:ADMIN./admin/authorization.clientErrors:count":0}}
[junit4] 2> 62241 INFO
(TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[CE91D734C7FAA56D])
[ ] o.a.s.c.a.s.TestComputePlanAction * Live nodes: [127.0.0.1:10000_solr,
127.0.0.1:10002_solr, 127.0.0.1:10003_solr]
[junit4] 2> 62242 INFO
(TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[CE91D734C7FAA56D])
[ ] o.a.s.c.a.s.TestComputePlanAction * Collection
testNodeWithMultipleReplicasLost state:
DocCollection(testNodeWithMultipleReplicasLost//clusterstate.json/5)={
[junit4] 2> "replicationFactor":"3",
[junit4] 2> "pullReplicas":"0",
[junit4] 2> "router":{"name":"compositeId"},
[junit4] 2> "maxShardsPerNode":"1",
[junit4] 2> "autoAddReplicas":"false",
[junit4] 2> "nrtReplicas":"3",
[junit4] 2> "tlogReplicas":"0",
[junit4] 2> "autoCreated":"true",
[junit4] 2> "shards":{
[junit4] 2> "shard2":{
[junit4] 2> "replicas":{
[junit4] 2> "core_node6":{
[junit4] 2>
"core":"testNodeWithMultipleReplicasLost_shard2_replica_n6",
[junit4] 2> "leader":"true",
[junit4] 2> "INDEX.sizeInBytes":123450000,
[junit4] 2> "node_name":"127.0.0.1:10002_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node4":{
[junit4] 2> "node_name":"127.0.0.1:10000_solr",
[junit4] 2>
"core":"testNodeWithMultipleReplicasLost_shard2_replica_n4",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInBytes":123450000},
[junit4] 2> "core_node5":{
[junit4] 2> "node_name":"127.0.0.1:10004_solr",
[junit4] 2>
"core":"testNodeWithMultipleReplicasLost_shard2_replica_n5",
[junit4] 2> "state":"down",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInBytes":123450000}},
[junit4] 2> "range":"0-7fffffff",
[junit4] 2> "state":"active"},
[junit4] 2> "shard1":{
[junit4] 2> "replicas":{
[junit4] 2> "core_node1":{
[junit4] 2>
"core":"testNodeWithMultipleReplicasLost_shard1_replica_n1",
[junit4] 2> "leader":"true",
[junit4] 2> "INDEX.sizeInBytes":123450000,
[junit4] 2> "node_name":"127.0.0.1:10003_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node2":{
[junit4] 2> "node_name":"127.0.0.1:10002_solr",
[junit4] 2>
"core":"testNodeWithMultipleReplicasLost_shard1_replica_n2",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInBytes":123450000},
[junit4] 2> "core_node3":{
[junit4] 2> "node_name":"127.0.0.1:10004_solr",
[junit4] 2>
"core":"testNodeWithMultipleReplicasLost_shard1_replica_n3",
[junit4] 2> "state":"down",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInBytes":123450000}},
[junit4] 2> "range":"80000000-ffffffff",
[junit4] 2> "state":"active"}}}
[junit4] 2> 62242 INFO
(TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[CE91D734C7FAA56D])
[ ] o.a.s.SolrTestCaseJ4 ###Ending testNodeWithMultipleReplicasLost
[junit4] 2> 62242 INFO
(TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[CE91D734C7FAA56D])
[ ] o.a.s.c.a.s.SimSolrCloudTestCase
[junit4] 2>
[junit4] 2> 62242 INFO
(TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[CE91D734C7FAA56D])
[ ] o.a.s.c.a.s.SimSolrCloudTestCase
#############################################
[junit4] 2> 62242 INFO
(TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[CE91D734C7FAA56D])
[ ] o.a.s.c.a.s.SimSolrCloudTestCase ############ FINAL CLUSTER STATS
############
[junit4] 2> 62242 INFO
(TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[CE91D734C7FAA56D])
[ ] o.a.s.c.a.s.SimSolrCloudTestCase
#############################################
[junit4] 2>
[junit4] 2> 62242 INFO
(TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[CE91D734C7FAA56D])
[ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Live nodes: 3
[junit4] 2> 62242 INFO
(TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[CE91D734C7FAA56D])
[ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Empty nodes: 0
[junit4] 2> 62242 INFO
(TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[CE91D734C7FAA56D])
[ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Dead nodes: 1
[junit4] 2> 62242 INFO
(TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[CE91D734C7FAA56D])
[ ] o.a.s.c.a.s.SimSolrCloudTestCase ## 127.0.0.1:10004_solr
[junit4] 2> 62242 INFO
(TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[CE91D734C7FAA56D])
[ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Collections:
[testNodeWithMultipleReplicasLost]
[junit4] 2> 62242 INFO
(TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[CE91D734C7FAA56D])
[ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Max replicas per node: 2
[junit4] 2> 62242 INFO
(TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[CE91D734C7FAA56D])
[ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Min replicas per node: 1
[junit4] 2> 62242 INFO
(TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[CE91D734C7FAA56D])
[ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Total replicas: 4
[junit4] 2> 62242 INFO
(TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[CE91D734C7FAA56D])
[ ] o.a.s.c.a.s.SimSolrCloudTestCase ## * testNodeWithMultipleReplicasLost
4
[junit4] 2> 62242 INFO
(TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[CE91D734C7FAA56D])
[ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - active 4
[junit4] 2> 62242 INFO
(TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[CE91D734C7FAA56D])
[ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Final Solr op counts
##########
[junit4] 2> 62242 INFO
(TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[CE91D734C7FAA56D])
[ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - CREATE 1
[junit4] 2> 62242 INFO
(TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[CE91D734C7FAA56D])
[ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - autoscaling 3
[junit4] 2> 62242 INFO
(TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[CE91D734C7FAA56D])
[ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - update 2
[junit4] 2> 62242 INFO
(TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[CE91D734C7FAA56D])
[ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Autoscaling event counts
###########
[junit4] 2> 62243 INFO
(TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[CE91D734C7FAA56D])
[ ] o.a.s.c.a.s.SimSolrCloudTestCase ## * Trigger: node_lost_trigger
[junit4] 2> 62243 INFO
(TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[CE91D734C7FAA56D])
[ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - BEFORE_ACTION 1
[junit4] 2> 62243 INFO
(TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[CE91D734C7FAA56D])
[ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - STARTED 1
[junit4] 2> 62254 DEBUG (AutoscalingActionExecutor-152-thread-1) [ ]
o.a.s.c.a.ScheduledTriggers -- processing took 1923 ms for event
id=73001e3e732bT7h5cqnja7ziypqi95z6l2x4sm
[junit4] 2> 62257 DEBUG (ScheduledTrigger-151-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 3
[junit4] 2> 62270 DEBUG (ScheduledTrigger-151-thread-4) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with
currently live nodes: 3
[junit4] 2> 62277 INFO
(TEST-TestComputePlanAction.testNodeLost-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.SolrTestCaseJ4 ###Starting testNodeLost
[junit4] 2> 62277 DEBUG
(TEST-TestComputePlanAction.testNodeLost-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version
13
[junit4] 2> 62278 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 13
[junit4] 2> 62278 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 62279 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 62279 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 13, lastZnodeVersion 13
[junit4] 2> 62280 INFO (simCloudManagerPool-150-thread-17) [ ]
o.a.s.c.ActionThrottle Throttling leader attempts - waiting for 353ms
[junit4] 2> 62281 INFO (simCloudManagerPool-150-thread-18) [ ]
o.a.s.c.ActionThrottle Throttling leader attempts - waiting for 190ms
[junit4] 2> 62282 INFO (simCloudManagerPool-150-thread-19) [ ]
o.a.s.c.ActionThrottle Throttling leader attempts - waiting for 229ms
[junit4] 2> 62282 INFO (simCloudManagerPool-150-thread-20) [ ]
o.a.s.c.ActionThrottle Throttling leader attempts - waiting for 124ms
[junit4] 2> 62283 DEBUG
(TEST-TestComputePlanAction.testNodeLost-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.c.a.s.SimNodeStateProvider - removing dead node values:
127.0.0.1:10004_solr
[junit4] 2> 62283 INFO
(TEST-TestComputePlanAction.testNodeLost-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing
object cache...
[junit4] 2> 62283 WARN (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Interrupted
[junit4] 2> java.lang.InterruptedException
[junit4] 2> at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014)
[junit4] 2> at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048)
[junit4] 2> at
org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:166)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 62291 DEBUG
(TEST-TestComputePlanAction.testNodeLost-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 62306 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 62307 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode
version 14
[junit4] 2> 62308 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10000_solr]
[junit4] 2> 62308 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 14, lastZnodeVersion -1
[junit4] 2> 62308 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 14
[junit4] 2> 62310 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 62310 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 14, lastZnodeVersion 14
[junit4] 2> 62311 DEBUG (ScheduledTrigger-154-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 1
[junit4] 2> 62332 DEBUG (ScheduledTrigger-154-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 1
[junit4] 2> 62354 DEBUG (ScheduledTrigger-154-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 1
[junit4] 2> 62378 DEBUG (ScheduledTrigger-154-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 1
[junit4] 2> 62400 DEBUG (ScheduledTrigger-154-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 1
[junit4] 2> 62422 DEBUG (ScheduledTrigger-154-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 1
[junit4] 2> 62443 DEBUG (ScheduledTrigger-154-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 1
[junit4] 2> 62459 DEBUG (simCloudManagerPool-153-thread-1) [ ]
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 62462 DEBUG (simCloudManagerPool-153-thread-1) [ ]
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version
15
[junit4] 2> 62526 DEBUG (simCloudManagerPool-153-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10000_solr]
[junit4] 2> 62526 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 15
[junit4] 2> 62526 DEBUG (ScheduledTrigger-154-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 1
[junit4] 2> 62526 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 62527 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 15, lastZnodeVersion 15
[junit4] 2> 62534 DEBUG (simCloudManagerPool-153-thread-2) [ ]
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 62534 DEBUG (simCloudManagerPool-153-thread-2) [ ]
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version
16
[junit4] 2> 62535 DEBUG (simCloudManagerPool-153-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10000_solr]
[junit4] 2> 62535 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 16
[junit4] 2> 62535 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 62535 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 16, lastZnodeVersion 16
[junit4] 2> 62553 DEBUG (ScheduledTrigger-154-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 1
[junit4] 2> 62574 DEBUG (ScheduledTrigger-154-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 1
[junit4] 2> 62595 DEBUG (ScheduledTrigger-154-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 1
[junit4] 2> 62615 DEBUG (ScheduledTrigger-154-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 1
[junit4] 2> 62638 DEBUG (ScheduledTrigger-154-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 1
[junit4] 2> 62667 DEBUG (ScheduledTrigger-154-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2
[junit4] 2> 62669 DEBUG (simCloudManagerPool-153-thread-3) [ ]
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 62670 DEBUG (simCloudManagerPool-153-thread-3) [ ]
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version
17
[junit4] 2> 62670 DEBUG (simCloudManagerPool-153-thread-3) [ ]
o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10005_solr,
127.0.0.1:10000_solr]
[junit4] 2> 62670 DEBUG (simCloudManagerPool-153-thread-3) [ ]
o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10005_solr,
127.0.0.1:10000_solr]
[junit4] 2> 62670 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 17
[junit4] 2> 62671 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 62671 DEBUG (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 17, lastZnodeVersion 17
[junit4] 2> 62671 DEBUG (ScheduledTrigger-154-thread-4) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with
currently live nodes: 2
[junit4] 2> 62671 DEBUG (simCloudManagerPool-153-thread-4) [ ]
o.a.s.c.o.ClusterStateMutator building a new cName: testNodeLost
[junit4] 2> 62686 DEBUG (simCloudManagerPool-153-thread-4) [ ]
o.a.s.c.s.c.a.PolicyHelper Creating a new session
[junit4] 2> 62686 DEBUG (simCloudManagerPool-153-thread-4) [ ]
o.a.s.c.s.c.a.PolicyHelper New session created
[junit4] 2> 62688 DEBUG (ScheduledTrigger-154-thread-3) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2
[junit4] 2> 62689 INFO (simCloudManagerPool-153-thread-4) [ ]
o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 126471135
sessionWrapper.createTime 126471000946535, this.sessionWrapper.createTime
126471000946535
[junit4] 2> 62689 DEBUG (simCloudManagerPool-153-thread-4) [ ]
o.a.s.c.s.c.a.PolicyHelper session set to NULL
[junit4] 2> 62753 DEBUG (ScheduledTrigger-154-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with
currently live nodes: 2
[junit4] 2> 62753 DEBUG (ScheduledTrigger-154-thread-1) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2
[junit4] 2> 62764 DEBUG
(TEST-TestComputePlanAction.testNodeLost-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.c.a.s.TestComputePlanAction -- cluster state: znodeVersion: 10
[junit4] 2> live nodes:[127.0.0.1:10000_solr, 127.0.0.1:10005_solr]
[junit4] 2>
collections:{testNodeLost=DocCollection(testNodeLost//clusterstate.json/9)={
[junit4] 2> "replicationFactor":"2",
[junit4] 2> "pullReplicas":"0",
[junit4] 2> "router":{"name":"compositeId"},
[junit4] 2> "maxShardsPerNode":"1",
[junit4] 2> "autoAddReplicas":"false",
[junit4] 2> "nrtReplicas":"2",
[junit4] 2> "tlogReplicas":"0",
[junit4] 2> "autoCreated":"true",
[junit4] 2> "shards":{"shard1":{
[junit4] 2> "replicas":{
[junit4] 2> "core_node1":{
[junit4] 2> "node_name":"127.0.0.1:10005_solr",
[junit4] 2> "core":"testNodeLost_shard1_replica_n1",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInBytes":123450000},
[junit4] 2> "core_node2":{
[junit4] 2> "node_name":"127.0.0.1:10000_solr",
[junit4] 2> "core":"testNodeLost_shard1_replica_n2",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInBytes":123450000}},
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active"}}}}
[junit4] 2> 62765 INFO
(TEST-TestComputePlanAction.testNodeLost-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.c.a.s.TestComputePlanAction Stopped_node : 127.0.0.1:10005_solr
[junit4] 2> 62775 DEBUG (ScheduledTrigger-154-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with
currently live nodes: 2
[junit4] 2> 62775 DEBUG (simCloudManagerPool-153-thread-7) [ ]
o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeLost /
shard1: {"core_node2":{
[junit4] 2> "node_name":"127.0.0.1:10000_solr",
[junit4] 2> "core":"testNodeLost_shard1_replica_n2",
[junit4] 2> "leader":"true",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInBytes":123450000,
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"testNodeLost"}}
[junit4] 2> 62775 DEBUG (ScheduledTrigger-154-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10005_solr
[junit4] 2> 62775 DEBUG (ScheduledTrigger-154-thread-4) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2
[junit4] 2> 62775 DEBUG (ScheduledTrigger-154-thread-4) [ ]
o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10005_solr
[junit4] 2> 62798 DEBUG (ScheduledTrigger-154-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with
currently live nodes: 2
[junit4] 2> 62798 DEBUG (ScheduledTrigger-154-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2
[junit4] 2> 62829 DEBUG (ScheduledTrigger-154-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with
currently live nodes: 2
[junit4] 2> 62829 DEBUG (ScheduledTrigger-154-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2
[junit4] 2> 62854 DEBUG (ScheduledTrigger-154-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with
currently live nodes: 2
[junit4] 2> 62854 DEBUG (ScheduledTrigger-154-thread-3) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2
[junit4] 2> 62881 DEBUG (ScheduledTrigger-154-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with
currently live nodes: 2
[junit4] 2> 62881 DEBUG (ScheduledTrigger-154-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2
[junit4] 2> 62912 DEBUG (ScheduledTrigger-154-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with
currently live nodes: 2
[junit4] 2> 62912 DEBUG (ScheduledTrigger-154-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2
[junit4] 2> 62944 DEBUG (ScheduledTrigger-154-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with
currently live nodes: 2
[junit4] 2> 62944 DEBUG (ScheduledTrigger-154-thread-2) [ ]
o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost
nodes: [127.0.0.1:10005_solr]
[junit4] 2> 62944 DEBUG (ScheduledTrigger-154-thread-4) [ ]
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with
currently live nodes: 2
[junit4] 2> 62946 DEBUG (ScheduledTrigger-154-thread-2) [ ]
o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"73075c0b2e81T7h5cqnja7ziypqi95z6l2x4sr",
[junit4] 2> "source":"node_lost_trigger",
[junit4] 2> "eventTime":126475446201985,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[126475446201985],
[junit4] 2> "nodeNames":["127.0.0.1:10005_solr"]}}
[junit4] 2> 62958 DEBUG (AutoscalingActionExecutor-155-thread-1) [ ]
o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"73075c0b2e81T7h5cqnja7ziypqi95z6l2x4sr",
[junit4] 2> "source":"node_lost_trigger",
[junit4] 2> "eventTime":126475446201985,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[126475446201985],
[junit4] 2> "_enqueue_time_":126484006645835,
[junit4] 2> "nodeNames":["127.0.0.1:10005_solr"]}}
[junit4] 2> 62958 DEBUG (AutoscalingActionExecutor-155-thread-1) [ ]
o.a.s.c.a.ComputePlanAction -- processing event: {
[junit4] 2> "id":"73075c0b2e81T7h5cqnja7ziypqi95z6l2x4sr",
[junit4] 2> "source":"node_lost_trigger",
[junit4] 2> "eventTime":126475446201985,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[126475446201985],
[junit4] 2> "_enqueue_time_":126484006645835,
[junit4] 2> "nodeNames":["127.0.0.1:10005_solr"]}} with context
properties: {BEFORE_ACTION=[compute_plan]}
[junit4] 2> 62958 DEBUG (AutoscalingActionExecutor-155-thread-1) [ ]
o.a.s.c.s.c.a.PolicyHelper Creating a new session
[junit4] 2> 62958 DEBUG (AutoscalingActionExecutor-155-thread-1) [ ]
o.a.s.c.s.c.a.PolicyHelper New session created
[junit4] 2> 62962 INFO (AutoscalingActionExecutor-155-thread-1) [ ]
o.a.s.c.a.ComputePlanAction Computed Plan:
action=MOVEREPLICA&collection=testNodeLost&targetNode=127.0.0.1:10006_solr&inPlaceMove=true&replica=core_node1
[junit4] 2> 62962 INFO (AutoscalingActionExecutor-155-thread-1) [ ]
o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 126484775
sessionWrapper.createTime 126484612903935, this.sessionWrapper.createTime
126484612903935
[junit4] 2> 62962 DEBUG (AutoscalingActionExecutor-155-thread-1) [ ]
o.a.s.c.s.c.a.PolicyHelper session set to NULL
[junit4] 2> 62973 INFO
(TEST-TestComputePlanAction.testNodeLost-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.c.a.s.TestComputePlanAction -------------_ FINAL STATE --------------
[junit4] 2> 62973 INFO
(TEST-TestComputePlanAction.testNodeLost-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.c.a.s.TestComputePlanAction * Node values: {
[junit4] 2> "127.0.0.1:10006_solr":{
[junit4] 2> "sysprop.java.vendor":"Oracle Corporation",
[junit4] 2> "cores":0,
[junit4] 2> "host":"127.0.0.1",
[junit4] 2> "freedisk":1000,
[junit4] 2> "sysprop.java.version":"1.8.0_144",
[junit4] 2> "ip_1":"127",
[junit4] 2> "ip_3":"0",
[junit4] 2> "ip_2":"0",
[junit4] 2> "ip_4":"1",
[junit4] 2> "node":"127.0.0.1:10006_solr",
[junit4] 2> "metrics:solr.jvm:buffers.direct.Count":0,
[junit4] 2> "port":10006,
[junit4] 2> "heapUsage":123450000,
[junit4] 2> "sysLoadAvg":1.0,
[junit4] 2>
"metrics:solr.node:ADMIN./admin/authorization.clientErrors:count":0},
[junit4] 2> "127.0.0.1:10005_solr":{
[junit4] 2> "sysprop.java.vendor":"Oracle Corporation",
[junit4] 2> "cores":1,
[junit4] 2> "host":"127.0.0.1",
[junit4] 2> "freedisk":990,
[junit4] 2> "sysprop.java.version":"1.8.0_144",
[junit4] 2> "ip_1":"127",
[junit4] 2> "ip_3":"0",
[junit4] 2> "ip_2":"0",
[junit4] 2> "ip_4":"1",
[junit4] 2> "node":"127.0.0.1:10005_solr",
[junit4] 2> "metrics:solr.jvm:buffers.direct.Count":0,
[junit4] 2> "port":10005,
[junit4] 2> "heapUsage":123450000,
[junit4] 2> "sysLoadAvg":1.0,
[junit4] 2>
"metrics:solr.node:ADMIN./admin/authorization.clientErrors:count":0},
[junit4] 2> "127.0.0.1:10000_solr":{
[junit4] 2> "sysprop.java.vendor":"Oracle Corporation",
[junit4] 2> "cores":1,
[junit4] 2> "host":"127.0.0.1",
[junit4] 2> "freedisk":960,
[junit4] 2> "sysprop.java.version":"1.8.0_144",
[junit4] 2> "ip_1":"127",
[junit4] 2> "ip_3":"0",
[junit4] 2> "ip_2":"0",
[junit4] 2> "ip_4":"1",
[junit4] 2> "node":"127.0.0.1:10000_solr",
[junit4] 2> "metrics:solr.jvm:buffers.direct.Count":0,
[junit4] 2> "port":10000,
[junit4] 2> "heapUsage":123450000,
[junit4] 2> "sysLoadAvg":1.0,
[junit4] 2>
"metrics:solr.node:ADMIN./admin/authorization.clientErrors:count":0}}
[junit4] 2> 62973 INFO
(TEST-TestComputePlanAction.testNodeLost-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.c.a.s.TestComputePlanAction * Live nodes: [127.0.0.1:10000_solr]
[junit4] 2> 62975 INFO
(TEST-TestComputePlanAction.testNodeLost-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.c.a.s.TestComputePlanAction * Collection testNodeLost state:
DocCollection(testNodeLost//clusterstate.json/11)={
[junit4] 2> "replicationFactor":"2",
[junit4] 2> "pullReplicas":"0",
[junit4] 2> "router":{"name":"compositeId"},
[junit4] 2> "maxShardsPerNode":"1",
[junit4] 2> "autoAddReplicas":"false",
[junit4] 2> "nrtReplicas":"2",
[junit4] 2> "tlogReplicas":"0",
[junit4] 2> "autoCreated":"true",
[junit4] 2> "shards":{"shard1":{
[junit4] 2> "replicas":{
[junit4] 2> "core_node1":{
[junit4] 2> "node_name":"127.0.0.1:10005_solr",
[junit4] 2> "core":"testNodeLost_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInBytes":123450000},
[junit4] 2> "core_node2":{
[junit4] 2> "core":"testNodeLost_shard1_replica_n2",
[junit4] 2> "leader":"true",
[junit4] 2> "INDEX.sizeInBytes":123450000,
[junit4] 2> "node_name":"127.0.0.1:10000_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT"}},
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active"}}}
[junit4] 2> 62975 INFO
(TEST-TestComputePlanAction.testNodeLost-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.SolrTestCaseJ4 ###Ending testNodeLost
[junit4] 2> 62976 INFO
(TEST-TestComputePlanAction.testNodeLost-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.c.a.s.SimSolrCloudTestCase
[junit4] 2>
[junit4] 2> 62976 INFO
(TEST-TestComputePlanAction.testNodeLost-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2> 62976 INFO
(TEST-TestComputePlanAction.testNodeLost-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.c.a.s.SimSolrCloudTestCase ############ FINAL CLUSTER STATS ############
[junit4] 2> 62976 INFO
(TEST-TestComputePlanAction.testNodeLost-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2>
[junit4] 2> 62976 INFO
(TEST-TestComputePlanAction.testNodeLost-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.c.a.s.SimSolrCloudTestCase ## Live nodes: 1
[junit4] 2> 62976 INFO
(TEST-TestComputePlanAction.testNodeLost-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.c.a.s.SimSolrCloudTestCase ## Empty nodes: 0
[junit4] 2> 62976 INFO
(TEST-TestComputePlanAction.testNodeLost-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.c.a.s.SimSolrCloudTestCase ## Dead nodes: 2
[junit4] 2> 62976 INFO
(TEST-TestComputePlanAction.testNodeLost-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.c.a.s.SimSolrCloudTestCase ## 127.0.0.1:10005_solr
[junit4] 2> 62976 INFO
(TEST-TestComputePlanAction.testNodeLost-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.c.a.s.SimSolrCloudTestCase ## 127.0.0.1:10006_solr
[junit4] 2> 62976 INFO
(TEST-TestComputePlanAction.testNodeLost-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.c.a.s.SimSolrCloudTestCase ## Collections: [testNodeLost]
[junit4] 2> 62976 INFO
(TEST-TestComputePlanAction.testNodeLost-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.c.a.s.SimSolrCloudTestCase ## Max replicas per node: 1
[junit4] 2> 62976 INFO
(TEST-TestComputePlanAction.testNodeLost-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.c.a.s.SimSolrCloudTestCase ## Min replicas per node: 1
[junit4] 2> 62976 INFO
(TEST-TestComputePlanAction.testNodeLost-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.c.a.s.SimSolrCloudTestCase ## Total replicas: 1
[junit4] 2> 62976 INFO
(TEST-TestComputePlanAction.testNodeLost-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.c.a.s.SimSolrCloudTestCase ## * testNodeLost 1
[junit4] 2> 62976 INFO
(TEST-TestComputePlanAction.testNodeLost-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.c.a.s.SimSolrCloudTestCase ## - active 1
[junit4] 2> 62976 INFO
(TEST-TestComputePlanAction.testNodeLost-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.c.a.s.SimSolrCloudTestCase ######### Final Solr op counts ##########
[junit4] 2> 62976 INFO
(TEST-TestComputePlanAction.testNodeLost-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.c.a.s.SimSolrCloudTestCase ## - CREATE 1
[junit4] 2> 62976 INFO
(TEST-TestComputePlanAction.testNodeLost-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.c.a.s.SimSolrCloudTestCase ## - autoscaling 3
[junit4] 2> 62976 INFO
(TEST-TestComputePlanAction.testNodeLost-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.c.a.s.SimSolrCloudTestCase ## - update 2
[junit4] 2> 62976 INFO
(TEST-TestComputePlanAction.testNodeLost-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.c.a.s.SimSolrCloudTestCase ######### Autoscaling event counts ###########
[junit4] 2> 62976 INFO
(TEST-TestComputePlanAction.testNodeLost-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.c.a.s.SimSolrCloudTestCase ## * Trigger: node_lost_trigger
[junit4] 2> 62976 INFO
(TEST-TestComputePlanAction.testNodeLost-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.c.a.s.SimSolrCloudTestCase ## - BEFORE_ACTION 1
[junit4] 2> 62976 INFO
(TEST-TestComputePlanAction.testNodeLost-seed#[CE91D734C7FAA56D]) [ ]
o.a.s.c.a.s.SimSolrCloudTestCase ## - STARTED 1
[junit4] 2> 62978 WARN (Simulated OverseerAutoScalingTriggerThread) [
] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are
closed, exiting.
[junit4] 2> 62991 WARN (AutoscalingActionExecutor-155-thread-1) [ ]
o.a.s.c.a.ScheduledTriggers Exception executing actions
[junit4] 2> org.apache.lucene.store.AlreadyClosedException:
ScheduledTrigger node_lost_trigger has been closed.
[junit4] 2> at
org.apache.solr.cloud.autoscaling.ScheduledTriggers$ScheduledTrigger.dequeue(ScheduledTriggers.java:493)
[junit4] 2> at
org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:305)
[junit4] 2> at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[junit4] 2> at
java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 62991 DEBUG
(SUITE-TestComputePlanAction-seed#[CE91D734C7FAA56D]-worker) [ ]
o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 62993 DEBUG (AutoscalingActionExecutor-155-thread-1) [ ]
o.a.s.c.a.ScheduledTriggers -- processing took 1750 ms for event
id=73075c0b2e81T7h5cqnja7ziypqi95z6l2x4sr
[junit4] 2> NOTE: leaving temporary files on disk at:
/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.sim.TestComputePlanAction_CE91D734C7FAA56D-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70),
sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@58dff434),
locale=sq-AL, timezone=America/Indiana/Marengo
[junit4] 2> NOTE: Mac OS X 10.11.6 x86_64/Oracle Corporation 1.8.0_144
(64-bit)/cpus=3,threads=1,free=251250104,total=536870912
[junit4] 2> NOTE: All tests run in this JVM: [QueryResultKeyTest,
TestCloudSchemaless, TestComputePlanAction]
[junit4] Completed [7/766 (1!)] on J0 in 2.07s, 3 tests, 1 error <<<
FAILURES!
[...truncated 50618 lines...]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
