Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2377/
1 tests failed. FAILED: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testEventQueue Error Message: action wasn't interrupted Stack Trace: java.lang.AssertionError: action wasn't interrupted at __randomizedtesting.SeedInfo.seed([5A2E7CCD051D1093:939B3E630C7AD666]:0) at org.junit.Assert.fail(Assert.java:93) at org.junit.Assert.assertTrue(Assert.java:43) at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testEventQueue(TriggerIntegrationTest.java:726) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934) at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970) at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368) at java.lang.Thread.run(Thread.java:748) Build Log: [...truncated 12804 lines...] [junit4] Suite: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest [junit4] 2> 513991 INFO (SUITE-TriggerIntegrationTest-seed#[5A2E7CCD051D1093]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom [junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_5A2E7CCD051D1093-001/init-core-data-001 [junit4] 2> 513992 WARN (SUITE-TriggerIntegrationTest-seed#[5A2E7CCD051D1093]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=2 numCloses=2 [junit4] 2> 513992 INFO (SUITE-TriggerIntegrationTest-seed#[5A2E7CCD051D1093]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false [junit4] 2> 513993 INFO (SUITE-TriggerIntegrationTest-seed#[5A2E7CCD051D1093]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN) [junit4] 2> 513993 INFO (SUITE-TriggerIntegrationTest-seed#[5A2E7CCD051D1093]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_5A2E7CCD051D1093-001/tempDir-001 [junit4] 2> 513994 INFO (SUITE-TriggerIntegrationTest-seed#[5A2E7CCD051D1093]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER [junit4] 2> 513994 INFO (Thread-869) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0 [junit4] 2> 513994 INFO (Thread-869) [ ] o.a.s.c.ZkTestServer Starting server [junit4] 2> 514000 ERROR (Thread-869) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes [junit4] 2> 514094 INFO (SUITE-TriggerIntegrationTest-seed#[5A2E7CCD051D1093]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:36380 [junit4] 2> 514096 INFO (zkConnectionManagerCallback-925-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 514098 INFO (jetty-launcher-922-thread-1) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T21:27:37Z, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8 [junit4] 2> 514098 INFO (jetty-launcher-922-thread-2) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T21:27:37Z, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8 [junit4] 2> 514099 INFO (jetty-launcher-922-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0 [junit4] 2> 514099 INFO (jetty-launcher-922-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults [junit4] 2> 514099 INFO (jetty-launcher-922-thread-1) [ ] o.e.j.s.session Scavenging every 600000ms [junit4] 2> 514099 INFO (jetty-launcher-922-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0 [junit4] 2> 514099 INFO (jetty-launcher-922-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults [junit4] 2> 514099 INFO (jetty-launcher-922-thread-2) [ ] o.e.j.s.session Scavenging every 600000ms [junit4] 2> 514100 INFO (jetty-launcher-922-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@a8a5538{/solr,null,AVAILABLE} [junit4] 2> 514100 INFO (jetty-launcher-922-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@df5d073{/solr,null,AVAILABLE} [junit4] 2> 514101 INFO (jetty-launcher-922-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@6a36538e{HTTP/1.1,[http/1.1]}{127.0.0.1:47025} [junit4] 2> 514101 INFO (jetty-launcher-922-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@73c2aaa1{HTTP/1.1,[http/1.1]}{127.0.0.1:46988} [junit4] 2> 514101 INFO (jetty-launcher-922-thread-1) [ ] o.e.j.s.Server Started @516317ms [junit4] 2> 514101 INFO (jetty-launcher-922-thread-2) [ ] o.e.j.s.Server Started @516317ms [junit4] 2> 514101 INFO (jetty-launcher-922-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=47025} [junit4] 2> 514101 INFO (jetty-launcher-922-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=46988} [junit4] 2> 514101 ERROR (jetty-launcher-922-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete. [junit4] 2> 514101 ERROR (jetty-launcher-922-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete. [junit4] 2> 514101 INFO (jetty-launcher-922-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0 [junit4] 2> 514101 INFO (jetty-launcher-922-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0 [junit4] 2> 514101 INFO (jetty-launcher-922-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null [junit4] 2> 514101 INFO (jetty-launcher-922-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null [junit4] 2> 514101 INFO (jetty-launcher-922-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null [junit4] 2> 514101 INFO (jetty-launcher-922-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null [junit4] 2> 514101 INFO (jetty-launcher-922-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-25T19:15:47.025Z [junit4] 2> 514102 INFO (jetty-launcher-922-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-25T19:15:47.025Z [junit4] 2> 514103 INFO (zkConnectionManagerCallback-927-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 514103 INFO (jetty-launcher-922-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading... [junit4] 2> 514111 INFO (jetty-launcher-922-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:36380/solr [junit4] 2> 514112 INFO (zkConnectionManagerCallback-933-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 514113 INFO (zkConnectionManagerCallback-929-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 514113 INFO (jetty-launcher-922-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading... [junit4] 2> 514115 INFO (zkConnectionManagerCallback-935-thread-1-processing-n:127.0.0.1:47025_solr) [n:127.0.0.1:47025_solr ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 514132 INFO (jetty-launcher-922-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:36380/solr [junit4] 2> 514133 INFO (zkConnectionManagerCallback-940-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 514137 INFO (zkConnectionManagerCallback-942-thread-1-processing-n:127.0.0.1:46988_solr) [n:127.0.0.1:46988_solr ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 514146 INFO (jetty-launcher-922-thread-2) [n:127.0.0.1:46988_solr ] o.a.s.c.Overseer Overseer (id=null) closing [junit4] 2> 514146 INFO (jetty-launcher-922-thread-2) [n:127.0.0.1:46988_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:46988_solr [junit4] 2> 514147 INFO (jetty-launcher-922-thread-2) [n:127.0.0.1:46988_solr ] o.a.s.c.Overseer Overseer (id=73357700753915910-127.0.0.1:46988_solr-n_0000000000) starting [junit4] 2> 514155 INFO (jetty-launcher-922-thread-2) [n:127.0.0.1:46988_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46988_solr [junit4] 2> 514156 DEBUG (OverseerAutoScalingTriggerThread-73357700753915910-127.0.0.1:46988_solr-n_0000000000) [n:127.0.0.1:46988_solr ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger [junit4] 2> 514160 INFO (OverseerStateUpdate-73357700753915910-127.0.0.1:46988_solr-n_0000000000) [n:127.0.0.1:46988_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 514161 DEBUG (OverseerAutoScalingTriggerThread-73357700753915910-127.0.0.1:46988_solr-n_0000000000) [n:127.0.0.1:46988_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1 [junit4] 2> 514161 DEBUG (OverseerAutoScalingTriggerThread-73357700753915910-127.0.0.1:46988_solr-n_0000000000) [n:127.0.0.1:46988_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:46988_solr] [junit4] 2> 514161 DEBUG (OverseerAutoScalingTriggerThread-73357700753915910-127.0.0.1:46988_solr-n_0000000000) [n:127.0.0.1:46988_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1 [junit4] 2> 514161 DEBUG (OverseerAutoScalingTriggerThread-73357700753915910-127.0.0.1:46988_solr-n_0000000000) [n:127.0.0.1:46988_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1 [junit4] 2> 514164 DEBUG (OverseerAutoScalingTriggerThread-73357700753915910-127.0.0.1:46988_solr-n_0000000000) [n:127.0.0.1:46988_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers [junit4] 2> 514164 DEBUG (OverseerAutoScalingTriggerThread-73357700753915910-127.0.0.1:46988_solr-n_0000000000) [n:127.0.0.1:46988_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1 [junit4] 2> 514165 DEBUG (ScheduledTrigger-1901-thread-1) [n:127.0.0.1:46988_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1 [junit4] 2> 514207 INFO (jetty-launcher-922-thread-1) [n:127.0.0.1:47025_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 514209 INFO (jetty-launcher-922-thread-1) [n:127.0.0.1:47025_solr ] o.a.s.c.Overseer Overseer (id=null) closing [junit4] 2> 514211 INFO (jetty-launcher-922-thread-1) [n:127.0.0.1:47025_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:47025_solr [junit4] 2> 514212 INFO (zkCallback-941-thread-1-processing-n:127.0.0.1:46988_solr) [n:127.0.0.1:46988_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2) [junit4] 2> 514220 INFO (zkCallback-934-thread-1-processing-n:127.0.0.1:47025_solr) [n:127.0.0.1:47025_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2) [junit4] 2> 514250 INFO (jetty-launcher-922-thread-2) [n:127.0.0.1:46988_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46988.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4debf922 [junit4] 2> 514257 INFO (jetty-launcher-922-thread-2) [n:127.0.0.1:46988_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46988.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4debf922 [junit4] 2> 514257 INFO (jetty-launcher-922-thread-2) [n:127.0.0.1:46988_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46988.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4debf922 [junit4] 2> 514258 INFO (jetty-launcher-922-thread-2) [n:127.0.0.1:46988_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_5A2E7CCD051D1093-001/tempDir-001/node2/. [junit4] 2> 514262 INFO (jetty-launcher-922-thread-1) [n:127.0.0.1:47025_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_47025.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4debf922 [junit4] 2> 514270 INFO (jetty-launcher-922-thread-1) [n:127.0.0.1:47025_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_47025.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4debf922 [junit4] 2> 514270 INFO (jetty-launcher-922-thread-1) [n:127.0.0.1:47025_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_47025.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4debf922 [junit4] 2> 514271 INFO (jetty-launcher-922-thread-1) [n:127.0.0.1:47025_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_5A2E7CCD051D1093-001/tempDir-001/node1/. [junit4] 2> 514301 INFO (zkConnectionManagerCallback-947-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 514303 INFO (zkConnectionManagerCallback-951-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 514304 INFO (SUITE-TriggerIntegrationTest-seed#[5A2E7CCD051D1093]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2) [junit4] 2> 514305 INFO (SUITE-TriggerIntegrationTest-seed#[5A2E7CCD051D1093]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:36380/solr ready [junit4] 2> 514322 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testNodeMarkersRegistration [junit4] 2> 514324 INFO (qtp556248361-4305) [n:127.0.0.1:47025_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :overseerstatus with params action=OVERSEERSTATUS&wt=javabin&version=2 and sendToOCPQueue=true [junit4] 2> 514329 INFO (qtp556248361-4305) [n:127.0.0.1:47025_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=OVERSEERSTATUS&wt=javabin&version=2} status=0 QTime=5 [junit4] 2> 514330 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.c.a.ScheduledTriggers -- removing trigger: .auto_add_replicas [junit4] 2> 514331 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.c.a.TriggerIntegrationTest /autoscaling.json reset, new znode version 2 [junit4] 2> 514332 DEBUG (zkCallback-941-thread-1-processing-n:127.0.0.1:46988_solr) [n:127.0.0.1:46988_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2 [junit4] 2> 514332 DEBUG (OverseerAutoScalingTriggerThread-73357700753915910-127.0.0.1:46988_solr-n_0000000000) [n:127.0.0.1:46988_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2 [junit4] 2> 514332 DEBUG (OverseerAutoScalingTriggerThread-73357700753915910-127.0.0.1:46988_solr-n_0000000000) [n:127.0.0.1:46988_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers [junit4] 2> 514332 DEBUG (OverseerAutoScalingTriggerThread-73357700753915910-127.0.0.1:46988_solr-n_0000000000) [n:127.0.0.1:46988_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers [junit4] 2> 514333 DEBUG (OverseerAutoScalingTriggerThread-73357700753915910-127.0.0.1:46988_solr-n_0000000000) [n:127.0.0.1:46988_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2 [junit4] 2> 514333 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2) [junit4] 2> 514334 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@73c2aaa1{HTTP/1.1,[http/1.1]}{127.0.0.1:0} [junit4] 2> 514334 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=962004601 [junit4] 2> 514334 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null [junit4] 2> 514335 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1652d50b: rootName = solr_46988, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@18d66963 [junit4] 2> 514341 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null [junit4] 2> 514341 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@6fdd068f: rootName = solr_46988, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@3f6717f [junit4] 2> 514346 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null [junit4] 2> 514346 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@67602787: rootName = solr_46988, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@15162d18 [junit4] 2> 514347 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:46988_solr [junit4] 2> 514347 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null [junit4] 2> 514347 INFO (zkCallback-941-thread-1-processing-n:127.0.0.1:46988_solr) [n:127.0.0.1:46988_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1) [junit4] 2> 514348 INFO (zkCallback-934-thread-1-processing-n:127.0.0.1:47025_solr) [n:127.0.0.1:47025_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1) [junit4] 2> 514348 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores [junit4] 2> 514350 INFO (zkCallback-950-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1) [junit4] 2> 514350 INFO (zkCallback-950-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1) [junit4] 2> 514353 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.c.Overseer Overseer (id=73357700753915910-127.0.0.1:46988_solr-n_0000000000) closing [junit4] 2> 514353 INFO (OverseerStateUpdate-73357700753915910-127.0.0.1:46988_solr-n_0000000000) [n:127.0.0.1:46988_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:46988_solr [junit4] 2> 514353 WARN (OverseerAutoScalingTriggerThread-73357700753915910-127.0.0.1:46988_solr-n_0000000000) [n:127.0.0.1:46988_solr ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting. [junit4] 2> 514353 DEBUG (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly [junit4] 2> 514354 INFO (zkCallback-934-thread-1-processing-n:127.0.0.1:47025_solr) [n:127.0.0.1:47025_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:47025_solr [junit4] 2> 514354 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@df5d073{/solr,null,UNAVAILABLE} [junit4] 2> 514355 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.e.j.s.session Stopped scavenging [junit4] 2> 514355 INFO (zkCallback-934-thread-1-processing-n:127.0.0.1:47025_solr) [n:127.0.0.1:47025_solr ] o.a.s.c.Overseer Overseer (id=73357700753915908-127.0.0.1:47025_solr-n_0000000001) starting [junit4] 2> 514360 DEBUG (OverseerAutoScalingTriggerThread-73357700753915908-127.0.0.1:47025_solr-n_0000000001) [n:127.0.0.1:47025_solr ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger [junit4] 2> 514360 DEBUG (OverseerAutoScalingTriggerThread-73357700753915908-127.0.0.1:47025_solr-n_0000000001) [n:127.0.0.1:47025_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3 [junit4] 2> 514361 DEBUG (OverseerAutoScalingTriggerThread-73357700753915908-127.0.0.1:47025_solr-n_0000000001) [n:127.0.0.1:47025_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:47025_solr] [junit4] 2> 514361 DEBUG (OverseerAutoScalingTriggerThread-73357700753915908-127.0.0.1:47025_solr-n_0000000001) [n:127.0.0.1:47025_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion -1 [junit4] 2> 514361 DEBUG (OverseerAutoScalingTriggerThread-73357700753915908-127.0.0.1:47025_solr-n_0000000001) [n:127.0.0.1:47025_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3 [junit4] 2> 514362 DEBUG (OverseerAutoScalingTriggerThread-73357700753915908-127.0.0.1:47025_solr-n_0000000001) [n:127.0.0.1:47025_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers [junit4] 2> 514362 DEBUG (OverseerAutoScalingTriggerThread-73357700753915908-127.0.0.1:47025_solr-n_0000000001) [n:127.0.0.1:47025_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3 [junit4] 2> 514363 DEBUG (ScheduledTrigger-1910-thread-1) [n:127.0.0.1:47025_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1 [junit4] 2> 515363 DEBUG (ScheduledTrigger-1910-thread-1) [n:127.0.0.1:47025_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1 [junit4] 2> 516365 DEBUG (ScheduledTrigger-1910-thread-2) [n:127.0.0.1:47025_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1 [junit4] 2> 517365 DEBUG (ScheduledTrigger-1910-thread-1) [n:127.0.0.1:47025_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1 [junit4] 2> 518365 DEBUG (ScheduledTrigger-1910-thread-1) [n:127.0.0.1:47025_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1 [junit4] 2> 519356 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T21:27:37Z, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8 [junit4] 2> 519357 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0 [junit4] 2> 519357 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.e.j.s.session No SessionScavenger set, using defaults [junit4] 2> 519357 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.e.j.s.session Scavenging every 600000ms [junit4] 2> 519358 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6ac06f1b{/solr,null,AVAILABLE} [junit4] 2> 519358 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@773a1118{HTTP/1.1,[http/1.1]}{127.0.0.1:40277} [junit4] 2> 519358 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.e.j.s.Server Started @521574ms [junit4] 2> 519358 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=40277} [junit4] 2> 519359 ERROR (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete. [junit4] 2> 519360 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0 [junit4] 2> 519360 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null [junit4] 2> 519360 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null [junit4] 2> 519360 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-25T19:15:52.284Z [junit4] 2> 519365 DEBUG (ScheduledTrigger-1910-thread-1) [n:127.0.0.1:47025_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1 [junit4] 2> 519366 INFO (zkConnectionManagerCallback-954-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 519366 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading... [junit4] 2> 519374 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:36380/solr [junit4] 2> 519375 INFO (zkConnectionManagerCallback-958-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 519378 INFO (zkConnectionManagerCallback-960-thread-1-processing-n:127.0.0.1:40277_solr) [n:127.0.0.1:40277_solr ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 519382 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [n:127.0.0.1:40277_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 519383 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [n:127.0.0.1:40277_solr ] o.a.s.c.Overseer Overseer (id=null) closing [junit4] 2> 519384 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [n:127.0.0.1:40277_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40277_solr [junit4] 2> 519386 INFO (zkCallback-950-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2) [junit4] 2> 519386 INFO (zkCallback-950-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2) [junit4] 2> 519386 INFO (zkCallback-934-thread-1-processing-n:127.0.0.1:47025_solr) [n:127.0.0.1:47025_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2) [junit4] 2> 519389 INFO (zkCallback-959-thread-1-processing-n:127.0.0.1:40277_solr) [n:127.0.0.1:40277_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2) [junit4] 2> 519462 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [n:127.0.0.1:40277_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40277.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4debf922 [junit4] 2> 519470 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [n:127.0.0.1:40277_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40277.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4debf922 [junit4] 2> 519470 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [n:127.0.0.1:40277_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40277.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4debf922 [junit4] 2> 519472 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [n:127.0.0.1:40277_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_5A2E7CCD051D1093-001/tempDir-001/node3/. [junit4] 2> 519504 INFO (qtp161197342-4378) [n:127.0.0.1:40277_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :overseerstatus with params action=OVERSEERSTATUS&wt=javabin&version=2 and sendToOCPQueue=true [junit4] 2> 519510 INFO (qtp161197342-4378) [n:127.0.0.1:40277_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=OVERSEERSTATUS&wt=javabin&version=2} status=0 QTime=6 [junit4] 2> 519511 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T21:27:37Z, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8 [junit4] 2> 519512 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0 [junit4] 2> 519512 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.e.j.s.session No SessionScavenger set, using defaults [junit4] 2> 519512 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.e.j.s.session Scavenging every 660000ms [junit4] 2> 519513 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1c8c1604{/solr,null,AVAILABLE} [junit4] 2> 519513 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@5e75ef24{HTTP/1.1,[http/1.1]}{127.0.0.1:37968} [junit4] 2> 519513 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.e.j.s.Server Started @521729ms [junit4] 2> 519513 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=37968} [junit4] 2> 519514 ERROR (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete. [junit4] 2> 519514 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0 [junit4] 2> 519514 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null [junit4] 2> 519514 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null [junit4] 2> 519514 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-25T19:15:52.438Z [junit4] 2> 519520 INFO (zkConnectionManagerCallback-964-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 519521 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading... [junit4] 2> 519528 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:36380/solr [junit4] 2> 519529 INFO (zkConnectionManagerCallback-968-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 519532 INFO (zkConnectionManagerCallback-970-thread-1-processing-n:127.0.0.1:37968_solr) [n:127.0.0.1:37968_solr ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 519536 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [n:127.0.0.1:37968_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2) [junit4] 2> 519538 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [n:127.0.0.1:37968_solr ] o.a.s.c.Overseer Overseer (id=null) closing [junit4] 2> 519539 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [n:127.0.0.1:37968_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37968_solr [junit4] 2> 519540 INFO (zkCallback-950-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3) [junit4] 2> 519540 INFO (zkCallback-950-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3) [junit4] 2> 519541 INFO (zkCallback-959-thread-1-processing-n:127.0.0.1:40277_solr) [n:127.0.0.1:40277_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3) [junit4] 2> 519541 INFO (zkCallback-934-thread-1-processing-n:127.0.0.1:47025_solr) [n:127.0.0.1:47025_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3) [junit4] 2> 519547 INFO (zkCallback-969-thread-1-processing-n:127.0.0.1:37968_solr) [n:127.0.0.1:37968_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3) [junit4] 2> 519656 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [n:127.0.0.1:37968_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37968.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4debf922 [junit4] 2> 519665 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [n:127.0.0.1:37968_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37968.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4debf922 [junit4] 2> 519666 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [n:127.0.0.1:37968_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37968.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4debf922 [junit4] 2> 519667 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [n:127.0.0.1:37968_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_5A2E7CCD051D1093-001/tempDir-001/node4/. [junit4] 2> 519703 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.c.a.TriggerIntegrationTest ====== KILL OVERSEER 1 [junit4] 2> 519704 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@6a36538e{HTTP/1.1,[http/1.1]}{127.0.0.1:0} [junit4] 2> 519705 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=885627216 [junit4] 2> 519705 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null [junit4] 2> 519705 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5191fbca: rootName = solr_47025, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@49fa54ae [junit4] 2> 519712 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null [junit4] 2> 519712 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@176623dc: rootName = solr_47025, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@3f6717f [junit4] 2> 519716 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null [junit4] 2> 519716 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@23f3bc5b: rootName = solr_47025, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@15162d18 [junit4] 2> 519717 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:47025_solr [junit4] 2> 519718 INFO (zkCallback-969-thread-1-processing-n:127.0.0.1:37968_solr) [n:127.0.0.1:37968_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2) [junit4] 2> 519718 INFO (zkCallback-959-thread-1-processing-n:127.0.0.1:40277_solr) [n:127.0.0.1:40277_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2) [junit4] 2> 519718 INFO (zkCallback-934-thread-1-processing-n:127.0.0.1:47025_solr) [n:127.0.0.1:47025_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2) [junit4] 2> 519718 INFO (zkCallback-950-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2) [junit4] 2> 519718 INFO (zkCallback-950-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2) [junit4] 2> 519718 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null [junit4] 2> 519719 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores [junit4] 2> 519719 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.c.Overseer Overseer (id=73357700753915908-127.0.0.1:47025_solr-n_0000000001) closing [junit4] 2> 519719 INFO (OverseerStateUpdate-73357700753915908-127.0.0.1:47025_solr-n_0000000001) [n:127.0.0.1:47025_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:47025_solr [junit4] 2> 519721 WARN (OverseerAutoScalingTriggerThread-73357700753915908-127.0.0.1:47025_solr-n_0000000001) [n:127.0.0.1:47025_solr ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting. [junit4] 2> 519721 DEBUG (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly [junit4] 2> 519723 INFO (zkCallback-959-thread-1-processing-n:127.0.0.1:40277_solr) [n:127.0.0.1:40277_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:40277_solr [junit4] 2> 519723 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@a8a5538{/solr,null,UNAVAILABLE} [junit4] 2> 519723 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.e.j.s.session Stopped scavenging [junit4] 2> 519724 INFO (zkCallback-959-thread-1-processing-n:127.0.0.1:40277_solr) [n:127.0.0.1:40277_solr ] o.a.s.c.Overseer Overseer (id=73357700753915915-127.0.0.1:40277_solr-n_0000000002) starting [junit4] 2> 519728 DEBUG (OverseerAutoScalingTriggerThread-73357700753915915-127.0.0.1:40277_solr-n_0000000002) [n:127.0.0.1:40277_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3 [junit4] 2> 519729 DEBUG (OverseerAutoScalingTriggerThread-73357700753915915-127.0.0.1:40277_solr-n_0000000002) [n:127.0.0.1:40277_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:37968_solr, 127.0.0.1:40277_solr] [junit4] 2> 519729 DEBUG (OverseerAutoScalingTriggerThread-73357700753915915-127.0.0.1:40277_solr-n_0000000002) [n:127.0.0.1:40277_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion -1 [junit4] 2> 519729 DEBUG (OverseerAutoScalingTriggerThread-73357700753915915-127.0.0.1:40277_solr-n_0000000002) [n:127.0.0.1:40277_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3 [junit4] 2> 519732 DEBUG (OverseerAutoScalingTriggerThread-73357700753915915-127.0.0.1:40277_solr-n_0000000002) [n:127.0.0.1:40277_solr ] o.a.s.c.a.NodeLostTrigger Adding lost node from marker path: 127.0.0.1:47025_solr [junit4] 2> 519733 DEBUG (OverseerAutoScalingTriggerThread-73357700753915915-127.0.0.1:40277_solr-n_0000000002) [n:127.0.0.1:40277_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers [junit4] 2> 519733 DEBUG (OverseerAutoScalingTriggerThread-73357700753915915-127.0.0.1:40277_solr-n_0000000002) [n:127.0.0.1:40277_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3 [junit4] 2> 519733 DEBUG (ScheduledTrigger-1927-thread-1) [n:127.0.0.1:40277_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 [junit4] 2> 520734 DEBUG (ScheduledTrigger-1927-thread-1) [n:127.0.0.1:40277_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 [junit4] 2> 521735 DEBUG (ScheduledTrigger-1927-thread-2) [n:127.0.0.1:40277_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 [junit4] 2> 522736 DEBUG (ScheduledTrigger-1927-thread-1) [n:127.0.0.1:40277_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 [junit4] 2> 523736 DEBUG (ScheduledTrigger-1927-thread-1) [n:127.0.0.1:40277_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 [junit4] 2> 524733 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.c.a.TriggerIntegrationTest ====== ADD TRIGGERS [junit4] 2> 524736 DEBUG (ScheduledTrigger-1927-thread-1) [n:127.0.0.1:40277_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 [junit4] 2> 524752 INFO (qtp1728949472-4407) [n:127.0.0.1:37968_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0 [junit4] 2> 524755 INFO (qtp161197342-4381) [n:127.0.0.1:40277_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0 [junit4] 2> 524755 DEBUG (qtp1728949472-4403) [n:127.0.0.1:37968_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration [junit4] 2> 524756 INFO (qtp1728949472-4403) [n:127.0.0.1:37968_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=7 [junit4] 2> 524756 DEBUG (zkCallback-959-thread-1-processing-n:127.0.0.1:40277_solr) [n:127.0.0.1:40277_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4 [junit4] 2> 524759 DEBUG (zkCallback-959-thread-1-processing-n:127.0.0.1:40277_solr) [n:127.0.0.1:40277_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:37968_solr, 127.0.0.1:40277_solr] [junit4] 2> 524759 DEBUG (zkCallback-959-thread-1-processing-n:127.0.0.1:40277_solr) [n:127.0.0.1:40277_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_triggerMR instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestEventMarkerAction}], enabled=true} [junit4] 2> 524759 DEBUG (zkCallback-959-thread-1-processing-n:127.0.0.1:40277_solr) [n:127.0.0.1:40277_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:37968_solr, 127.0.0.1:40277_solr] [junit4] 2> 524759 DEBUG (OverseerAutoScalingTriggerThread-73357700753915915-127.0.0.1:40277_solr-n_0000000002) [n:127.0.0.1:40277_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4 [junit4] 2> 524760 INFO (qtp1728949472-4407) [n:127.0.0.1:37968_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0 [junit4] 2> 524761 INFO (qtp161197342-4383) [n:127.0.0.1:40277_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0 [junit4] 2> 524761 INFO (OverseerAutoScalingTriggerThread-73357700753915915-127.0.0.1:40277_solr-n_0000000002) [n:127.0.0.1:40277_solr ] o.a.s.c.a.TriggerIntegrationTest TestEventMarkerAction init [junit4] 2> 524762 DEBUG (qtp161197342-4379) [n:127.0.0.1:40277_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration [junit4] 2> 524762 INFO (qtp161197342-4379) [n:127.0.0.1:40277_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=4 [junit4] 2> 524763 DEBUG (zkCallback-959-thread-1-processing-n:127.0.0.1:40277_solr) [n:127.0.0.1:40277_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5 [junit4] 2> 524763 INFO (qtp1728949472-4405) [n:127.0.0.1:37968_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :overseerstatus with params action=OVERSEERSTATUS&wt=javabin&version=2 and sendToOCPQueue=true [junit4] 2> 524763 DEBUG (zkCallback-959-thread-1-processing-n:127.0.0.1:40277_solr) [n:127.0.0.1:40277_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:37968_solr, 127.0.0.1:40277_solr] [junit4] 2> 524763 DEBUG (zkCallback-959-thread-1-processing-n:127.0.0.1:40277_solr) [n:127.0.0.1:40277_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:37968_solr, 127.0.0.1:40277_solr] [junit4] 2> 524763 DEBUG (zkCallback-959-thread-1-processing-n:127.0.0.1:40277_solr) [n:127.0.0.1:40277_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_triggerMR instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestEventMarkerAction}], enabled=true} [junit4] 2> 524764 DEBUG (zkCallback-959-thread-1-processing-n:127.0.0.1:40277_solr) [n:127.0.0.1:40277_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:37968_solr, 127.0.0.1:40277_solr] [junit4] 2> 524764 DEBUG (OverseerAutoScalingTriggerThread-73357700753915915-127.0.0.1:40277_solr-n_0000000002) [n:127.0.0.1:40277_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 4 [junit4] 2> 524764 DEBUG (ScheduledTrigger-1927-thread-3) [n:127.0.0.1:40277_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerMR [junit4] 2> 524764 DEBUG (OverseerAutoScalingTriggerThread-73357700753915915-127.0.0.1:40277_solr-n_0000000002) [n:127.0.0.1:40277_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5 [junit4] 2> 524765 DEBUG (ScheduledTrigger-1927-thread-3) [n:127.0.0.1:40277_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2 [junit4] 2> 524766 INFO (OverseerAutoScalingTriggerThread-73357700753915915-127.0.0.1:40277_solr-n_0000000002) [n:127.0.0.1:40277_solr ] o.a.s.c.a.TriggerIntegrationTest TestEventMarkerAction init [junit4] 2> 524767 DEBUG (OverseerAutoScalingTriggerThread-73357700753915915-127.0.0.1:40277_solr-n_0000000002) [n:127.0.0.1:40277_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5 [junit4] 2> 524767 DEBUG (ScheduledTrigger-1927-thread-1) [n:127.0.0.1:40277_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_triggerMR with currently live nodes: 2 [junit4] 2> 524773 INFO (qtp1728949472-4405) [n:127.0.0.1:37968_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=OVERSEERSTATUS&wt=javabin&version=2} status=0 QTime=9 [junit4] 2> 524773 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.c.a.TriggerIntegrationTest ====== ADD NODE 1 [junit4] 2> 524773 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T21:27:37Z, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8 [junit4] 2> 524774 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0 [junit4] 2> 524774 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.e.j.s.session No SessionScavenger set, using defaults [junit4] 2> 524774 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.e.j.s.session Scavenging every 660000ms [junit4] 2> 524775 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@40b58257{/solr,null,AVAILABLE} [junit4] 2> 524775 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@356a1f5d{HTTP/1.1,[http/1.1]}{127.0.0.1:45977} [junit4] 2> 524775 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.e.j.s.Server Started @526991ms [junit4] 2> 524775 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=45977} [junit4] 2> 524775 ERROR (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete. [junit4] 2> 524776 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0 [junit4] 2> 524776 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null [junit4] 2> 524776 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null [junit4] 2> 524776 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-25T19:15:57.700Z [junit4] 2> 524777 INFO (zkConnectionManagerCallback-974-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 524778 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading... [junit4] 2> 524787 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:36380/solr [junit4] 2> 524788 INFO (zkConnectionManagerCallback-978-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 524791 INFO (zkConnectionManagerCallback-980-thread-1-processing-n:127.0.0.1:45977_solr) [n:127.0.0.1:45977_solr ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 524797 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [n:127.0.0.1:45977_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2) [junit4] 2> 524801 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [n:127.0.0.1:45977_solr ] o.a.s.c.Overseer Overseer (id=null) closing [junit4] 2> 524803 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [n:127.0.0.1:45977_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45977_solr [junit4] 2> 524806 INFO (zkCallback-950-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3) [junit4] 2> 524806 INFO (zkCallback-969-thread-1-processing-n:127.0.0.1:37968_solr) [n:127.0.0.1:37968_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3) [junit4] 2> 524807 INFO (zkCallback-950-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3) [junit4] 2> 524808 INFO (zkCallback-979-thread-1-processing-n:127.0.0.1:45977_solr) [n:127.0.0.1:45977_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3) [junit4] 2> 524812 INFO (zkCallback-959-thread-2-processing-n:127.0.0.1:40277_solr) [n:127.0.0.1:40277_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3) [junit4] 2> 524903 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [n:127.0.0.1:45977_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45977.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4debf922 [junit4] 2> 524912 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [n:127.0.0.1:45977_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45977.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4debf922 [junit4] 2> 524912 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [n:127.0.0.1:45977_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45977.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4debf922 [junit4] 2> 524913 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [n:127.0.0.1:45977_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_5A2E7CCD051D1093-001/tempDir-001/node5/. [junit4] 2> 525737 DEBUG (ScheduledTrigger-1927-thread-4) [n:127.0.0.1:40277_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 [junit4] 2> 525767 DEBUG (ScheduledTrigger-1927-thread-2) [n:127.0.0.1:40277_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerMR [junit4] 2> 525767 DEBUG (ScheduledTrigger-1927-thread-2) [n:127.0.0.1:40277_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 525767 DEBUG (ScheduledTrigger-1927-thread-2) [n:127.0.0.1:40277_solr ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:45977_solr at time 19838065911079552 [junit4] 2> 525767 DEBUG (ScheduledTrigger-1927-thread-1) [n:127.0.0.1:40277_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_triggerMR with currently live nodes: 3 [junit4] 2> 526738 DEBUG (ScheduledTrigger-1927-thread-3) [n:127.0.0.1:40277_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 [junit4] 2> 526768 DEBUG (ScheduledTrigger-1927-thread-4) [n:127.0.0.1:40277_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerMR [junit4] 2> 526768 DEBUG (ScheduledTrigger-1927-thread-4) [n:127.0.0.1:40277_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3 [junit4] 2> 526769 DEBUG (ScheduledTrigger-1927-thread-4) [n:127.0.0.1:40277_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_triggerMR firing registered processor for nodes: [127.0.0.1:45977_solr] added at times [19838065911079552], now=19838066913136771 [junit4] 2> 526769 DEBUG (ScheduledTrigger-1927-thread-2) [n:127.0.0.1:40277_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_triggerMR with currently live nodes: 3 [junit4] 2> 526770 DEBUG (ScheduledTrigger-1927-thread-4) [n:127.0.0.1:40277_solr ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: { [junit4] 2> "id":"467a9da7a92280T6zq3j0642to5x4wf8gy6w4yyi", [junit4] 2> "source":"node_added_triggerMR", [junit4] 2> "eventTime":19838065911079552, [junit4] 2> "eventType":"NODEADDED", [junit4] 2> "properties":{ [junit4] 2> "eventTimes":[19838065911079552], [junit4] 2> "nodeNames":["127.0.0.1:45977_solr"]}} [junit4] 2> 526770 DEBUG (ScheduledTrigger-1927-thread-4) [n:127.0.0.1:40277_solr ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_lost_triggerMR, node_added_triggerMR, .auto_add_replicas] [junit4] 2> 526771 INFO (ScheduledTrigger-1927-thread-4) [n:127.0.0.1:40277_solr ] o.a.s.c.a.SystemLogListener Collection .system does not exist, disabling logging. [junit4] 2> 526771 DEBUG (AutoscalingActionExecutor-1928-thread-1-processing-n:127.0.0.1:40277_solr) [n:127.0.0.1:40277_solr ] o.a.s.c.a.ScheduledTriggers -- processing actions for { [junit4] 2> "id":"467a9da7a92280T6zq3j0642to5x4wf8gy6w4yyi", [junit4] 2> "source":"node_added_triggerMR", [junit4] 2> "eventTime":19838065911079552, [junit4] 2> "eventType":"NODEADDED", [junit4] 2> "properties":{ [junit4] 2> "eventTimes":[19838065911079552], [junit4] 2> "_enqueue_time_":19838066914121225, [junit4] 2> "nodeNames":["127.0.0.1:45977_solr"]}} [junit4] 2> 526772 INFO (OverseerCollectionConfigSetProcessor-73357700753915915-127.0.0.1:40277_solr-n_0000000002) [n:127.0.0.1:40277_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist. Requestor may have disconnected from ZooKeeper [junit4] 2> 526773 DEBUG (AutoscalingActionExecutor-1928-thread-1-processing-n:127.0.0.1:40277_solr) [n:127.0.0.1:40277_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_triggerMR after 5000ms [junit4] 2> 526773 DEBUG (AutoscalingActionExecutor-1928-thread-1-processing-n:127.0.0.1:40277_solr) [n:127.0.0.1:40277_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_triggerMR after 5000ms [junit4] 2> 526773 DEBUG (AutoscalingActionExecutor-1928-thread-1-processing-n:127.0.0.1:40277_solr) [n:127.0.0.1:40277_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 5000ms [junit4] 2> 526773 DEBUG (AutoscalingActionExecutor-1928-thread-1-processing-n:127.0.0.1:40277_solr) [n:127.0.0.1:40277_solr ] o.a.s.c.a.ScheduledTriggers -- processing took 1 ms for event id=467a9da7a92280T6zq3j0642to5x4wf8gy6w4yyi [junit4] 2> 529959 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.c.a.TriggerIntegrationTest ====== KILL OVERSEER 2 [junit4] 2> 529960 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@773a1118{HTTP/1.1,[http/1.1]}{127.0.0.1:0} [junit4] 2> 529960 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1637394884 [junit4] 2> 529961 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null [junit4] 2> 529961 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@28be4c8a: rootName = solr_40277, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@70253baf [junit4] 2> 529972 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null [junit4] 2> 529972 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@395eb898: rootName = solr_40277, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@3f6717f [junit4] 2> 529976 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null [junit4] 2> 529976 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@24239d0: rootName = solr_40277, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@15162d18 [junit4] 2> 529978 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:40277_solr [junit4] 2> 529978 INFO (zkCallback-969-thread-1-processing-n:127.0.0.1:37968_solr) [n:127.0.0.1:37968_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2) [junit4] 2> 529979 INFO (zkCallback-950-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2) [junit4] 2> 529979 INFO (zkCallback-979-thread-1-processing-n:127.0.0.1:45977_solr) [n:127.0.0.1:45977_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2) [junit4] 2> 529979 INFO (zkCallback-950-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2) [junit4] 2> 529979 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null [junit4] 2> 529979 INFO (zkCallback-959-thread-2-processing-n:127.0.0.1:40277_solr) [n:127.0.0.1:40277_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2) [junit4] 2> 529979 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores [junit4] 2> 529979 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.c.Overseer Overseer (id=73357700753915915-127.0.0.1:40277_solr-n_0000000002) closing [junit4] 2> 529979 INFO (OverseerStateUpdate-73357700753915915-127.0.0.1:40277_solr-n_0000000002) [n:127.0.0.1:40277_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:40277_solr [junit4] 2> 529980 WARN (OverseerAutoScalingTriggerThread-73357700753915915-127.0.0.1:40277_solr-n_0000000002) [n:127.0.0.1:40277_solr ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting. [junit4] 2> 529980 DEBUG (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly [junit4] 2> 529989 INFO (zkCallback-969-thread-1-processing-n:127.0.0.1:37968_solr) [n:127.0.0.1:37968_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:37968_solr [junit4] 2> 529990 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@6ac06f1b{/solr,null,UNAVAILABLE} [junit4] 2> 529990 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.e.j.s.session Stopped scavenging [junit4] 2> 529990 INFO (zkCallback-969-thread-1-processing-n:127.0.0.1:37968_solr) [n:127.0.0.1:37968_solr ] o.a.s.c.Overseer Overseer (id=73357700753915918-127.0.0.1:37968_solr-n_0000000003) starting [junit4] 2> 529995 DEBUG (OverseerAutoScalingTriggerThread-73357700753915918-127.0.0.1:37968_solr-n_0000000003) [n:127.0.0.1:37968_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5 [junit4] 2> 529996 DEBUG (OverseerAutoScalingTriggerThread-73357700753915918-127.0.0.1:37968_solr-n_0000000003) [n:127.0.0.1:37968_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:45977_solr, 127.0.0.1:37968_solr] [junit4] 2> 529996 DEBUG (OverseerAutoScalingTriggerThread-73357700753915918-127.0.0.1:37968_solr-n_0000000003) [n:127.0.0.1:37968_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:45977_solr, 127.0.0.1:37968_solr] [junit4] 2> 529996 DEBUG (OverseerAutoScalingTriggerThread-73357700753915918-127.0.0.1:37968_solr-n_0000000003) [n:127.0.0.1:37968_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_triggerMR instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestEventMarkerAction}], enabled=true} [junit4] 2> 529997 DEBUG (OverseerAutoScalingTriggerThread-73357700753915918-127.0.0.1:37968_solr-n_0000000003) [n:127.0.0.1:37968_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:45977_solr, 127.0.0.1:37968_solr] [junit4] 2> 529997 DEBUG (OverseerAutoScalingTriggerThread-73357700753915918-127.0.0.1:37968_solr-n_0000000003) [n:127.0.0.1:37968_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion -1 [junit4] 2> 529997 DEBUG (OverseerAutoScalingTriggerThread-73357700753915918-127.0.0.1:37968_solr-n_0000000003) [n:127.0.0.1:37968_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5 [junit4] 2> 529997 INFO (OverseerAutoScalingTriggerThread-73357700753915918-127.0.0.1:37968_solr-n_0000000003) [n:127.0.0.1:37968_solr ] o.a.s.c.a.TriggerIntegrationTest TestEventMarkerAction init [junit4] 2> 529997 DEBUG (OverseerAutoScalingTriggerThread-73357700753915918-127.0.0.1:37968_solr-n_0000000003) [n:127.0.0.1:37968_solr ] o.a.s.c.a.NodeLostTrigger Adding lost node from marker path: 127.0.0.1:40277_solr [junit4] 2> 529998 INFO (OverseerAutoScalingTriggerThread-73357700753915918-127.0.0.1:37968_solr-n_0000000003) [n:127.0.0.1:37968_solr ] o.a.s.c.a.TriggerIntegrationTest TestEventMarkerAction init [junit4] 2> 529998 DEBUG (ScheduledTrigger-1938-thread-1) [n:127.0.0.1:37968_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_triggerMR with currently live nodes: 2 [junit4] 2> 529998 DEBUG (OverseerAutoScalingTriggerThread-73357700753915918-127.0.0.1:37968_solr-n_0000000003) [n:127.0.0.1:37968_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5 [junit4] 2> 529998 DEBUG (ScheduledTrigger-1938-thread-1) [n:127.0.0.1:37968_solr ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:40277_solr [junit4] 2> 529999 DEBUG (ScheduledTrigger-1938-thread-2) [n:127.0.0.1:37968_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerMR [junit4] 2> 529999 DEBUG (ScheduledTrigger-1938-thread-2) [n:127.0.0.1:37968_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2 [junit4] 2> 529999 DEBUG (ScheduledTrigger-1938-thread-3) [n:127.0.0.1:37968_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 [junit4] 2> 529999 DEBUG (ScheduledTrigger-1938-thread-3) [n:127.0.0.1:37968_solr ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:40277_solr [junit4] 2> 530999 DEBUG (ScheduledTrigger-1938-thread-2) [n:127.0.0.1:37968_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerMR [junit4] 2> 530999 DEBUG (ScheduledTrigger-1938-thread-2) [n:127.0.0.1:37968_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2 [junit4] 2> 531000 DEBUG (ScheduledTrigger-1938-thread-4) [n:127.0.0.1:37968_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 [junit4] 2> 531001 DEBUG (ScheduledTrigger-1938-thread-3) [n:127.0.0.1:37968_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_triggerMR with currently live nodes: 2 [junit4] 2> 531001 DEBUG (ScheduledTrigger-1938-thread-3) [n:127.0.0.1:37968_solr ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:40277_solr] [junit4] 2> 531001 DEBUG (ScheduledTrigger-1938-thread-3) [n:127.0.0.1:37968_solr ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: { [junit4] 2> "id":"467a9ea3e4319eT6zq3j0642to5x4wf8gy6w4yyk", [junit4] 2> "source":"node_lost_triggerMR", [junit4] 2> "eventTime":19838070142808478, [junit4] 2> "eventType":"NODELOST", [junit4] 2> "properties":{ [junit4] 2> "eventTimes":[19838070142808478], [junit4] 2> "nodeNames":["127.0.0.1:40277_solr"]}} [junit4] 2> 531001 DEBUG (ScheduledTrigger-1938-thread-3) [n:127.0.0.1:37968_solr ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_lost_triggerMR, node_added_triggerMR, .auto_add_replicas] [junit4] 2> 531002 INFO (ScheduledTrigger-1938-thread-3) [n:127.0.0.1:37968_solr ] o.a.s.c.a.SystemLogListener Collection .system does not exist, disabling logging. [junit4] 2> 531002 DEBUG (AutoscalingActionExecutor-1939-thread-1-processing-n:127.0.0.1:37968_solr) [n:127.0.0.1:37968_solr ] o.a.s.c.a.ScheduledTriggers -- processing actions for { [junit4] 2> "id":"467a9ea3e4319eT6zq3j0642to5x4wf8gy6w4yyk", [junit4] 2> "source":"node_lost_triggerMR", [junit4] 2> "eventTime":19838070142808478, [junit4] 2> "eventType":"NODELOST", [junit4] 2> "properties":{ [junit4] 2> "eventTimes":[19838070142808478], [junit4] 2> "_enqueue_time_":19838071145571294, [junit4] 2> "nodeNames":["127.0.0.1:40277_solr"]}} [junit4] 2> 531003 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[5A2E7CCD051D1093]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testNodeMarkersRegistration [junit4] 2> 531003 DEBUG (AutoscalingActionExecutor-1939-thread-1-processing-n:127.0.0.1:37968_solr) [n:127.0.0.1:37968_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_triggerMR after 5000ms [junit4] 2> 531004 DEBUG (AutoscalingActionExecutor-1939-thread-1-processing-n:127.0.0.1:37968_solr) [n:127.0.0.1:37968_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_triggerMR after 5000ms [junit4] 2> 531004 DEBUG (AutoscalingActionExecutor-1939-thread-1-processing-n:127.0.0.1:37968_solr) [n:127.0.0.1:37968_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 5000ms [junit4] 2> 531004 DEBUG (AutoscalingActionExecutor-1939-thread-1-processing-n:127.0.0.1:37968_solr) [n:127.0.0.1:37968_solr ] o.a.s.c.a.ScheduledTriggers -- processing took 1 ms for event id=467a9ea3e4319eT6zq3j0642to5x4wf8gy6w4yyk [junit4] 2> 531018 INFO (TEST-TriggerIntegrationTest.testMetricTrigger-seed#[5A2E7CCD051D1093]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testMetricTrigger [junit4] 2> 531020 INFO (qtp1728949472-4407) [n:127.0.0.1:37968_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :overseerstatus with params action=OVERSEERSTATUS&wt=javabin&version=2 and sendToOCPQueue=true [junit4] 2> 531025 INFO (qtp1728949472-4407) [n:127.0.0.1:37968_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=OVERSEERSTATUS&wt=javabin&version=2} status=0 QTime=4 [junit4] 2> 531025 INFO (TEST-TriggerIntegrationTest.testMetricTrigger-seed#[5A2E7CCD051D1093]) [ ] o.a.s.c.a.ScheduledTriggers -- removing trigger: node_lost_triggerMR [junit4] 2> 531026 INFO (TEST-TriggerIntegrationTest.testMetricTrigger-seed#[5A2E7CCD051D1093]) [ ] o.a.s.c.a.ScheduledTriggers -- removing trigger: node_added_triggerMR [junit4] 2> 531027 INFO (TEST-TriggerIntegrationTest.testMetricTrigger-seed#[5A2E7CCD051D1093]) [ ] o.a.s.c.a.ScheduledTriggers -- removing trigger: .auto_add_replicas [junit4] 2> 531028 INFO (TEST-TriggerIntegrationTest.testMetricTrigger-seed#[5A2E7CCD051D1093]) [ ] o.a.s.c.a.TriggerIntegrationTest /autoscaling.json reset, new znode version 6 [junit4] 2> 531028 DEBUG (zkCallback-969-thread-1-processing-n:127.0.0.1:37968_solr) [n:127.0.0.1:37968_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 6 [junit4] 2> 531028 DEBUG (OverseerAutoScalingTriggerThread-73357700753915918-127.0.0.1:37968_solr-n_0000000003) [n:127.0.0.1:37968_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6 [junit4] 2> 531028 DEBUG (OverseerAutoScalingTriggerThread-73357700753915918-127.0.0.1:37968_solr-n_0000000003) [n:127.0.0.1:37968_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers [junit4] 2> 531029 DEBUG (OverseerAutoScalingTriggerThread-73357700753915918-127.0.0.1:37968_solr-n_0000000003) [n:127.0.0.1:37968_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers [junit4] 2> 531029 DEBUG (OverseerAutoScalingTriggerThread-73357700753915918-127.0.0.1:37968_solr-n_0000000003) [n:127.0.0.1:37968_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion 6 [junit4] 2> 531029 INFO (TEST-TriggerIntegrationTest.testMetricTrigger-seed#[5A2E7CCD051D1093]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2) [junit4] 2> 531031 INFO (TEST-TriggerIntegrationTest.testMetricTrigger-seed#[5A2E7CCD051D1093]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@5e75ef24{HTTP/1.1,[http/1.1]}{127.0.0.1:0} [junit4] 2> 531031 INFO (TEST-TriggerIntegrationTest.testMetricTrigger-seed#[5A2E7CCD051D1093]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=187357777 [junit4] 2> 531031 INFO (TEST-TriggerIntegrationTest.testMetricTrigger-seed#[5A2E7CCD051D1093]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null [junit4] 2> 531031 INFO (TEST-TriggerIntegrationTest.testMetricTrigger-seed#[5A2E7CCD051D1093]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@7742ebd6: rootName = solr_37968, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@4e2efac3 [junit4] 2> 531038 INFO (TEST-TriggerIntegrationTest.testMetricTrigger-seed#[5A2E7CCD051D1093]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null [junit4] 2> 531038 INFO (TEST-TriggerIntegrationTest.testMetricTrigger-seed#[5A2E7CCD051D1093]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@234f854d: rootName = solr_37968, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@3f6717f [junit4] 2> 531042 INFO (TEST-TriggerIntegrationTest.testMetricTrigger-seed#[5A2E7CCD051D1093]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null [junit4] 2> 531042 INFO (TEST-TriggerIntegrationTest.testMetricTrigger-seed#[5A2E7CCD051D1093]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@26d1490: rootName = solr_37968, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@15162d18 [junit4] 2> 531043 INFO (TEST-TriggerIntegrationTest.testMetricTrigger-seed#[5A2E7CCD051D1093]) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:37968_solr [junit4] 2> 531043 INFO (TEST-TriggerIntegrationTest.testMetricTrigger-seed#[5A2E7CCD051D1093]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null [junit4] 2> 531044 INFO (zkCallback-979-thread-1-processing-n:127.0.0.1:45977_solr) [n:127.0.0.1:45977_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1) [junit4] 2> 531044 INFO (TEST-TriggerIntegrationTest.testMetricTrigger-seed#[5A2E7CCD051D1093]) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores [junit4] 2> 531044 INFO (TEST-TriggerIntegrationTest.testMetricTrigger-seed#[5A2E7CCD051D1093]) [ ] o.a.s.c.Overseer Overseer (id=73357700753915918-127.0.0.1:37968_solr-n_0000000003) closing [junit4] 2> 531044 INFO (OverseerStateUpdate-73357700753915918-127.0.0.1:37968_solr-n_0000000003) [n:127.0.0.1:37968_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:37968_solr [junit4] 2> 531044 INFO (zkCallback-950-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1) [junit4] 2> 531044 INFO (zkCallback-950-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1) [junit4] 2> 531044 WARN (OverseerAutoScalingTriggerThread-73357700753915918-127.0.0.1:37968_solr-n_0000000003) [n:127.0.0.1:37968_solr ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting. [junit4] 2> 531045 INFO (zkCallback-950-thread-3) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1) [junit4] 2> 531045 INFO (zkCallback-969-thread-1-processing-n:127.0.0.1:37968_solr) [n:127.0.0.1:37968_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1) [junit4] 2> 531045 DEBUG (TEST-TriggerIntegrationTest.testMetricTrigger-seed#[5A2E7CCD051D1093]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly [junit4] 2> 531046 INFO (zkCallback-979-thread-1-processing-n:127.0.0.1:45977_solr) [n:127.0.0.1:45977_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:45977_solr [junit4] 2> 531046 INFO (TEST-TriggerIntegrationTest.testMetricTrigger-seed#[5A2E7CCD051D1093]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@1c8c1604{/solr,null,UNAVAILABLE} [junit4] 2> 531046 INFO (TEST-TriggerIntegrationTest.testMetricTrigger-seed#[5A2E7CCD051D1093]) [ ] o.e.j.s.session Stopped scavenging [junit4] 2> 531047 INFO (zkCallback-979-thread-1-processing-n:127.0.0.1:45977_solr) [n:127.0.0.1:45977_solr ] o.a.s.c.Overseer Overseer (id=73357700753915921-127.0.0.1:45977_solr-n_0000000004) starting [junit4] 2> 531052 DEBUG (OverseerAutoScalingTriggerThread-73357700753915921-127.0.0.1:45977_solr-n_0000000004) [n:127.0.0.1:45977_solr ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger [junit4] 2> 531053 DEBUG (OverseerAutoScalingTriggerThread-73357700753915921-127.0.0.1:45977_solr-n_0000000004) [n:127.0.0.1:45977_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 7 [junit4] 2> 531053 DEBUG (OverseerAutoScalingTriggerThread-73357700753915921-127.0.0.1:45977_solr-n_0000000004) [n:127.0.0.1:45977_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:45977_solr] [junit4] 2> 531053 DEBUG (OverseerAutoScalingTriggerThread-73357700753915921-127.0.0.1:45977_solr-n_0000000004) [n:127.0.0.1:45977_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion -1 [junit4] 2> 531053 DEBUG (OverseerAutoScalingTriggerThread-73357700753915921-127.0.0.1:45977_solr-n_0000000004) [n:127.0.0.1:45977_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 7 [junit4] 2> 531055 DEBUG (OverseerAutoScalingTriggerThread-73357700753915921-127.0.0.1:45977_solr-n_0000000004) [n:127.0.0.1:45977_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers [junit4] 2> 531055 DEBUG (OverseerAutoScalingTriggerThread-73357700753915921-127.0.0.1:45977_solr-n_0000000004) [n:127.0.0.1:45977_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion 7 [junit4] 2> 531055 DEBUG (ScheduledTrigger-1943-thread-1) [n:127.0.0.1:45977_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1 [junit4] 2> 532056 DEBUG (ScheduledTrigger-1943-thread-1) [n:127.0.0.1:45977_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1 [junit4] 2> 533057 DEBUG (ScheduledTrigger-1943-thread-2) [n:127.0.0.1:45977_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1 [junit4] 2> 534058 DEBUG (ScheduledTrigger-1943-thread-1) [n:127.0.0.1:45977_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1 [junit4] 2> 535058 DEBUG (ScheduledTrigger-1943-thread-1) [n:127.0.0.1:45977_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1 [junit4] 2> 536048 INFO (TEST-TriggerIntegrationTest.testMetricTrigger-seed#[5A2E7CCD051D1093]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T21:27:37Z, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8 [junit4] 2> 536049 INFO (TEST-TriggerIntegrationTest.testMetricTrigger-seed#[5A2E7CCD051D1093]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0 [junit4] 2> 536049 INFO (TEST-TriggerIntegrationTest.testMetricTrigger-seed#[5A2E7CCD051D1093]) [ ] o.e.j.s.session No SessionScavenger set, using defaults [junit4] 2> 536049 INFO (TEST-TriggerIntegrationTest.testMetricTrigger-seed#[5A2E7CCD051D1093]) [ ] o.e.j.s.session Scavenging every 600000ms [junit4] 2> 536049 INFO (TEST-TriggerIntegrationTest.testMetricTrigger-seed#[5A2E7CCD051D1093]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@551d29ec{/solr,null,AVAILABLE} [junit4] 2> 536049 INFO (TEST-TriggerIntegrationTest.testMetricTrigger-seed#[5A2E7CCD051D1093]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@65fb3011{HTTP/1.1,[http/1.1]}{127.0.0.1:53288} [junit4] 2> 536049 INFO (TEST-TriggerIntegrationTest.testMetricTrigger-seed#[5A2E7CCD051D1093]) [ ] o.e.j.s.Server Started @538266ms [junit4] 2> 536049 INFO (TEST-TriggerIntegrationTest.testMetricTrigger-seed#[5A2E7CCD051D1093]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=53288} [junit4] 2> 536050 ERROR (TEST-TriggerIntegrationTest.testMetricTrigger-seed#[5A2E7CCD051D1093]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete. [junit4] 2> 536050 INFO (TEST-TriggerIntegrationTest.testMetricTrigger-seed#[5A [...truncated too long message...] .c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0) [junit4] 2> 783821 INFO (zkCallback-1186-thread-3-processing-n:127.0.0.1:55096_solr) [n:127.0.0.1:55096_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0) [junit4] 2> 783821 INFO (zkCallback-1142-thread-4-processing-n:127.0.0.1:52368_solr) [n:127.0.0.1:52368_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0) [junit4] 2> 783820 INFO (zkCallback-1206-thread-3-processing-n:127.0.0.1:46317_solr) [n:127.0.0.1:46317_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0) [junit4] 2> 783822 INFO (coreCloseExecutor-2209-thread-1) [n:127.0.0.1:57487_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.SolrCore [collection1_shard1_replica_n1] CLOSING SolrCore org.apache.solr.core.SolrCore@44795c89 [junit4] 2> 783820 INFO (zkCallback-1196-thread-3-processing-n:127.0.0.1:57487_solr) [n:127.0.0.1:57487_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0) [junit4] 2> 783822 INFO (coreCloseExecutor-2209-thread-1) [n:127.0.0.1:57487_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1.shard1.replica_n1, tag=1148804233 [junit4] 2> 783822 INFO (coreCloseExecutor-2209-thread-1) [n:127.0.0.1:57487_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5d8b66b5: rootName = solr_57487, domain = solr.core.collection1.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.collection1.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@5e1d041 [junit4] 2> 783880 INFO (coreCloseExecutor-2205-thread-1) [n:127.0.0.1:46317_solr c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_n5] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection1.shard1.leader, tag=1389440352 [junit4] 2> 783886 INFO (jetty-closer-923-thread-4) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@33f929e1{/solr,null,UNAVAILABLE} [junit4] 2> 783886 INFO (jetty-closer-923-thread-4) [ ] o.e.j.s.session Stopped scavenging [junit4] 2> 783890 INFO (coreCloseExecutor-2206-thread-1) [n:127.0.0.1:55096_solr c:collection1 s:shard1 r:core_node10 x:collection1_shard1_replica_n9] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection1.shard1.leader, tag=153574802 [junit4] 2> 783893 INFO (jetty-closer-923-thread-2) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@708911df{/solr,null,UNAVAILABLE} [junit4] 2> 783894 INFO (jetty-closer-923-thread-2) [ ] o.e.j.s.session Stopped scavenging [junit4] 2> 783902 INFO (coreCloseExecutor-2208-thread-1) [n:127.0.0.1:60379_solr c:collection1 s:shard1 r:core_node8 x:collection1_shard1_replica_n7] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection1.shard1.leader, tag=1700508537 [junit4] 2> 783902 INFO (coreCloseExecutor-2207-thread-1) [n:127.0.0.1:52368_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection1.shard1.leader, tag=1623130568 [junit4] 2> 783903 INFO (coreCloseExecutor-2209-thread-1) [n:127.0.0.1:57487_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection1.shard1.leader, tag=1148804233 [junit4] 2> 783908 INFO (jetty-closer-923-thread-3) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@1668b13{/solr,null,UNAVAILABLE} [junit4] 2> 783909 INFO (jetty-closer-923-thread-1) [ ] o.a.s.c.Overseer Overseer (id=73357700753915967-127.0.0.1:52368_solr-n_0000000019) closing [junit4] 2> 783909 INFO (OverseerStateUpdate-73357700753915967-127.0.0.1:52368_solr-n_0000000019) [n:127.0.0.1:52368_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:52368_solr [junit4] 2> 783910 WARN (OverseerAutoScalingTriggerThread-73357700753915967-127.0.0.1:52368_solr-n_0000000019) [n:127.0.0.1:52368_solr ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting. [junit4] 2> 783910 DEBUG (jetty-closer-923-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly [junit4] 2> 783910 INFO (jetty-closer-923-thread-3) [ ] o.e.j.s.session Stopped scavenging [junit4] 2> 783914 INFO (jetty-closer-923-thread-5) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@45f38e83{/solr,null,UNAVAILABLE} [junit4] 2> 783914 INFO (jetty-closer-923-thread-5) [ ] o.e.j.s.session Stopped scavenging [junit4] 2> 783915 INFO (jetty-closer-923-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@7f875616{/solr,null,UNAVAILABLE} [junit4] 2> 783915 INFO (jetty-closer-923-thread-1) [ ] o.e.j.s.session Stopped scavenging [junit4] 2> 783916 ERROR (SUITE-TriggerIntegrationTest-seed#[5A2E7CCD051D1093]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes [junit4] 2> 783916 INFO (SUITE-TriggerIntegrationTest-seed#[5A2E7CCD051D1093]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:36380 36380 [junit4] 2> 784015 INFO (Thread-869) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:36380 36380 [junit4] 2> 784016 WARN (Thread-869) [ ] o.a.s.c.ZkTestServer Watch limit violations: [junit4] 2> Maximum concurrent create/delete watches above limit: [junit4] 2> [junit4] 2> 42 /solr/aliases.json [junit4] 2> 42 /solr/clusterprops.json [junit4] 2> 27 /solr/security.json [junit4] 2> 20 /solr/collections/collection1/terms/shard1 [junit4] 2> 10 /solr/collections/testMetricTrigger/terms/shard1 [junit4] 2> 9 /solr/configs/conf [junit4] 2> 2 /solr/collections/testMetricTrigger/terms/shard2 [junit4] 2> [junit4] 2> Maximum concurrent data watches above limit: [junit4] 2> [junit4] 2> 56 /solr/collections/collection1/state.json [junit4] 2> 42 /solr/clusterstate.json [junit4] 2> 34 /solr/collections/testMetricTrigger/state.json [junit4] 2> 20 /solr/autoscaling.json [junit4] 2> 5 /solr/overseer_elect/election/73357700753915967-127.0.0.1:52368_solr-n_0000000019 [junit4] 2> 2 /solr/collections/collection1/leader_elect/shard1/election/73357700753915982-core_node3-n_0000000001 [junit4] 2> 2 /solr/collections/collection1/leader_elect/shard1/election/73357700753915967-core_node4-n_0000000000 [junit4] 2> 2 /solr/overseer_elect/election/73357700753915982-127.0.0.1:57487_solr-n_0000000024 [junit4] 2> [junit4] 2> Maximum concurrent children watches above limit: [junit4] 2> [junit4] 2> 382 /solr/live_nodes [junit4] 2> 61 /solr/collections [junit4] 2> 20 /solr/overseer/queue-work [junit4] 2> 20 /solr/autoscaling/events/.auto_add_replicas [junit4] 2> 20 /solr/overseer/queue [junit4] 2> 20 /solr/overseer/collection-queue-work [junit4] 2> 2 /solr/autoscaling/events/node_lost_triggerMR [junit4] 2> 2 /solr/autoscaling/events/node_added_triggerCTOOR [junit4] 2> [junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_5A2E7CCD051D1093-001 [junit4] 2> Feb 25, 2018 7:20:17 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks [junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s). [junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {id=PostingsFormat(name=Direct), x_s=PostingsFormat(name=Memory)}, docValues:{}, maxPointsInLeafNode=1173, maxMBSortInHeap=7.351452710078859, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@50a7a0f0), locale=fr, timezone=Asia/Chungking [junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=198703712,total=511180800 [junit4] 2> NOTE: All tests run in this JVM: [TestUseDocValuesAsStored2, TestRandomFaceting, SuggesterWFSTTest, TestMergePolicyConfig, TestLeaderInitiatedRecoveryThread, TestFastOutputStream, TestAnalyzeInfixSuggestions, MetricsConfigTest, InfoHandlerTest, TestLegacyFieldReuse, UpdateParamsTest, TestOmitPositions, TestEmbeddedSolrServerSchemaAPI, SearchRateTriggerTest, SolrXmlInZkTest, TestNumericTerms32, CdcrReplicationDistributedZkTest, IndexBasedSpellCheckerTest, IgnoreCommitOptimizeUpdateProcessorFactoryTest, EchoParamsTest, CoreAdminCreateDiscoverTest, LeaderFailureAfterFreshStartTest, TestGeoJSONResponseWriter, ManagedSchemaRoundRobinCloudTest, TestCSVResponseWriter, DirectUpdateHandlerTest, TestSkipOverseerOperations, CreateCollectionCleanupTest, ChaosMonkeySafeLeaderWithPullReplicasTest, TestLMDirichletSimilarityFactory, SpellCheckCollatorTest, ZkNodePropsTest, CloudMLTQParserTest, DirectoryFactoryTest, BasicDistributedZkTest, TestStressLucene, DistributedFacetPivotLongTailTest, ZkSolrClientTest, DocumentAnalysisRequestHandlerTest, MBeansHandlerTest, TestDocSet, TestDistributedSearch, TestRemoteStreaming, TestDynamicFieldResource, CollectionTooManyReplicasTest, TestCollationField, TestMacros, HighlighterConfigTest, TestConfigSetImmutable, AssignBackwardCompatibilityTest, CurrencyRangeFacetCloudTest, TriggerIntegrationTest] [junit4] Completed [216/778 (1!)] on J0 in 270.89s, 14 tests, 1 failure <<< FAILURES! [...truncated 50260 lines...]
--------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org