Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/3284/
1 tests failed. FAILED: org.apache.solr.cloud.OverseerRolesTest.testOverseerRole Error Message: Timed out waiting for overseer state change Stack Trace: java.lang.AssertionError: Timed out waiting for overseer state change at __randomizedtesting.SeedInfo.seed([62094B24FCF12C8B:83C2B6B0C7421A5A]:0) at org.junit.Assert.fail(Assert.java:88) at org.apache.solr.cloud.OverseerRolesTest.waitForNewOverseer(OverseerRolesTest.java:63) at org.apache.solr.cloud.OverseerRolesTest.testOverseerRole(OverseerRolesTest.java:145) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938) at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974) at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368) at java.base/java.lang.Thread.run(Thread.java:834) Build Log: [...truncated 14486 lines...] [junit4] Suite: org.apache.solr.cloud.OverseerRolesTest [junit4] 2> 1665148 INFO (SUITE-OverseerRolesTest-seed#[62094B24FCF12C8B]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom [junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.OverseerRolesTest_62094B24FCF12C8B-001/init-core-data-001 [junit4] 2> 1665149 INFO (SUITE-OverseerRolesTest-seed#[62094B24FCF12C8B]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false [junit4] 2> 1665149 INFO (SUITE-OverseerRolesTest-seed#[62094B24FCF12C8B]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason="", ssl=0.0/0.0, value=0.0/0.0, clientAuth=0.0/0.0) [junit4] 2> 1665150 INFO (SUITE-OverseerRolesTest-seed#[62094B24FCF12C8B]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.OverseerRolesTest_62094B24FCF12C8B-001/tempDir-001 [junit4] 2> 1665150 INFO (SUITE-OverseerRolesTest-seed#[62094B24FCF12C8B]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER [junit4] 2> 1665151 INFO (ZkTestServer Run Thread) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0 [junit4] 2> 1665151 INFO (ZkTestServer Run Thread) [ ] o.a.s.c.ZkTestServer Starting server [junit4] 2> 1665251 INFO (SUITE-OverseerRolesTest-seed#[62094B24FCF12C8B]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:39507 [junit4] 2> 1665251 INFO (SUITE-OverseerRolesTest-seed#[62094B24FCF12C8B]-worker) [ ] o.a.s.c.ZkTestServer parse host and port list: 127.0.0.1:39507 [junit4] 2> 1665251 INFO (SUITE-OverseerRolesTest-seed#[62094B24FCF12C8B]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1 39507 [junit4] 2> 1665271 INFO (zkConnectionManagerCallback-8135-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 1665290 INFO (zkConnectionManagerCallback-8137-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 1665296 INFO (zkConnectionManagerCallback-8139-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 1665300 WARN (jetty-launcher-8140-thread-1) [ ] o.e.j.s.AbstractConnector Ignoring deprecated socket close linger time [junit4] 2> 1665301 INFO (jetty-launcher-8140-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Start Jetty (original configured port=0) [junit4] 2> 1665301 INFO (jetty-launcher-8140-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 1 ... [junit4] 2> 1665301 INFO (jetty-launcher-8140-thread-1) [ ] o.e.j.s.Server jetty-9.4.14.v20181114; built: 2018-11-14T21:20:31.478Z; git: c4550056e785fb5665914545889f21dc136ad9e6; jvm 11.0.1+13-LTS [junit4] 2> 1665304 WARN (jetty-launcher-8140-thread-2) [ ] o.e.j.s.AbstractConnector Ignoring deprecated socket close linger time [junit4] 2> 1665304 INFO (jetty-launcher-8140-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Start Jetty (original configured port=0) [junit4] 2> 1665304 INFO (jetty-launcher-8140-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 1 ... [junit4] 2> 1665304 INFO (jetty-launcher-8140-thread-2) [ ] o.e.j.s.Server jetty-9.4.14.v20181114; built: 2018-11-14T21:20:31.478Z; git: c4550056e785fb5665914545889f21dc136ad9e6; jvm 11.0.1+13-LTS [junit4] 2> 1665308 WARN (jetty-launcher-8140-thread-3) [ ] o.e.j.s.AbstractConnector Ignoring deprecated socket close linger time [junit4] 2> 1665309 INFO (jetty-launcher-8140-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Start Jetty (original configured port=0) [junit4] 2> 1665309 INFO (jetty-launcher-8140-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 1 ... [junit4] 2> 1665309 INFO (jetty-launcher-8140-thread-3) [ ] o.e.j.s.Server jetty-9.4.14.v20181114; built: 2018-11-14T21:20:31.478Z; git: c4550056e785fb5665914545889f21dc136ad9e6; jvm 11.0.1+13-LTS [junit4] 2> 1665312 INFO (jetty-launcher-8140-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0 [junit4] 2> 1665312 INFO (jetty-launcher-8140-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults [junit4] 2> 1665312 INFO (jetty-launcher-8140-thread-2) [ ] o.e.j.s.session node0 Scavenging every 600000ms [junit4] 2> 1665312 INFO (jetty-launcher-8140-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@31b15ede{/solr,null,AVAILABLE} [junit4] 2> 1665312 WARN (jetty-launcher-8140-thread-4) [ ] o.e.j.s.AbstractConnector Ignoring deprecated socket close linger time [junit4] 2> 1665313 INFO (jetty-launcher-8140-thread-4) [ ] o.a.s.c.s.e.JettySolrRunner Start Jetty (original configured port=0) [junit4] 2> 1665313 INFO (jetty-launcher-8140-thread-4) [ ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 1 ... [junit4] 2> 1665313 INFO (jetty-launcher-8140-thread-4) [ ] o.e.j.s.Server jetty-9.4.14.v20181114; built: 2018-11-14T21:20:31.478Z; git: c4550056e785fb5665914545889f21dc136ad9e6; jvm 11.0.1+13-LTS [junit4] 2> 1665324 INFO (jetty-launcher-8140-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0 [junit4] 2> 1665324 INFO (jetty-launcher-8140-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults [junit4] 2> 1665324 INFO (jetty-launcher-8140-thread-1) [ ] o.e.j.s.session node0 Scavenging every 600000ms [junit4] 2> 1665325 INFO (jetty-launcher-8140-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7446e59a{/solr,null,AVAILABLE} [junit4] 2> 1665327 INFO (jetty-launcher-8140-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@6269e125{HTTP/1.1,[http/1.1, h2c]}{127.0.0.1:41935} [junit4] 2> 1665327 INFO (jetty-launcher-8140-thread-2) [ ] o.e.j.s.Server Started @1665381ms [junit4] 2> 1665327 INFO (jetty-launcher-8140-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=41935} [junit4] 2> 1665328 ERROR (jetty-launcher-8140-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete. [junit4] 2> 1665328 INFO (jetty-launcher-8140-thread-2) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory [junit4] 2> 1665328 INFO (jetty-launcher-8140-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 9.0.0 [junit4] 2> 1665328 INFO (jetty-launcher-8140-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null [junit4] 2> 1665328 INFO (jetty-launcher-8140-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null [junit4] 2> 1665328 INFO (jetty-launcher-8140-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2019-04-15T20:15:10.885248Z [junit4] 2> 1665330 INFO (jetty-launcher-8140-thread-3) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0 [junit4] 2> 1665330 INFO (jetty-launcher-8140-thread-3) [ ] o.e.j.s.session No SessionScavenger set, using defaults [junit4] 2> 1665330 INFO (jetty-launcher-8140-thread-3) [ ] o.e.j.s.session node0 Scavenging every 600000ms [junit4] 2> 1665331 INFO (jetty-launcher-8140-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@226122b5{/solr,null,AVAILABLE} [junit4] 2> 1665331 INFO (jetty-launcher-8140-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@35a35d59{HTTP/1.1,[http/1.1, h2c]}{127.0.0.1:46527} [junit4] 2> 1665331 INFO (jetty-launcher-8140-thread-1) [ ] o.e.j.s.Server Started @1665385ms [junit4] 2> 1665331 INFO (jetty-launcher-8140-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=46527} [junit4] 2> 1665332 ERROR (jetty-launcher-8140-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete. [junit4] 2> 1665332 INFO (jetty-launcher-8140-thread-1) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory [junit4] 2> 1665332 INFO (jetty-launcher-8140-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 9.0.0 [junit4] 2> 1665332 INFO (jetty-launcher-8140-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null [junit4] 2> 1665332 INFO (jetty-launcher-8140-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null [junit4] 2> 1665332 INFO (jetty-launcher-8140-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2019-04-15T20:15:10.889275Z [junit4] 2> 1665332 INFO (jetty-launcher-8140-thread-3) [ ] o.e.j.s.AbstractConnector Started ServerConnector@b29eeca{HTTP/1.1,[http/1.1, h2c]}{127.0.0.1:37155} [junit4] 2> 1665332 INFO (jetty-launcher-8140-thread-3) [ ] o.e.j.s.Server Started @1665386ms [junit4] 2> 1665332 INFO (jetty-launcher-8140-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=37155} [junit4] 2> 1665333 ERROR (jetty-launcher-8140-thread-3) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete. [junit4] 2> 1665333 INFO (jetty-launcher-8140-thread-3) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory [junit4] 2> 1665335 INFO (jetty-launcher-8140-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 9.0.0 [junit4] 2> 1665335 INFO (jetty-launcher-8140-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null [junit4] 2> 1665335 INFO (jetty-launcher-8140-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null [junit4] 2> 1665335 INFO (jetty-launcher-8140-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2019-04-15T20:15:10.892374Z [junit4] 2> 1665337 INFO (jetty-launcher-8140-thread-4) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0 [junit4] 2> 1665337 INFO (jetty-launcher-8140-thread-4) [ ] o.e.j.s.session No SessionScavenger set, using defaults [junit4] 2> 1665337 INFO (jetty-launcher-8140-thread-4) [ ] o.e.j.s.session node0 Scavenging every 660000ms [junit4] 2> 1665337 INFO (jetty-launcher-8140-thread-4) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5d4cf1d7{/solr,null,AVAILABLE} [junit4] 2> 1665338 INFO (jetty-launcher-8140-thread-4) [ ] o.e.j.s.AbstractConnector Started ServerConnector@40c0ab01{HTTP/1.1,[http/1.1, h2c]}{127.0.0.1:42464} [junit4] 2> 1665338 INFO (jetty-launcher-8140-thread-4) [ ] o.e.j.s.Server Started @1665391ms [junit4] 2> 1665338 INFO (jetty-launcher-8140-thread-4) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=42464} [junit4] 2> 1665338 ERROR (jetty-launcher-8140-thread-4) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete. [junit4] 2> 1665338 INFO (jetty-launcher-8140-thread-4) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory [junit4] 2> 1665338 INFO (jetty-launcher-8140-thread-4) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 9.0.0 [junit4] 2> 1665338 INFO (jetty-launcher-8140-thread-4) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null [junit4] 2> 1665338 INFO (jetty-launcher-8140-thread-4) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null [junit4] 2> 1665338 INFO (jetty-launcher-8140-thread-4) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2019-04-15T20:15:10.895945Z [junit4] 2> 1665347 INFO (zkConnectionManagerCallback-8146-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 1665347 INFO (jetty-launcher-8140-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading... [junit4] 2> 1665348 INFO (zkConnectionManagerCallback-8144-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 1665348 INFO (zkConnectionManagerCallback-8142-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 1665350 INFO (jetty-launcher-8140-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading... [junit4] 2> 1665356 INFO (jetty-launcher-8140-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading... [junit4] 2> 1665377 INFO (zkConnectionManagerCallback-8148-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 1665378 INFO (jetty-launcher-8140-thread-4) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading... [junit4] 2> 1665518 INFO (jetty-launcher-8140-thread-3) [ ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true] [junit4] 2> 1665532 WARN (jetty-launcher-8140-thread-3) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for SslContextFactory@2a6cca23[provider=null,keyStore=null,trustStore=null] [junit4] 2> 1665536 INFO (jetty-launcher-8140-thread-2) [ ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true] [junit4] 2> 1665538 WARN (jetty-launcher-8140-thread-3) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for SslContextFactory@5de3b08f[provider=null,keyStore=null,trustStore=null] [junit4] 2> 1665540 INFO (jetty-launcher-8140-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39507/solr [junit4] 2> 1665540 WARN (jetty-launcher-8140-thread-2) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for SslContextFactory@153f1521[provider=null,keyStore=null,trustStore=null] [junit4] 2> 1665544 INFO (zkConnectionManagerCallback-8159-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 1665545 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x103d71f6b310007, likely client has closed socket [junit4] 2> 1665546 WARN (jetty-launcher-8140-thread-2) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for SslContextFactory@23d75b59[provider=null,keyStore=null,trustStore=null] [junit4] 2> 1665548 INFO (jetty-launcher-8140-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39507/solr [junit4] 2> 1665573 INFO (zkConnectionManagerCallback-8163-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 1665573 INFO (zkConnectionManagerCallback-8166-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 1665579 INFO (zkConnectionManagerCallback-8168-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 1665599 INFO (jetty-launcher-8140-thread-3) [n:127.0.0.1:37155_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:37155_solr [junit4] 2> 1665601 INFO (jetty-launcher-8140-thread-3) [n:127.0.0.1:37155_solr ] o.a.s.c.Overseer Overseer (id=73138548910391304-127.0.0.1:37155_solr-n_0000000000) starting [junit4] 2> 1665639 INFO (zkConnectionManagerCallback-8177-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 1665643 INFO (jetty-launcher-8140-thread-3) [n:127.0.0.1:37155_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:39507/solr ready [junit4] 2> 1665644 INFO (jetty-launcher-8140-thread-1) [ ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true] [junit4] 2> 1665645 INFO (OverseerStateUpdate-73138548910391304-127.0.0.1:37155_solr-n_0000000000) [n:127.0.0.1:37155_solr ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:37155_solr [junit4] 2> 1665646 INFO (jetty-launcher-8140-thread-3) [n:127.0.0.1:37155_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37155_solr [junit4] 2> 1665647 WARN (jetty-launcher-8140-thread-1) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for SslContextFactory@5d1ae629[provider=null,keyStore=null,trustStore=null] [junit4] 2> 1665692 WARN (jetty-launcher-8140-thread-1) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for SslContextFactory@4f4317c9[provider=null,keyStore=null,trustStore=null] [junit4] 2> 1665693 INFO (jetty-launcher-8140-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39507/solr [junit4] 2> 1665693 INFO (jetty-launcher-8140-thread-4) [ ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true] [junit4] 2> 1665694 INFO (zkCallback-8162-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 1665696 WARN (jetty-launcher-8140-thread-4) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for SslContextFactory@50aad72c[provider=null,keyStore=null,trustStore=null] [junit4] 2> 1665699 INFO (zkCallback-8176-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 1665699 INFO (zkConnectionManagerCallback-8183-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 1665700 WARN (jetty-launcher-8140-thread-4) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for SslContextFactory@78189dd[provider=null,keyStore=null,trustStore=null] [junit4] 2> 1665702 INFO (jetty-launcher-8140-thread-4) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39507/solr [junit4] 2> 1665707 INFO (zkConnectionManagerCallback-8189-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 1665710 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x103d71f6b31000d, likely client has closed socket [junit4] 2> 1665755 INFO (jetty-launcher-8140-thread-2) [n:127.0.0.1:41935_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 1665763 INFO (jetty-launcher-8140-thread-2) [n:127.0.0.1:41935_solr ] o.a.s.c.ZkController Publish node=127.0.0.1:41935_solr as DOWN [junit4] 2> 1665764 INFO (jetty-launcher-8140-thread-2) [n:127.0.0.1:41935_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores [junit4] 2> 1665764 INFO (jetty-launcher-8140-thread-2) [n:127.0.0.1:41935_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41935_solr [junit4] 2> 1665766 INFO (zkCallback-8176-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2) [junit4] 2> 1665766 INFO (zkCallback-8162-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2) [junit4] 2> 1665767 INFO (zkConnectionManagerCallback-8193-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 1665776 INFO (jetty-launcher-8140-thread-4) [n:127.0.0.1:42464_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2) [junit4] 2> 1665776 INFO (zkConnectionManagerCallback-8191-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 1665779 INFO (zkCallback-8167-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2) [junit4] 2> 1665782 INFO (jetty-launcher-8140-thread-3) [n:127.0.0.1:37155_solr ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory. [junit4] 2> 1665784 INFO (jetty-launcher-8140-thread-1) [n:127.0.0.1:46527_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2) [junit4] 2> 1665788 INFO (jetty-launcher-8140-thread-4) [n:127.0.0.1:42464_solr ] o.a.s.c.ZkController Publish node=127.0.0.1:42464_solr as DOWN [junit4] 2> 1665789 INFO (jetty-launcher-8140-thread-4) [n:127.0.0.1:42464_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores [junit4] 2> 1665789 INFO (jetty-launcher-8140-thread-4) [n:127.0.0.1:42464_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42464_solr [junit4] 2> 1665791 INFO (zkCallback-8176-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3) [junit4] 2> 1665791 INFO (zkCallback-8162-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3) [junit4] 2> 1665792 INFO (zkCallback-8167-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3) [junit4] 2> 1665800 INFO (jetty-launcher-8140-thread-1) [n:127.0.0.1:46527_solr ] o.a.s.c.ZkController Publish node=127.0.0.1:46527_solr as DOWN [junit4] 2> 1665801 INFO (jetty-launcher-8140-thread-1) [n:127.0.0.1:46527_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores [junit4] 2> 1665801 INFO (jetty-launcher-8140-thread-1) [n:127.0.0.1:46527_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46527_solr [junit4] 2> 1665803 INFO (zkCallback-8167-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4) [junit4] 2> 1665803 INFO (zkCallback-8176-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4) [junit4] 2> 1665804 INFO (zkCallback-8162-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4) [junit4] 2> 1665815 INFO (zkCallback-8192-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (4) [junit4] 2> 1665815 INFO (zkCallback-8190-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (4) [junit4] 2> 1665827 INFO (zkConnectionManagerCallback-8203-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 1665829 INFO (jetty-launcher-8140-thread-2) [n:127.0.0.1:41935_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4) [junit4] 2> 1665830 INFO (jetty-launcher-8140-thread-2) [n:127.0.0.1:41935_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:39507/solr ready [junit4] 2> 1665842 INFO (jetty-launcher-8140-thread-3) [n:127.0.0.1:37155_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37155.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1b7451af [junit4] 2> 1665843 INFO (jetty-launcher-8140-thread-2) [n:127.0.0.1:41935_solr ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory. [junit4] 2> 1665864 INFO (zkConnectionManagerCallback-8208-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 1665865 INFO (zkConnectionManagerCallback-8214-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 1665867 INFO (jetty-launcher-8140-thread-4) [n:127.0.0.1:42464_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4) [junit4] 2> 1665867 INFO (jetty-launcher-8140-thread-3) [n:127.0.0.1:37155_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37155.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1b7451af [junit4] 2> 1665867 INFO (jetty-launcher-8140-thread-1) [n:127.0.0.1:46527_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4) [junit4] 2> 1665868 INFO (jetty-launcher-8140-thread-3) [n:127.0.0.1:37155_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37155.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1b7451af [junit4] 2> 1665869 INFO (jetty-launcher-8140-thread-4) [n:127.0.0.1:42464_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:39507/solr ready [junit4] 2> 1665869 INFO (jetty-launcher-8140-thread-1) [n:127.0.0.1:46527_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:39507/solr ready [junit4] 2> 1665869 INFO (jetty-launcher-8140-thread-4) [n:127.0.0.1:42464_solr ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory. [junit4] 2> 1665872 INFO (jetty-launcher-8140-thread-3) [n:127.0.0.1:37155_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.OverseerRolesTest_62094B24FCF12C8B-001/tempDir-001/node3/. [junit4] 2> 1665888 INFO (jetty-launcher-8140-thread-1) [n:127.0.0.1:46527_solr ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory. [junit4] 2> 1665929 INFO (jetty-launcher-8140-thread-4) [n:127.0.0.1:42464_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42464.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1b7451af [junit4] 2> 1665929 INFO (jetty-launcher-8140-thread-2) [n:127.0.0.1:41935_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41935.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1b7451af [junit4] 2> 1666034 INFO (jetty-launcher-8140-thread-4) [n:127.0.0.1:42464_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42464.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1b7451af [junit4] 2> 1666034 INFO (jetty-launcher-8140-thread-4) [n:127.0.0.1:42464_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42464.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1b7451af [junit4] 2> 1666042 INFO (jetty-launcher-8140-thread-4) [n:127.0.0.1:42464_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.OverseerRolesTest_62094B24FCF12C8B-001/tempDir-001/node4/. [junit4] 2> 1666064 INFO (jetty-launcher-8140-thread-1) [n:127.0.0.1:46527_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46527.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1b7451af [junit4] 2> 1666072 INFO (jetty-launcher-8140-thread-2) [n:127.0.0.1:41935_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41935.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1b7451af [junit4] 2> 1666072 INFO (jetty-launcher-8140-thread-2) [n:127.0.0.1:41935_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41935.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1b7451af [junit4] 2> 1666074 INFO (jetty-launcher-8140-thread-2) [n:127.0.0.1:41935_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.OverseerRolesTest_62094B24FCF12C8B-001/tempDir-001/node2/. [junit4] 2> 1666137 INFO (jetty-launcher-8140-thread-1) [n:127.0.0.1:46527_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46527.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1b7451af [junit4] 2> 1666137 INFO (jetty-launcher-8140-thread-1) [n:127.0.0.1:46527_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46527.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1b7451af [junit4] 2> 1666139 INFO (jetty-launcher-8140-thread-1) [n:127.0.0.1:46527_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.OverseerRolesTest_62094B24FCF12C8B-001/tempDir-001/node1/. [junit4] 2> 1666333 INFO (zkConnectionManagerCallback-8221-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 1666335 INFO (SUITE-OverseerRolesTest-seed#[62094B24FCF12C8B]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4) [junit4] 2> 1666336 INFO (SUITE-OverseerRolesTest-seed#[62094B24FCF12C8B]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:39507/solr ready [junit4] 2> 1666380 INFO (TEST-OverseerRolesTest.testOverseerRole-seed#[62094B24FCF12C8B]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testOverseerRole [junit4] 2> 1666381 INFO (TEST-OverseerRolesTest.testOverseerRole-seed#[62094B24FCF12C8B]) [ ] o.a.s.c.OverseerRolesTest Overseer: 127.0.0.1:37155_solr [junit4] 2> 1666382 INFO (TEST-OverseerRolesTest.testOverseerRole-seed#[62094B24FCF12C8B]) [ ] o.a.s.c.OverseerRolesTest Election queue: [junit4] 2> 1666383 INFO (TEST-OverseerRolesTest.testOverseerRole-seed#[62094B24FCF12C8B]) [ ] o.a.s.c.OverseerRolesTest ### Setting overseer designate 127.0.0.1:46527_solr [junit4] 2> 1666387 INFO (qtp878210393-20889) [n:127.0.0.1:42464_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addrole with params node=127.0.0.1:46527_solr&role=overseer&action=ADDROLE&wt=javabin&version=2 and sendToOCPQueue=true [junit4] 2> 1666398 INFO (OverseerThreadFactory-7546-thread-1-processing-n:127.0.0.1:37155_solr) [n:127.0.0.1:37155_solr ] o.a.s.c.a.c.OverseerRoleCmd Overseer role added to 127.0.0.1:46527_solr [junit4] 2> 1666411 INFO (qtp878210393-20889) [n:127.0.0.1:42464_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:46527_solr&role=overseer&action=ADDROLE&wt=javabin&version=2} status=0 QTime=23 [junit4] 2> 1666413 INFO (Thread-3136) [ ] o.a.s.c.OverseerNodePrioritizer prioritizing overseer nodes at 73138548910391304-127.0.0.1:37155_solr-n_0000000000 overseer designates are [127.0.0.1:46527_solr] [junit4] 2> 1666413 INFO (Thread-3136) [ ] o.a.s.c.OverseerNodePrioritizer sorted nodes [73138548910391304-127.0.0.1:37155_solr-n_0000000000, 73138548910391306-127.0.0.1:41935_solr-n_0000000001, 73138548910391310-127.0.0.1:42464_solr-n_0000000002, 73138548910391311-127.0.0.1:46527_solr-n_0000000003] [junit4] 2> 1666413 INFO (Thread-3136) [ ] o.a.s.c.OverseerNodePrioritizer asking node 73138548910391311-127.0.0.1:46527_solr-n_0000000003 to come join election at head [junit4] 2> 1666415 INFO (qtp1147157588-20855) [n:127.0.0.1:46527_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={op=rejoinAtHead&qt=/admin/cores&action=OVERSEEROP&electionNode=73138548910391311-127.0.0.1:46527_solr-n_0000000003&wt=javabin&version=2} status=0 QTime=0 [junit4] 2> 1666416 INFO (Thread-3136) [ ] o.a.s.c.OverseerNodePrioritizer asking the old first in line 73138548910391306-127.0.0.1:41935_solr-n_0000000001 to rejoin election [junit4] 2> 1666417 INFO (qtp636615717-20860) [n:127.0.0.1:41935_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={op=rejoin&qt=/admin/cores&action=OVERSEEROP&electionNode=73138548910391306-127.0.0.1:41935_solr-n_0000000001&wt=javabin&version=2} status=0 QTime=0 [junit4] 2> 1666420 INFO (OverseerStateUpdate-73138548910391304-127.0.0.1:37155_solr-n_0000000000) [n:127.0.0.1:37155_solr ] o.a.s.c.Overseer Quit command received { [junit4] 2> "operation":"quit", [junit4] 2> "id":"73138548910391304-127.0.0.1:37155_solr-n_0000000000"} 127.0.0.1:37155_solr [junit4] 2> 1666421 INFO (OverseerStateUpdate-73138548910391304-127.0.0.1:37155_solr-n_0000000000) [n:127.0.0.1:37155_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:37155_solr [junit4] 2> 1666431 WARN (OverseerExitThread) [ ] o.a.s.c.Overseer I (id=73138548910391304-127.0.0.1:37155_solr-n_0000000000) am exiting, but I'm still the leader [junit4] 2> 1666433 INFO (OverseerExitThread) [ ] o.a.s.c.Overseer Overseer (id=73138548910391304-127.0.0.1:37155_solr-n_0000000000) closing [junit4] 2> 1666434 INFO (OverseerAutoScalingTriggerThread-73138548910391304-127.0.0.1:37155_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting. [junit4] 2> 1666435 INFO (OverseerExitThread) [ ] o.a.s.c.Overseer Overseer (id=73138548910391304-127.0.0.1:37155_solr-n_0000000000) closing [junit4] 2> 1666435 INFO (OverseerExitThread) [ ] o.a.s.c.Overseer Overseer (id=73138548910391304-127.0.0.1:37155_solr-n_0000000000) closing [junit4] 2> 1666436 INFO (zkCallback-8167-thread-1) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:41935_solr [junit4] 2> 1666437 INFO (OverseerExitThread) [ ] o.a.s.c.Overseer Overseer (id=73138548910391304-127.0.0.1:37155_solr-n_0000000000) closing [junit4] 2> 1666437 INFO (zkCallback-8167-thread-1) [n:127.0.0.1:41935_solr ] o.a.s.c.Overseer Overseer (id=73138548910391306-127.0.0.1:41935_solr-n_0000000001) starting [junit4] 2> 1666448 INFO (OverseerStateUpdate-73138548910391306-127.0.0.1:41935_solr-n_0000000001) [n:127.0.0.1:41935_solr ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:41935_solr [junit4] 2> 1666452 INFO (OverseerCollectionConfigSetProcessor-73138548910391306-127.0.0.1:41935_solr-n_0000000001) [n:127.0.0.1:41935_solr ] o.a.s.c.OverseerNodePrioritizer prioritizing overseer nodes at 73138548910391306-127.0.0.1:41935_solr-n_0000000001 overseer designates are [127.0.0.1:46527_solr] [junit4] 2> 1666453 INFO (OverseerCollectionConfigSetProcessor-73138548910391306-127.0.0.1:41935_solr-n_0000000001) [n:127.0.0.1:41935_solr ] o.a.s.c.OverseerNodePrioritizer sorted nodes [73138548910391306-127.0.0.1:41935_solr-n_0000000001, 73138548910391310-127.0.0.1:42464_solr-n_0000000002, 73138548910391311-127.0.0.1:46527_solr-n_0000000003, 73138548910391304-127.0.0.1:37155_solr-n_0000000004] [junit4] 2> 1666453 INFO (OverseerCollectionConfigSetProcessor-73138548910391306-127.0.0.1:41935_solr-n_0000000001) [n:127.0.0.1:41935_solr ] o.a.s.c.OverseerNodePrioritizer asking node 73138548910391311-127.0.0.1:46527_solr-n_0000000003 to come join election at head [junit4] 2> 1666457 INFO (qtp1147157588-20875) [n:127.0.0.1:46527_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={op=rejoinAtHead&qt=/admin/cores&action=OVERSEEROP&electionNode=73138548910391311-127.0.0.1:46527_solr-n_0000000003&wt=javabin&version=2} status=0 QTime=0 [junit4] 2> 1666458 INFO (OverseerCollectionConfigSetProcessor-73138548910391306-127.0.0.1:41935_solr-n_0000000001) [n:127.0.0.1:41935_solr ] o.a.s.c.OverseerNodePrioritizer asking the old first in line 73138548910391310-127.0.0.1:42464_solr-n_0000000002 to rejoin election [junit4] 2> 1666462 INFO (qtp878210393-20891) [n:127.0.0.1:42464_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={op=rejoin&qt=/admin/cores&action=OVERSEEROP&electionNode=73138548910391310-127.0.0.1:42464_solr-n_0000000002&wt=javabin&version=2} status=0 QTime=0 [junit4] 2> 1666464 INFO (OverseerStateUpdate-73138548910391306-127.0.0.1:41935_solr-n_0000000001) [n:127.0.0.1:41935_solr ] o.a.s.c.Overseer Quit command received { [junit4] 2> "operation":"quit", [junit4] 2> "id":"73138548910391306-127.0.0.1:41935_solr-n_0000000001"} 127.0.0.1:41935_solr [junit4] 2> 1666465 INFO (OverseerStateUpdate-73138548910391306-127.0.0.1:41935_solr-n_0000000001) [n:127.0.0.1:41935_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:41935_solr [junit4] 2> 1666467 WARN (OverseerExitThread) [ ] o.a.s.c.Overseer I (id=73138548910391306-127.0.0.1:41935_solr-n_0000000001) am exiting, but I'm still the leader [junit4] 2> 1666468 INFO (OverseerExitThread) [ ] o.a.s.c.Overseer Overseer (id=73138548910391306-127.0.0.1:41935_solr-n_0000000001) closing [junit4] 2> 1666468 INFO (OverseerAutoScalingTriggerThread-73138548910391306-127.0.0.1:41935_solr-n_0000000001) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting. [junit4] 2> 1666472 INFO (zkCallback-8192-thread-1) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:42464_solr [junit4] 2> 1666473 INFO (zkCallback-8192-thread-1) [n:127.0.0.1:42464_solr ] o.a.s.c.Overseer Overseer (id=73138548910391310-127.0.0.1:42464_solr-n_0000000002) starting [junit4] 2> 1666478 INFO (OverseerStateUpdate-73138548910391310-127.0.0.1:42464_solr-n_0000000002) [n:127.0.0.1:42464_solr ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:42464_solr [junit4] 2> 1666480 INFO (OverseerCollectionConfigSetProcessor-73138548910391310-127.0.0.1:42464_solr-n_0000000002) [n:127.0.0.1:42464_solr ] o.a.s.c.OverseerNodePrioritizer prioritizing overseer nodes at 73138548910391310-127.0.0.1:42464_solr-n_0000000002 overseer designates are [127.0.0.1:46527_solr] [junit4] 2> 1666481 INFO (OverseerCollectionConfigSetProcessor-73138548910391310-127.0.0.1:42464_solr-n_0000000002) [n:127.0.0.1:42464_solr ] o.a.s.c.OverseerNodePrioritizer sorted nodes [73138548910391310-127.0.0.1:42464_solr-n_0000000002, 73138548910391311-127.0.0.1:46527_solr-n_0000000003, 73138548910391304-127.0.0.1:37155_solr-n_0000000004] [junit4] 2> 1666483 INFO (OverseerStateUpdate-73138548910391310-127.0.0.1:42464_solr-n_0000000002) [n:127.0.0.1:42464_solr ] o.a.s.c.Overseer Quit command received { [junit4] 2> "operation":"quit", [junit4] 2> "id":"73138548910391310-127.0.0.1:42464_solr-n_0000000002"} 127.0.0.1:42464_solr [junit4] 2> 1666485 INFO (OverseerStateUpdate-73138548910391310-127.0.0.1:42464_solr-n_0000000002) [n:127.0.0.1:42464_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:42464_solr [junit4] 2> 1666485 INFO (OverseerExitThread) [ ] o.a.s.c.Overseer Overseer (id=73138548910391306-127.0.0.1:41935_solr-n_0000000001) closing [junit4] 2> 1666485 INFO (OverseerExitThread) [ ] o.a.s.c.Overseer Overseer (id=73138548910391306-127.0.0.1:41935_solr-n_0000000001) closing [junit4] 2> 1666493 INFO (OverseerExitThread) [ ] o.a.s.c.Overseer Overseer (id=73138548910391306-127.0.0.1:41935_solr-n_0000000001) closing [junit4] 2> 1666494 WARN (OverseerExitThread) [ ] o.a.s.c.Overseer I (id=73138548910391310-127.0.0.1:42464_solr-n_0000000002) am exiting, but I'm still the leader [junit4] 2> 1666496 INFO (OverseerExitThread) [ ] o.a.s.c.Overseer Overseer (id=73138548910391310-127.0.0.1:42464_solr-n_0000000002) closing [junit4] 2> 1666496 INFO (zkCallback-8190-thread-1) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:46527_solr [junit4] 2> 1666497 INFO (zkCallback-8190-thread-1) [n:127.0.0.1:46527_solr ] o.a.s.c.Overseer Overseer (id=73138548910391311-127.0.0.1:46527_solr-n_0000000003) starting [junit4] 2> 1666504 INFO (OverseerStateUpdate-73138548910391311-127.0.0.1:46527_solr-n_0000000003) [n:127.0.0.1:46527_solr ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:46527_solr [junit4] 2> 1666504 INFO (OverseerAutoScalingTriggerThread-73138548910391310-127.0.0.1:42464_solr-n_0000000002) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed, exiting. [junit4] 2> 1666519 INFO (OverseerExitThread) [ ] o.a.s.c.Overseer Overseer (id=73138548910391310-127.0.0.1:42464_solr-n_0000000002) closing [junit4] 2> 1666519 INFO (OverseerExitThread) [ ] o.a.s.c.Overseer Overseer (id=73138548910391310-127.0.0.1:42464_solr-n_0000000002) closing [junit4] 2> 1666520 INFO (TEST-OverseerRolesTest.testOverseerRole-seed#[62094B24FCF12C8B]) [ ] o.a.s.c.OverseerRolesTest ### Adding another overseer designate 127.0.0.1:42464_solr [junit4] 2> 1666521 INFO (qtp878210393-20892) [n:127.0.0.1:42464_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addrole with params node=127.0.0.1:42464_solr&role=overseer&action=ADDROLE&wt=javabin&version=2 and sendToOCPQueue=true [junit4] 2> 1666527 INFO (OverseerExitThread) [ ] o.a.s.c.Overseer Overseer (id=73138548910391310-127.0.0.1:42464_solr-n_0000000002) closing [junit4] 2> 1666549 INFO (OverseerThreadFactory-7577-thread-1-processing-n:127.0.0.1:46527_solr) [n:127.0.0.1:46527_solr ] o.a.s.c.a.c.OverseerRoleCmd Overseer role added to 127.0.0.1:42464_solr [junit4] 2> 1666552 INFO (qtp878210393-20892) [n:127.0.0.1:42464_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:42464_solr&role=overseer&action=ADDROLE&wt=javabin&version=2} status=0 QTime=31 [junit4] 2> 1666561 INFO (TEST-OverseerRolesTest.testOverseerRole-seed#[62094B24FCF12C8B]) [ ] o.a.s.c.OverseerRolesTest Overseer: 127.0.0.1:46527_solr [junit4] 2> 1666561 INFO (TEST-OverseerRolesTest.testOverseerRole-seed#[62094B24FCF12C8B]) [ ] o.a.s.c.OverseerRolesTest Election queue: [junit4] 2> 1666563 INFO (TEST-OverseerRolesTest.testOverseerRole-seed#[62094B24FCF12C8B]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1288519915 [junit4] 2> 1666563 INFO (TEST-OverseerRolesTest.testOverseerRole-seed#[62094B24FCF12C8B]) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:46527_solr [junit4] 2> 1666564 INFO (TEST-OverseerRolesTest.testOverseerRole-seed#[62094B24FCF12C8B]) [ ] o.a.s.c.ZkController Publish this node as DOWN... [junit4] 2> 1666564 INFO (TEST-OverseerRolesTest.testOverseerRole-seed#[62094B24FCF12C8B]) [ ] o.a.s.c.ZkController Publish node=127.0.0.1:46527_solr as DOWN [junit4] 2> 1666575 INFO (TEST-OverseerRolesTest.testOverseerRole-seed#[62094B24FCF12C8B]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null [junit4] 2> 1666575 INFO (TEST-OverseerRolesTest.testOverseerRole-seed#[62094B24FCF12C8B]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1e440df4: rootName = solr_46527, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@19f956dc [junit4] 2> 1666586 INFO (TEST-OverseerRolesTest.testOverseerRole-seed#[62094B24FCF12C8B]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null [junit4] 2> 1666586 INFO (TEST-OverseerRolesTest.testOverseerRole-seed#[62094B24FCF12C8B]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1f5668df: rootName = solr_46527, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@1b12394d [junit4] 2> 1666593 INFO (TEST-OverseerRolesTest.testOverseerRole-seed#[62094B24FCF12C8B]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null [junit4] 2> 1666593 INFO (TEST-OverseerRolesTest.testOverseerRole-seed#[62094B24FCF12C8B]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@780e5cfd: rootName = solr_46527, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@615b52e5 [junit4] 2> 1666593 INFO (TEST-OverseerRolesTest.testOverseerRole-seed#[62094B24FCF12C8B]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null [junit4] 2> 1666595 INFO (closeThreadPool-8225-thread-1) [ ] o.a.s.c.Overseer Overseer (id=73138548910391311-127.0.0.1:46527_solr-n_0000000003) closing [junit4] 2> 1666595 INFO (OverseerStateUpdate-73138548910391311-127.0.0.1:46527_solr-n_0000000003) [n:127.0.0.1:46527_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:46527_solr [junit4] 2> 1666595 INFO (OverseerAutoScalingTriggerThread-73138548910391311-127.0.0.1:46527_solr-n_0000000003) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting. [junit4] 2> 1666605 INFO (zkCallback-8162-thread-1) [ ] o.a.s.c.Overseer Overseer (id=73138548910391304-127.0.0.1:37155_solr-n_0000000000) closing [junit4] 2> 1666605 INFO (TEST-OverseerRolesTest.testOverseerRole-seed#[62094B24FCF12C8B]) [ ] o.a.s.c.Overseer Overseer (id=73138548910391311-127.0.0.1:46527_solr-n_0000000003) closing [junit4] 2> 1666605 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x103d71f6b31000f, likely client has closed socket [junit4] 2> 1666605 INFO (closeThreadPool-8225-thread-2) [ ] o.a.s.c.Overseer Overseer (id=73138548910391311-127.0.0.1:46527_solr-n_0000000003) closing [junit4] 2> 1666606 INFO (zkCallback-8162-thread-1) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:37155_solr [junit4] 2> 1666606 INFO (zkCallback-8192-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3) [junit4] 2> 1666606 INFO (zkCallback-8176-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3) [junit4] 2> 1666606 INFO (zkCallback-8167-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3) [junit4] 2> 1666608 INFO (zkCallback-8162-thread-1) [n:127.0.0.1:37155_solr ] o.a.s.c.Overseer Overseer (id=73138548910391304-127.0.0.1:37155_solr-n_0000000004) starting [junit4] 2> 1666608 INFO (TEST-OverseerRolesTest.testOverseerRole-seed#[62094B24FCF12C8B]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@35a35d59{HTTP/1.1,[http/1.1, h2c]}{127.0.0.1:0} [junit4] 2> 1666613 INFO (TEST-OverseerRolesTest.testOverseerRole-seed#[62094B24FCF12C8B]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@7446e59a{/solr,null,UNAVAILABLE} [junit4] 2> 1666615 INFO (zkCallback-8202-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3) [junit4] 2> 1666615 INFO (TEST-OverseerRolesTest.testOverseerRole-seed#[62094B24FCF12C8B]) [ ] o.e.j.s.session node0 Stopped scavenging [junit4] 2> 1666616 INFO (zkCallback-8220-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3) [junit4] 2> 1666616 INFO (zkCallback-8207-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3) [junit4] 2> 1666617 INFO (zkCallback-8162-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3) [junit4] 2> 1666618 INFO (OverseerStateUpdate-73138548910391304-127.0.0.1:37155_solr-n_0000000004) [n:127.0.0.1:37155_solr ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:37155_solr [junit4] 2> 1666622 INFO (OverseerCollectionConfigSetProcessor-73138548910391304-127.0.0.1:37155_solr-n_0000000004) [n:127.0.0.1:37155_solr ] o.a.s.c.OverseerNodePrioritizer prioritizing overseer nodes at 73138548910391304-127.0.0.1:37155_solr-n_0000000004 overseer designates are [127.0.0.1:46527_solr, 127.0.0.1:42464_solr] [junit4] 2> 1666623 INFO (OverseerCollectionConfigSetProcessor-73138548910391304-127.0.0.1:37155_solr-n_0000000004) [n:127.0.0.1:37155_solr ] o.a.s.c.OverseerNodePrioritizer sorted nodes [73138548910391304-127.0.0.1:37155_solr-n_0000000004, 73138548910391306-127.0.0.1:41935_solr-n_0000000005, 73138548910391310-127.0.0.1:42464_solr-n_0000000006] [junit4] 2> 1666623 INFO (OverseerCollectionConfigSetProcessor-73138548910391304-127.0.0.1:37155_solr-n_0000000004) [n:127.0.0.1:37155_solr ] o.a.s.c.OverseerNodePrioritizer asking node 73138548910391310-127.0.0.1:42464_solr-n_0000000006 to come join election at head [junit4] 2> 1666625 INFO (qtp878210393-20889) [n:127.0.0.1:42464_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={op=rejoinAtHead&qt=/admin/cores&action=OVERSEEROP&electionNode=73138548910391310-127.0.0.1:42464_solr-n_0000000006&wt=javabin&version=2} status=0 QTime=0 [junit4] 2> 1666625 INFO (OverseerCollectionConfigSetProcessor-73138548910391304-127.0.0.1:37155_solr-n_0000000004) [n:127.0.0.1:37155_solr ] o.a.s.c.OverseerNodePrioritizer asking the old first in line 73138548910391306-127.0.0.1:41935_solr-n_0000000005 to rejoin election [junit4] 2> 1666626 INFO (qtp636615717-20861) [n:127.0.0.1:41935_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={op=rejoin&qt=/admin/cores&action=OVERSEEROP&electionNode=73138548910391306-127.0.0.1:41935_solr-n_0000000005&wt=javabin&version=2} status=0 QTime=0 [junit4] 2> 1666629 INFO (OverseerStateUpdate-73138548910391304-127.0.0.1:37155_solr-n_0000000004) [n:127.0.0.1:37155_solr ] o.a.s.c.Overseer Quit command received { [junit4] 2> "operation":"quit", [junit4] 2> "id":"73138548910391304-127.0.0.1:37155_solr-n_0000000004"} 127.0.0.1:37155_solr [junit4] 2> 1666630 INFO (OverseerStateUpdate-73138548910391304-127.0.0.1:37155_solr-n_0000000004) [n:127.0.0.1:37155_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:37155_solr [junit4] 2> 1666636 WARN (OverseerExitThread) [ ] o.a.s.c.Overseer I (id=73138548910391304-127.0.0.1:37155_solr-n_0000000004) am exiting, but I'm still the leader [junit4] 2> 1666643 INFO (OverseerExitThread) [ ] o.a.s.c.Overseer Overseer (id=73138548910391304-127.0.0.1:37155_solr-n_0000000004) closing [junit4] 2> 1666643 INFO (zkCallback-8167-thread-1) [ ] o.a.s.c.Overseer Overseer (id=73138548910391306-127.0.0.1:41935_solr-n_0000000001) closing [junit4] 2> 1666645 INFO (OverseerAutoScalingTriggerThread-73138548910391304-127.0.0.1:37155_solr-n_0000000004) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed, exiting. [junit4] 2> 1666645 INFO (zkCallback-8167-thread-1) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:41935_solr [junit4] 2> 1666646 INFO (zkCallback-8167-thread-1) [n:127.0.0.1:41935_solr ] o.a.s.c.Overseer Overseer (id=73138548910391306-127.0.0.1:41935_solr-n_0000000005) starting [junit4] 2> 1666662 INFO (OverseerStateUpdate-73138548910391306-127.0.0.1:41935_solr-n_0000000005) [n:127.0.0.1:41935_solr ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:41935_solr [junit4] 2> 1666668 WARN (ScheduledTrigger-7580-thread-1) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger ran but was already closed [junit4] 2> 1666669 INFO (OverseerExitThread) [ ] o.a.s.c.Overseer Overseer (id=73138548910391304-127.0.0.1:37155_solr-n_0000000004) closing [junit4] 2> 1666669 INFO (OverseerExitThread) [ ] o.a.s.c.Overseer Overseer (id=73138548910391304-127.0.0.1:37155_solr-n_0000000004) closing [junit4] 2> 1666670 INFO (OverseerExitThread) [ ] o.a.s.c.Overseer Overseer (id=73138548910391304-127.0.0.1:37155_solr-n_0000000004) closing [junit4] 2> 1666672 INFO (OverseerCollectionConfigSetProcessor-73138548910391306-127.0.0.1:41935_solr-n_0000000005) [n:127.0.0.1:41935_solr ] o.a.s.c.OverseerNodePrioritizer prioritizing overseer nodes at 73138548910391306-127.0.0.1:41935_solr-n_0000000005 overseer designates are [127.0.0.1:46527_solr, 127.0.0.1:42464_solr] [junit4] 2> 1666672 INFO (OverseerCollectionConfigSetProcessor-73138548910391306-127.0.0.1:41935_solr-n_0000000005) [n:127.0.0.1:41935_solr ] o.a.s.c.OverseerNodePrioritizer sorted nodes [73138548910391306-127.0.0.1:41935_solr-n_0000000005, 73138548910391310-127.0.0.1:42464_solr-n_0000000006, 73138548910391304-127.0.0.1:37155_solr-n_0000000007] [junit4] 2> 1666675 INFO (OverseerStateUpdate-73138548910391306-127.0.0.1:41935_solr-n_0000000005) [n:127.0.0.1:41935_solr ] o.a.s.c.Overseer Quit command received { [junit4] 2> "operation":"quit", [junit4] 2> "id":"73138548910391306-127.0.0.1:41935_solr-n_0000000005"} 127.0.0.1:41935_solr [junit4] 2> 1666675 INFO (OverseerStateUpdate-73138548910391306-127.0.0.1:41935_solr-n_0000000005) [n:127.0.0.1:41935_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:41935_solr [junit4] 2> 1666687 WARN (OverseerExitThread) [ ] o.a.s.c.Overseer I (id=73138548910391306-127.0.0.1:41935_solr-n_0000000005) am exiting, but I'm still the leader [junit4] 2> 1666689 INFO (zkCallback-8192-thread-1) [ ] o.a.s.c.Overseer Overseer (id=73138548910391310-127.0.0.1:42464_solr-n_0000000002) closing [junit4] 2> 1666689 INFO (zkCallback-8192-thread-1) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:42464_solr [junit4] 2> 1666690 INFO (zkCallback-8192-thread-1) [n:127.0.0.1:42464_solr ] o.a.s.c.Overseer Overseer (id=73138548910391310-127.0.0.1:42464_solr-n_0000000006) starting [junit4] 2> 1666694 INFO (OverseerExitThread) [ ] o.a.s.c.Overseer Overseer (id=73138548910391306-127.0.0.1:41935_solr-n_0000000005) closing [junit4] 2> 1666694 INFO (OverseerAutoScalingTriggerThread-73138548910391306-127.0.0.1:41935_solr-n_0000000005) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting. [junit4] 2> 1666702 INFO (OverseerStateUpdate-73138548910391310-127.0.0.1:42464_solr-n_0000000006) [n:127.0.0.1:42464_solr ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:42464_solr [junit4] 2> 1666704 INFO (OverseerExitThread) [ ] o.a.s.c.Overseer Overseer (id=73138548910391306-127.0.0.1:41935_solr-n_0000000005) closing [junit4] 2> 1666704 INFO (OverseerExitThread) [ ] o.a.s.c.Overseer Overseer (id=73138548910391306-127.0.0.1:41935_solr-n_0000000005) closing [junit4] 2> 1666705 INFO (OverseerExitThread) [ ] o.a.s.c.Overseer Overseer (id=73138548910391306-127.0.0.1:41935_solr-n_0000000005) closing [junit4] 2> 1666717 INFO (TEST-OverseerRolesTest.testOverseerRole-seed#[62094B24FCF12C8B]) [ ] o.a.s.c.OverseerRolesTest ### Adding last overseer designate 127.0.0.1:41935_solr [junit4] 2> 1666736 INFO (qtp1067742412-20873) [n:127.0.0.1:37155_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addrole with params node=127.0.0.1:41935_solr&role=overseer&action=ADDROLE&wt=javabin&version=2 and sendToOCPQueue=true [junit4] 2> 1666776 INFO (OverseerThreadFactory-7590-thread-1-processing-n:127.0.0.1:42464_solr) [n:127.0.0.1:42464_solr ] o.a.s.c.a.c.OverseerRoleCmd Overseer role added to 127.0.0.1:41935_solr [junit4] 2> 1666798 INFO (qtp1067742412-20873) [n:127.0.0.1:37155_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:41935_solr&role=overseer&action=ADDROLE&wt=javabin&version=2} status=0 QTime=62 [junit4] 2> 1666799 INFO (TEST-OverseerRolesTest.testOverseerRole-seed#[62094B24FCF12C8B]) [ ] o.a.s.c.OverseerRolesTest Overseer: 127.0.0.1:42464_solr [junit4] 2> 1666799 INFO (TEST-OverseerRolesTest.testOverseerRole-seed#[62094B24FCF12C8B]) [ ] o.a.s.c.OverseerRolesTest Election queue: [junit4] 2> 1666800 INFO (qtp1067742412-20865) [n:127.0.0.1:37155_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :removerole with params node=127.0.0.1:42464_solr&role=overseer&action=REMOVEROLE&wt=javabin&version=2 and sendToOCPQueue=true [junit4] 2> 1666810 INFO (OverseerCollectionConfigSetProcessor-73138548910391310-127.0.0.1:42464_solr-n_0000000006) [n:127.0.0.1:42464_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist. Requestor may have disconnected from ZooKeeper [junit4] 2> 1666811 INFO (OverseerThreadFactory-7590-thread-2-processing-n:127.0.0.1:42464_solr) [n:127.0.0.1:42464_solr ] o.a.s.c.a.c.OverseerRoleCmd Overseer role removed from 127.0.0.1:42464_solr [junit4] 2> 1666823 INFO (qtp1067742412-20865) [n:127.0.0.1:37155_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:42464_solr&role=overseer&action=REMOVEROLE&wt=javabin&version=2} status=0 QTime=23 [junit4] 2> 1666824 INFO (Thread-3139) [ ] o.a.s.c.OverseerNodePrioritizer prioritizing overseer nodes at 73138548910391310-127.0.0.1:42464_solr-n_0000000006 overseer designates are [127.0.0.1:46527_solr, 127.0.0.1:41935_solr] [junit4] 2> 1666824 INFO (Thread-3139) [ ] o.a.s.c.OverseerNodePrioritizer sorted nodes [73138548910391310-127.0.0.1:42464_solr-n_0000000006, 73138548910391304-127.0.0.1:37155_solr-n_0000000007, 73138548910391306-127.0.0.1:41935_solr-n_0000000008] [junit4] 2> 1666824 INFO (Thread-3139) [ ] o.a.s.c.OverseerNodePrioritizer asking node 73138548910391306-127.0.0.1:41935_solr-n_0000000008 to come join election at head [junit4] 2> 1666827 INFO (qtp636615717-20857) [n:127.0.0.1:41935_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={op=rejoinAtHead&qt=/admin/cores&action=OVERSEEROP&electionNode=73138548910391306-127.0.0.1:41935_solr-n_0000000008&wt=javabin&version=2} status=0 QTime=0 [junit4] 2> 1666828 INFO (Thread-3139) [ ] o.a.s.c.OverseerNodePrioritizer asking the old first in line 73138548910391304-127.0.0.1:37155_solr-n_0000000007 to rejoin election [junit4] 2> 1666830 INFO (qtp1067742412-20874) [n:127.0.0.1:37155_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={op=rejoin&qt=/admin/cores&action=OVERSEEROP&electionNode=73138548910391304-127.0.0.1:37155_solr-n_0000000007&wt=javabin&version=2} status=0 QTime=0 [junit4] 2> 1666841 INFO (OverseerStateUpdate-73138548910391310-127.0.0.1:42464_solr-n_0000000006) [n:127.0.0.1:42464_solr ] o.a.s.c.Overseer Quit command received { [junit4] 2> "operation":"quit", [junit4] 2> "id":"73138548910391310-127.0.0.1:42464_solr-n_0000000006"} 127.0.0.1:42464_solr [junit4] 2> 1666842 INFO (OverseerStateUpdate-73138548910391310-127.0.0.1:42464_solr-n_0000000006) [n:127.0.0.1:42464_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:42464_solr [junit4] 2> 1666860 WARN (OverseerExitThread) [ ] o.a.s.c.Overseer I (id=73138548910391310-127.0.0.1:42464_solr-n_0000000006) am exiting, but I'm still the leader [junit4] 2> 1666862 INFO (zkCallback-8162-thread-2) [ ] o.a.s.c.Overseer Overseer (id=73138548910391304-127.0.0.1:37155_solr-n_0000000004) closing [junit4] 2> 1666862 INFO (OverseerExitThread) [ ] o.a.s.c.Overseer Overseer (id=73138548910391310-127.0.0.1:42464_solr-n_0000000006) closing [junit4] 2> 1666862 INFO (zkCallback-8162-thread-2) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:37155_solr [junit4] 2> 1666862 INFO (OverseerAutoScalingTriggerThread-73138548910391310-127.0.0.1:42464_solr-n_0000000006) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting. [junit4] 2> 1666864 INFO (zkCallback-8162-thread-2) [n:127.0.0.1:37155_solr ] o.a.s.c.Overseer Overseer (id=73138548910391304-127.0.0.1:37155_solr-n_0000000007) starting [junit4] 2> 1666865 INFO (OverseerExitThread) [ ] o.a.s.c.Overseer Overseer (id=73138548910391310-127.0.0.1:42464_solr-n_0000000006) closing [junit4] 2> 1666865 INFO (OverseerExitThread) [ ] o.a.s.c.Overseer Overseer (id=73138548910391310-127.0.0.1:42464_solr-n_0000000006) closing [junit4] 2> 1666865 INFO (OverseerExitThread) [ ] o.a.s.c.Overseer Overseer (id=73138548910391310-127.0.0.1:42464_solr-n_0000000006) closing [junit4] 2> 1666870 INFO (OverseerStateUpdate-73138548910391304-127.0.0.1:37155_solr-n_0000000007) [n:127.0.0.1:37155_solr ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:37155_solr [junit4] 2> 1666872 INFO (OverseerCollectionConfigSetProcessor-73138548910391304-127.0.0.1:37155_solr-n_0000000007) [n:127.0.0.1:37155_solr ] o.a.s.c.OverseerNodePrioritizer prioritizing overseer nodes at 73138548910391304-127.0.0.1:37155_solr-n_0000000007 overseer designates are [127.0.0.1:46527_solr, 127.0.0.1:41935_solr] [junit4] 2> 1666872 INFO (OverseerCollectionConfigSetProcessor-73138548910391304-127.0.0.1:37155_solr-n_0000000007) [n:127.0.0.1:37155_solr ] o.a.s.c.OverseerNodePrioritizer sorted nodes [73138548910391304-127.0.0.1:37155_solr-n_0000000007, 73138548910391306-127.0.0.1:41935_solr-n_0000000008, 73138548910391310-127.0.0.1:42464_solr-n_0000000009] [junit4] 2> 1666884 INFO (OverseerStateUpdate-73138548910391304-127.0.0.1:37155_solr-n_0000000007) [n:127.0.0.1:37155_solr ] o.a.s.c.Overseer Quit command received { [junit4] 2> "operation":"quit", [junit4] 2> "id":"73138548910391304-127.0.0.1:37155_solr-n_0000000007"} 127.0.0.1:37155_solr [junit4] 2> 1666885 INFO (OverseerStateUpdate-73138548910391304-127.0.0.1:37155_solr-n_0000000007) [n:127.0.0.1:37155_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:37155_solr [junit4] 2> 1666899 WARN (OverseerExitThread) [ ] o.a.s.c.Overseer I (id=73138548910391304-127.0.0.1:37155_solr-n_0000000007) am exiting, but I'm still the leader [junit4] 2> 1666901 INFO (zkCallback-8167-thread-1) [ ] o.a.s.c.Overseer Overseer (id=73138548910391306-127.0.0.1:41935_solr-n_0000000005) closing [junit4] 2> 1666901 INFO (OverseerExitThread) [ ] o.a.s.c.Overseer Overseer (id=73138548910391304-127.0.0.1:37155_solr-n_0000000007) closing [junit4] 2> 1666901 INFO (OverseerAutoScalingTriggerThread-73138548910391304-127.0.0.1:37155_solr-n_0000000007) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting. [junit4] 2> 1666901 INFO (zkCallback-8167-thread-1) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:41935_solr [junit4] 2> 1666902 INFO (zkCallback-8167-thread-1) [n:127.0.0.1:41935_solr ] o.a.s.c.Overseer Overseer (id=73138548910391306-127.0.0.1:41935_solr-n_0000000008) starting [junit4] 2> 1666923 INFO (OverseerStateUpdate-73138548910391306-127.0.0.1:41935_solr-n_0000000008) [n:127.0.0.1:41935_solr ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:41935_solr [junit4] 2> 1666925 INFO (qtp1067742412-20870) [n:127.0.0.1:37155_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addrole with params node=127.0.0.1:42464_solr&role=overseer&action=ADDROLE&wt=javabin&version=2 and sendToOCPQueue=true [junit4] 2> 1666939 INFO (OverseerExitThread) [ ] o.a.s.c.Overseer Overseer (id=73138548910391304-127.0.0.1:37155_solr-n_0000000007) closing [junit4] 2> 1666939 INFO (OverseerExitThread) [ ] o.a.s.c.Overseer Overseer (id=73138548910391304-127.0.0.1:37155_solr-n_0000000007) closing [junit4] 2> 1666940 INFO (OverseerExitThread) [ ] o.a.s.c.Overseer Overseer (id=73138548910391304-127.0.0.1:37155_solr-n_0000000007) closing [junit4] 2> 1666949 INFO (OverseerThreadFactory-7598-thread-1-processing-n:127.0.0.1:41935_solr) [n:127.0.0.1:41935_solr ] o.a.s.c.a.c.OverseerRoleCmd Overseer role added to 127.0.0.1:42464_solr [junit4] 2> 1666956 INFO (qtp1067742412-20870) [n:127.0.0.1:37155_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:42464_solr&role=overseer&action=ADDROLE&wt=javabin&version=2} status=0 QTime=30 [junit4] 2> 1666957 INFO (TEST-OverseerRolesTest.testOverseerRole-seed#[62094B24FCF12C8B]) [ ] o.a.s.c.OverseerRolesTest ### Sending QUIT to overseer 127.0.0.1:41935_solr [junit4] 2> 1666960 INFO (OverseerStateUpdate-73138548910391306-127.0.0.1:41935_solr-n_0000000008) [n:127.0.0.1:41935_solr ] o.a.s.c.Overseer Quit command received { [junit4] 2> "operation":"quit", [junit4] 2> "id":"73138548910391306-127.0.0.1:41935_solr-n_0000000008"} 127.0.0.1:41935_solr [junit4] 2> 1666961 INFO (OverseerStateUpdate-73138548910391306-127.0.0.1:41935_solr-n_0000000008) [n:127.0.0.1:41935_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:41935_solr [junit4] 2> 1666973 WARN (OverseerExitThread) [ ] o.a.s.c.Overseer I (id=73138548910391306-127.0.0.1:41935_solr-n_0000000008) am exiting, but I'm still the leader [junit4] 2> 1666974 INFO (Thread-3140) [ ] o.a.s.c.OverseerNodePrioritizer prioritizing overseer nodes at 73138548910391306-127.0.0.1:41935_solr-n_0000000008 overseer designates are [127.0.0.1:46527_solr, 127.0.0.1:41935_solr, 127.0.0.1:42464_solr] [junit4] 2> 1666974 INFO (Thread-3140) [ ] o.a.s.c.OverseerNodePrioritizer sorted nodes [73138548910391306-127.0.0.1:41935_solr-n_0000000008, 73138548910391310-127.0.0.1:42464_solr-n_0000000009, 73138548910391304-127.0.0.1:37155_solr-n_0000000010] [junit4] 2> 1666975 INFO (Thread-3140) [ ] o.a.s.c.OverseerNodePrioritizer asking node 73138548910391306-127.0.0.1:41935_solr-n_0000000008 to come join election at head [junit4] 2> 1666975 INFO (zkCallback-8192-thread-1) [ ] o.a.s.c.Overseer Overseer (id=73138548910391310-127.0.0.1:42464_solr-n_0000000006) closing [junit4] 2> 1666975 INFO (OverseerExitThread) [ ] o.a.s.c.Overseer Overseer (id=73138548910391306-127.0.0.1:41935_solr-n_0000000008) closing [junit4] 2> 1666975 INFO (OverseerAutoScalingTriggerThread-73138548910391306-127.0.0.1:41935_solr-n_0000000008) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting. [junit4] 2> 1666976 INFO (zkCallback-8192-thread-1) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:42464_solr [junit4] 2> 1666977 INFO (qtp636615717-20860) [n:127.0.0.1:41935_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={op=rejoinAtHead&qt=/admin/cores&action=OVERSEEROP&electionNode=73138548910391306-127.0.0.1:41935_solr-n_0000000008&wt=javabin&version=2} status=0 QTime=0 [junit4] 2> 1666978 INFO (OverseerExitThread) [ ] o.a.s.c.Overseer Overseer (id=73138548910391306-127.0.0.1:41935_solr-n_0000000008) closing [junit4] 2> 1666978 INFO (OverseerExitThread) [ ] o.a.s.c.Overseer Overseer (id=73138548910391306-127.0.0.1:41935_solr-n_0000000008) closing [junit4] 2> 1666978 INFO (Thread-3140) [ ] o.a.s.c.OverseerNodePrioritizer asking the old first in line 73138548910391310-127.0.0.1:42464_solr-n_0000000009 to rejoin election [junit4] 2> 1666979 INFO (zkCallback-8192-thread-1) [n:127.0.0.1:42464_solr ] o.a.s.c.Overseer Overseer (id=73138548910391310-127.0.0.1:42464_solr-n_0000000009) starting [junit4] 2> 1666979 INFO (qtp878210393-20890) [n:127.0.0.1:42464_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={op=rejoin&qt=/admin/cores&action=OVERSEEROP&electionNode=73138548910391310-127.0.0.1:42464_solr-n_0000000009&wt=javabin&version=2} status=0 QTime=0 [junit4] 2> 1666980 INFO (OverseerExitThread) [ ] o.a.s.c.Overseer Overseer (id=73138548910391306-127.0.0.1:41935_solr-n_0000000008) closing [junit4] 2> 1666983 INFO (OverseerStateUpdate-73138548910391310-127.0.0.1:42464_solr-n_0000000009) [n:127.0.0.1:42464_solr ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:42464_solr [junit4] 2> 1666986 INFO (OverseerStateUpdate-73138548910391310-127.0.0.1:42464_solr-n_0000000009) [n:127.0.0.1:42464_solr ] o.a.s.c.Overseer Quit command received { [junit4] 2> "operation":"quit", [junit4] 2> "id":"73138548910391310-127.0.0.1:42464_solr-n_0000000009"} 127.0.0.1:42464_solr [junit4] 2> 1666986 INFO (OverseerStateUpdate-73138548910391310-127.0.0.1:42464_solr-n_0000000009) [n:127.0.0.1:42464_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:42464_solr [junit4] 2> 1666997 WARN (OverseerExitThread) [ ] o.a.s.c.Overseer I (id=73138548910391310-127.0.0.1:42464_solr-n_0000000009) am exiting, but I'm still the leader [junit4] 2> 1666998 INFO (zkCallback-8162-thread-2) [ ] o.a.s.c.Overseer Overseer (id=73138548910391304-127.0.0.1:37155_solr-n_0000000007) closing [junit4] 2> 1666999 INFO (OverseerExitThread) [ ] o.a.s.c.Overseer Overseer (id=73138548910391310-127.0.0.1:42464_solr-n_0000000009) closing [junit4] 2> 1666999 INFO (zkCallback-8162-thread-2) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:37155_solr [junit4] 2> 1667003 INFO (OverseerAutoScalingTriggerThread-73138548910391310-127.0.0.1:42464_solr-n_0000000009) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed, exiting. [junit4] 2> 1667004 INFO (OverseerExitThread) [ ] o.a.s.c.Overseer Overseer (id=73138548910391310-127.0.0.1:42464_solr-n_0000000009) closing [junit4] 2> 1667004 INFO (OverseerExitThread) [ ] o.a.s.c.Overseer Overseer (id=73138548910391310-127.0.0.1:42464_solr-n_0000000009) closing [junit4] 2> 1667004 INFO (zkCallback-8162-thread-2) [n:127.0.0.1:37155_solr ] o.a.s.c.Overseer Overseer (id=73138548910391304-127.0.0.1:37155_solr-n_0000000010) starting [junit4] 2> 1667004 INFO (OverseerExitThread) [ ] o.a.s.c.Overseer Overseer (id=73138548910391310-127.0.0.1:42464_solr-n_0000000009) closing [junit4] 2> 1667008 INFO (OverseerStateUpdate-73138548910391304-127.0.0.1:37155_solr-n_0000000010) [n:127.0.0.1:37155_solr ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:37155_solr [junit4] 2> 1667014 INFO (OverseerCollectionConfigSetProcessor-73138548910391304-127.0.0.1:37155_solr-n_0000000010) [n:127.0.0.1:37155_solr ] o.a.s.c.OverseerNodePrioritizer prioritizing overseer nodes at 73138548910391304-127.0.0.1:37155_solr-n_0000000010 overseer designates are [127.0.0.1:46527_solr, 127.0.0.1:41935_solr, 127.0.0.1:42464_solr] [junit4] 2> 1667014 INFO (OverseerCollectionConfigSetProcessor-73138548910391304-127.0.0.1:37155_solr-n_0000000010) [n:127.0.0.1:37155_solr ] o.a.s.c.OverseerNodePrioritizer sorted nodes [73138548910391304-127.0.0.1:37155_solr-n_0000000010, 73138548910391306-127.0.0.1:41935_solr-n_0000000011, 73138548910391310-127.0.0.1:42464_solr-n_0000000012] [junit4] 2> 1667018 INFO (OverseerStateUpdate-73138548910391304-127.0.0.1:37155_solr-n_0000000010) [n:127.0.0.1:37155_solr ] o.a.s.c.Overseer Quit command received { [junit4] 2> "operation":"quit", [junit4] 2> "id":"73138548910391304-127.0.0.1:37155_solr-n_0000000010"} 127.0.0.1:37155_solr [junit4] 2> 1667018 INFO (OverseerStateUpdate-73138548910391304-127.0.0.1:37155_solr-n_0000000010) [n:127.0.0.1:37155_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:37155_solr [junit4] 2> 1667019 WARN (OverseerExitThread) [ ] o.a.s.c.Overseer I (id=73138548910391304-127.0.0.1:37155_solr-n_0000000010) am exiting, but I'm still the leader [junit4] 2> 1667020 INFO (zkCallback-8167-thread-1) [ ] o.a.s.c.Overseer Overseer (id=73138548910391306-127.0.0.1:41935_solr-n_0000000008) closing [junit4] 2> 1667021 INFO (OverseerExitThread) [ ] o.a.s.c.Overseer Overseer (id=73138548910391304-127.0.0.1:37155_solr-n_0000000010) closing [junit4] 2> 1667021 INFO (OverseerAutoScalingTriggerThread-73138548910391304-127.0.0.1:37155_solr-n_0000000010) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting. [junit4] 2> 1667021 INFO (zkCallback-8167-thread-1) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:41935_solr [junit4] 2> 1667033 INFO (zkCallback-8167-thread-1) [n:127.0.0.1:41935_solr ] o.a.s.c.Overseer Overseer (id=73138548910391306-127.0.0.1:41935_solr-n_0000000011) starting [junit4] 2> 1667034 INFO (OverseerExitThread) [ ] o.a.s.c.Overseer Overseer (id=73138548910391304-127.0.0.1:37155_solr-n_0000000010) closing [junit4] 2> 1667034 INFO (OverseerExitThread) [ ] o.a.s.c.Overseer Overseer (id=73138548910391304-127.0.0.1:37155_solr-n_0000000010) closing [junit4] 2> 1667035 INFO (OverseerExitThread) [ ] o.a.s.c.Overseer Overseer (id=73138548910391304-127.0.0.1:37155_solr-n_0000000010) closing [junit4] 2> 1667039 INFO (OverseerStateUpdate-73138548910391306-127.0.0.1:41935_solr-n_0000000011) [n:127.0.0.1:41935_solr ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:41935_solr [junit4] 2> 1681993 INFO (TEST-OverseerRolesTest.testOverseerRole-seed#[62094B24FCF12C8B]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testOverseerRole [junit4] 2> NOTE: reproduce with: ant test -Dtestcase=OverseerRolesTest -Dtests.method=testOverseerRole -Dtests.seed=62094B24FCF12C8B -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=ti -Dtests.timezone=America/Buenos_Aires -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1 [junit4] FAILURE 15.7s J2 | OverseerRolesTest.testOverseerRole <<< [junit4] > Throwable #1: java.lang.AssertionError: Timed out waiting for overseer state change [junit4] > at __randomizedtesting.SeedInfo.seed([62094B24FCF12C8B:83C2B6B0C7421A5A]:0) [junit4] > at org.apache.solr.cloud.OverseerRolesTest.waitForNewOverseer(OverseerRolesTest.java:63) [junit4] > at org.apache.solr.cloud.OverseerRolesTest.testOverseerRole(OverseerRolesTest.java:145) [junit4] > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [junit4] > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) [junit4] > at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [junit4] > at java.base/java.lang.reflect.Method.invoke(Method.java:566) [junit4] > at java.base/java.lang.Thread.run(Thread.java:834) [junit4] 2> 1681997 INFO (jetty-closer-8226-thread-1) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=92073358 [junit4] 2> 1681997 INFO (jetty-closer-8226-thread-1) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:37155_solr [junit4] 2> 1681997 INFO (jetty-closer-8226-thread-2) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1950581156 [junit4] 2> 1681997 INFO (jetty-closer-8226-thread-2) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:41935_solr [junit4] 2> 1681998 INFO (jetty-closer-8226-thread-2) [ ] o.a.s.c.ZkController Publish this node as DOWN... [junit4] 2> 1681998 INFO (jetty-closer-8226-thread-1) [ ] o.a.s.c.ZkController Publish this node as DOWN... [junit4] 2> 1681998 INFO (jetty-closer-8226-thread-2) [ ] o.a.s.c.ZkController Publish node=127.0.0.1:41935_solr as DOWN [junit4] 2> 1681998 INFO (jetty-closer-8226-thread-1) [ ] o.a.s.c.ZkController Publish node=127.0.0.1:37155_solr as DOWN [junit4] 2> 1681999 INFO (jetty-closer-8226-thread-3) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=170740119 [junit4] 2> 1681999 INFO (jetty-closer-8226-thread-3) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:42464_solr [junit4] 2> 1682000 INFO (jetty-closer-8226-thread-3) [ ] o.a.s.c.ZkController Publish this node as DOWN... [junit4] 2> 1682000 INFO (jetty-closer-8226-thread-3) [ ] o.a.s.c.ZkController Publish node=127.0.0.1:42464_solr as DOWN [junit4] 2> 1682002 INFO (jetty-closer-8226-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null [junit4] 2> 1682003 INFO (jetty-closer-8226-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@174438a9: rootName = solr_41935, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@70986466 [junit4] 2> 1682003 INFO (jetty-closer-8226-thread-1) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores [junit4] 2> 1682007 INFO (jetty-closer-8226-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null [junit4] 2> 1682007 INFO (jetty-closer-8226-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@3148695b: rootName = solr_37155, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@6a85a5b0 [junit4] 2> 1682012 INFO (jetty-closer-8226-thread-3) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null [junit4] 2> 1682012 INFO (jetty-closer-8226-thread-3) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@229b4e64: rootName = solr_42464, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@6aaecc74 [junit4] 2> 1682014 INFO (jetty-closer-8226-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null [junit4] 2> 1682014 INFO (jetty-closer-8226-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1c66f67c: rootName = solr_41935, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@1b12394d [junit4] 2> 1682019 INFO (jetty-closer-8226-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null [junit4] 2> 1682019 INFO (jetty-closer-8226-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@3f1981dc: rootName = solr_37155, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@1b12394d [junit4] 2> 1682024 INFO (jetty-closer-8226-thread-3) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null [junit4] 2> 1682024 INFO (jetty-closer-8226-thread-3) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@f0aa63c: rootName = solr_42464, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@1b12394d [junit4] 2> 1682042 INFO (jetty-closer-8226-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null [junit4] 2> 1682042 INFO (jetty-closer-8226-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@28cef38: rootName = solr_41935, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@615b52e5 [junit4] 2> 1682042 INFO (jetty-closer-8226-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null [junit4] 2> 1682043 INFO (closeThreadPool-8236-thread-1) [ ] o.a.s.c.Overseer Overseer (id=73138548910391306-127.0.0.1:41935_solr-n_0000000011) closing [junit4] 2> 1682043 INFO (OverseerStateUpdate-73138548910391306-127.0.0.1:41935_solr-n_0000000011) [n:127.0.0.1:41935_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:41935_solr [junit4] 2> 1682044 INFO (OverseerAutoScalingTriggerThread-73138548910391306-127.0.0.1:41935_solr-n_0000000011) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting. [junit4] 2> 1682045 INFO (jetty-closer-8226-thread-3) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null [junit4] 2> 1682045 INFO (jetty-closer-8226-thread-3) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@60c6c9ee: rootName = solr_42464, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@615b52e5 [junit4] 2> 1682045 INFO (jetty-closer-8226-thread-3) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null [junit4] 2> 1682046 INFO (jetty-closer-8226-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null [junit4] 2> 1682046 INFO (jetty-closer-8226-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@22326f00: rootName = solr_37155, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@615b52e5 [junit4] 2> 1682047 INFO (jetty-closer-8226-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null [junit4] 2> 1682048 INFO (closeThreadPool-8237-thread-1) [ ] o.a.s.c.Overseer Overseer (id=73138548910391310-127.0.0.1:42464_solr-n_0000000009) closing [junit4] 2> 1682051 INFO (closeThreadPool-8237-thread-2) [ ] o.a.s.c.Overseer Overseer (id=73138548910391310-127.0.0.1:42464_solr-n_0000000009) closing [junit4] 2> 1682051 INFO (closeThreadPool-8236-thread-2) [ ] o.a.s.c.Overseer Overseer (id=73138548910391306-127.0.0.1:41935_solr-n_0000000011) closing [junit4] 2> 1682057 INFO (closeThreadPool-8238-thread-2) [ ] o.a.s.c.Overseer Overseer (id=73138548910391304-127.0.0.1:37155_solr-n_0000000010) closing [junit4] 2> 1682060 INFO (closeThreadPool-8238-thread-1) [ ] o.a.s.c.Overseer Overseer (id=73138548910391304-127.0.0.1:37155_solr-n_0000000010) closing [junit4] 2> 1682061 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x103d71f6b310010, likely client has closed socket [junit4] 2> 1682062 INFO (jetty-closer-8226-thread-2) [ ] o.a.s.c.Overseer Overseer (id=73138548910391306-127.0.0.1:41935_solr-n_0000000011) closing [junit4] 2> 1682063 INFO (jetty-closer-8226-thread-1) [ ] o.a.s.c.Overseer Overseer (id=73138548910391304-127.0.0.1:37155_solr-n_0000000010) closing [junit4] 2> 1682063 INFO (jetty-closer-8226-thread-3) [ ] o.a.s.c.Overseer Overseer (id=73138548910391310-127.0.0.1:42464_solr-n_0000000009) closing [junit4] 2> 1682064 INFO (jetty-closer-8226-thread-2) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@6269e125{HTTP/1.1,[http/1.1, h2c]}{127.0.0.1:0} [junit4] 2> 1682065 INFO (jetty-closer-8226-thread-2) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@31b15ede{/solr,null,UNAVAILABLE} [junit4] 2> 1682065 INFO (jetty-closer-8226-thread-2) [ ] o.e.j.s.session node0 Stopped scavenging [junit4] 2> 1682066 INFO (jetty-closer-8226-thread-1) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@b29eeca{HTTP/1.1,[http/1.1, h2c]}{127.0.0.1:0} [junit4] 2> 1682066 INFO (jetty-closer-8226-thread-3) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@40c0ab01{HTTP/1.1,[http/1.1, h2c]}{127.0.0.1:0} [junit4] 2> 1682067 INFO (jetty-closer-8226-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@226122b5{/solr,null,UNAVAILABLE} [junit4] 2> 1682067 INFO (jetty-closer-8226-thread-3) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@5d4cf1d7{/solr,null,UNAVAILABLE} [junit4] 2> 1682067 INFO (jetty-closer-8226-thread-3) [ ] o.e.j.s.session node0 Stopped scavenging [junit4] 2> 1682067 INFO (jetty-closer-8226-thread-1) [ ] o.e.j.s.session node0 Stopped scavenging [junit4] 2> 1682069 INFO (SUITE-OverseerRolesTest-seed#[62094B24FCF12C8B]-worker) [ ] o.a.s.c.ZkTestServer Shutting down ZkTestServer. [junit4] 2> 1682070 WARN (ZkTestServer Run Thread) [ ] o.a.s.c.ZkTestServer Watch limit violations: [junit4] 2> Maximum concurrent create/delete watches above limit: [junit4] 2> [junit4] 2> 9 /solr/aliases.json [junit4] 2> 9 /solr/clusterprops.json [junit4] 2> 4 /solr/security.json [junit4] 2> [junit4] 2> Maximum concurrent data watches above limit: [junit4] 2> [junit4] 2> 12 /solr/autoscaling.json [junit4] 2> 9 /solr/clusterstate.json [junit4] 2> [junit4] 2> Maximum concurrent children watches above limit: [junit4] 2> [junit4] 2> 12 /solr/overseer/queue-work [junit4] 2> 11 /solr/autoscaling/events/.scheduled_maintenance [junit4] 2> 10 /solr/autoscaling/events/.auto_add_replicas [junit4] 2> 9 /solr/live_nodes [junit4] 2> 9 /solr/collections [junit4] 2> 4 /solr/overseer/collection-queue-work [junit4] 2> 2 /solr/overseer/queue [junit4] 2> [junit4] 2> 1682072 INFO (SUITE-OverseerRolesTest-seed#[62094B24FCF12C8B]-worker) [ ] o.a.s.c.ZkTestServer parse host and port list: 127.0.0.1:39507 [junit4] 2> 1682072 INFO (SUITE-OverseerRolesTest-seed#[62094B24FCF12C8B]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1 39507 [junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.OverseerRolesTest_62094B24FCF12C8B-001 [junit4] 2> Apr 15, 2019 8:15:27 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks [junit4] 2> WARNING: Will linger awaiting termination of 9 leaked thread(s). [junit4] 2> NOTE: test params are: codec=Asserting(Lucene80): {}, docValues:{}, maxPointsInLeafNode=979, maxMBSortInHeap=6.505943696546584, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@140f7108), locale=ti, timezone=America/Buenos_Aires [junit4] 2> NOTE: Linux 4.4.0-137-generic amd64/Oracle Corporation 11.0.1 (64-bit)/cpus=4,threads=1,free=233677072,total=536870912 [junit4] 2> NOTE: All tests run in this JVM: [HLLUtilTest, DistributedFacetPivotSmallTest, UpdateParamsTest, MergeStrategyTest, TestPayloadScoreQParserPlugin, TestLegacyField, TestRecovery, MoveReplicaHDFSTest, CopyFieldTest, TestHttpShardHandlerFactory, TestAuthenticationFramework, SolrIndexSplitterTest, TestDynamicFieldResource, TestDelegationWithHadoopAuth, TestXmlQParser, TestWithCollection, NodeMarkersRegistrationTest, PeerSyncWithLeaderTest, RemoteQueryErrorTest, TestRangeQuery, OverseerTaskQueueTest, TestIBSimilarityFactory, MBeansHandlerTest, HdfsAutoAddReplicasIntegrationTest, OverseerModifyCollectionTest, DocumentBuilderTest, TestSolrIndexConfig, SortSpecParsingTest, BinaryUpdateRequestHandlerTest, TestSortableTextField, CoreAdminOperationTest, HdfsSyncSliceTest, HDFSCollectionsAPITest, TestPullReplica, DistributedQueryComponentCustomSortTest, NotRequiredUniqueKeyTest, TestStressInPlaceUpdates, TestConfigSetsAPIExclusivity, TestSuggestSpellingConverter, TestStreamBody, TestSimNodeAddedTrigger, TestLeaderElectionWithEmptyReplica, SuggesterTSTTest, RAMDirectoryFactoryTest, ReplicationFactorTest, TestPointFields, TestSimExtremeIndexing, ExplicitHLLTest, NodeAddedTriggerTest, TestUpdate, CollectionStateFormat2Test, AssignTest, DeleteStatusTest, TestSizeLimitedDistributedMap, TestSolr4Spatial2, AlternateDirectoryTest, TestPKIAuthenticationPlugin, TestSurroundQueryParser, TestMacroExpander, DateMathParserTest, DistributedDebugComponentTest, SearchHandlerTest, TestManagedSchemaAPI, UUIDFieldTest, TestSerializedLuceneMatchVersion, TestStressReorder, TestTrie, AtomicUpdateProcessorFactoryTest, ActionThrottleTest, TestManagedResource, TestSafeXMLParsing, TestCollationFieldDocValues, TestPerFieldSimilarityWithDefaultOverride, ConvertedLegacyTest, SuggestComponentTest, ParsingFieldUpdateProcessorsTest, SchemaVersionSpecificBehaviorTest, TestApiFramework, ClassificationUpdateProcessorTest, TaggingAttributeTest, SolrGangliaReporterTest, DateRangeFieldTest, HighlighterConfigTest, PingRequestHandlerTest, CustomCollectionTest, TimeRoutedAliasUpdateProcessorTest, ZkCLITest, TestReqParamsAPI, TestMiniSolrCloudClusterSSL, TriLevelCompositeIdRoutingTest, RecoveryAfterSoftCommitTest, GraphQueryTest, TriggerCooldownIntegrationTest, AuditLoggerPluginTest, TestRequestStatusCollectionAPI, TestSolrJ, TestDefaultStatsCache, DistributedUpdateProcessorTest, SpellCheckCollatorWithCollapseTest, ImplicitSnitchTest, SimpleFacetsTest, SOLR749Test, TestCharFilters, DistributedFacetPivotLargeTest, DistributedIntervalFacetingTest, TestLRUCache, TestExactStatsCache, LegacyCloudClusterPropTest, TestCustomDocTransformer, TestDocBasedVersionConstraints, TestScoreJoinQPNoScore, SolrMetricManagerTest, CustomHighlightComponentTest, CoreSorterTest, PrimitiveFieldTypeTest, TestConfigSetProperties, TestHdfsCloudBackupRestore, TestCloudPseudoReturnFields, TestRawResponseWriter, DocValuesNotIndexedTest, RangeFacetCloudTest, TestFileDictionaryLookup, FileBasedSpellCheckerTest, HttpSolrCallGetCoreTest, TestBackupRepositoryFactory, TestSolrCloudWithKerberosAlt, JWTAuthPluginIntegrationTest, SolrTestCaseJ4Test, HdfsChaosMonkeyNothingIsSafeTest, TestCoreDiscovery, SimpleCollectionCreateDeleteTest, TestSSLRandomization, UtilsToolTest, AutoScalingHandlerTest, FacetPivotSmallTest, DocValuesTest, TestEmbeddedSolrServerSchemaAPI, CursorPagingTest, DisMaxRequestHandlerTest, OutputWriterTest, SampleTest, SolrInfoBeanTest, TestDistributedGrouping, TestDistributedMissingSort, TestDistributedSearch, TestRandomFaceting, AsyncCallRequestStatusResponseTest, BasicDistributedZk2Test, ChaosMonkeySafeLeaderTest, CleanupOldIndexTest, ClusterStateTest, ClusterStateUpdateTest, CreateRoutedAliasTest, DeleteNodeTest, DistributedVersionInfoTest, LeaderElectionContextKeyTest, LeaderElectionIntegrationTest, LeaderTragicEventTest, LeaderVoteWaitTimeoutTest, MissingSegmentRecoveryTest, MultiThreadedOCPTest, OverriddenZkACLAndCredentialsProvidersTest, OverseerRolesTest] [junit4] Completed [495/854 (1!)] on J2 in 17.78s, 1 test, 1 failure <<< FAILURES! [...truncated 44527 lines...]
--------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org