Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/6408/ Java: 64bit/jdk-11.0.6 -XX:-UseCompressedOops -XX:+UseSerialGC
2 tests failed. FAILED: junit.framework.TestSuite.org.apache.solr.cloud.OverseerTest Error Message: 1 thread leaked from SUITE scope at org.apache.solr.cloud.OverseerTest: 1) Thread[id=10342, name=SessionTracker, state=TIMED_WAITING, group=TGRP-OverseerTest] at [email protected]/java.lang.Thread.sleep(Native Method) at app//org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:157) Stack Trace: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.OverseerTest: 1) Thread[id=10342, name=SessionTracker, state=TIMED_WAITING, group=TGRP-OverseerTest] at [email protected]/java.lang.Thread.sleep(Native Method) at app//org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:157) at __randomizedtesting.SeedInfo.seed([FE8B3A34B4F7525A]:0) FAILED: junit.framework.TestSuite.org.apache.solr.cloud.PackageManagerCLITest Error Message: 1 thread leaked from SUITE scope at org.apache.solr.cloud.PackageManagerCLITest: 1) Thread[id=16545, name=SessionTracker, state=TIMED_WAITING, group=TGRP-PackageManagerCLITest] at [email protected]/java.lang.Thread.sleep(Native Method) at app//org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:157) Stack Trace: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.PackageManagerCLITest: 1) Thread[id=16545, name=SessionTracker, state=TIMED_WAITING, group=TGRP-PackageManagerCLITest] at [email protected]/java.lang.Thread.sleep(Native Method) at app//org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:157) at __randomizedtesting.SeedInfo.seed([FE8B3A34B4F7525A]:0) Build Log: [...truncated 13651 lines...] [junit4] Suite: org.apache.solr.cloud.OverseerTest [junit4] 2> 1571517 INFO (SUITE-OverseerTest-seed#[FE8B3A34B4F7525A]-worker) [ ] o.a.s.SolrTestCase Setting 'solr.default.confdir' system property to test-framework derived value of '/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/server/solr/configsets/_default/conf' [junit4] 2> 1571518 INFO (SUITE-OverseerTest-seed#[FE8B3A34B4F7525A]-worker) [ ] o.a.s.SolrTestCaseJ4 Created dataDir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.OverseerTest_FE8B3A34B4F7525A-001/data-dir-92-001 [junit4] 2> 1571519 INFO (SUITE-OverseerTest-seed#[FE8B3A34B4F7525A]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true [junit4] 2> 1571520 INFO (SUITE-OverseerTest-seed#[FE8B3A34B4F7525A]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason="", value=0.0/0.0, ssl=0.0/0.0, clientAuth=0.0/0.0) w/ MAC_OS_X supressed clientAuth [junit4] 2> 1571520 INFO (SUITE-OverseerTest-seed#[FE8B3A34B4F7525A]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom [junit4] 2> 1571520 INFO (SUITE-OverseerTest-seed#[FE8B3A34B4F7525A]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER [junit4] 2> 1571521 INFO (ZkTestServer Run Thread) [ ] o.a.s.c.ZkTestServer client port: 0.0.0.0/0.0.0.0:0 [junit4] 2> 1571521 INFO (ZkTestServer Run Thread) [ ] o.a.s.c.ZkTestServer Starting server [junit4] 2> 1571522 WARN (ZkTestServer Run Thread) [ ] o.a.z.s.ServerCnxnFactory maxCnxns is not configured, using default value 0. [junit4] 2> 1571622 INFO (SUITE-OverseerTest-seed#[FE8B3A34B4F7525A]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port: 56332 [junit4] 2> 1571622 INFO (SUITE-OverseerTest-seed#[FE8B3A34B4F7525A]-worker) [ ] o.a.s.c.ZkTestServer waitForServerUp: 127.0.0.1:56332 [junit4] 2> 1571622 INFO (SUITE-OverseerTest-seed#[FE8B3A34B4F7525A]-worker) [ ] o.a.s.c.ZkTestServer parse host and port list: 127.0.0.1:56332 [junit4] 2> 1571622 INFO (SUITE-OverseerTest-seed#[FE8B3A34B4F7525A]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1 56332 [junit4] 2> 1571626 INFO (SUITE-OverseerTest-seed#[FE8B3A34B4F7525A]-worker) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 1571633 INFO (zkConnectionManagerCallback-7258-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 1571633 INFO (SUITE-OverseerTest-seed#[FE8B3A34B4F7525A]-worker) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 1571640 INFO (SUITE-OverseerTest-seed#[FE8B3A34B4F7525A]-worker) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 1571644 INFO (zkConnectionManagerCallback-7260-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 1571644 INFO (SUITE-OverseerTest-seed#[FE8B3A34B4F7525A]-worker) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 1571645 INFO (SUITE-OverseerTest-seed#[FE8B3A34B4F7525A]-worker) [ ] o.a.s.SolrTestCaseJ4 ####initCore [junit4] 2> 1571645 INFO (SUITE-OverseerTest-seed#[FE8B3A34B4F7525A]-worker) [ ] o.a.s.SolrTestCaseJ4 ####initCore end [junit4] 2> 1571658 INFO (TEST-OverseerTest.testExternalClusterStateChangeBehavior-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testExternalClusterStateChangeBehavior [junit4] 2> 1572152 INFO (TEST-OverseerTest.testExternalClusterStateChangeBehavior-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 1572159 INFO (zkConnectionManagerCallback-7264-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 1572159 INFO (TEST-OverseerTest.testExternalClusterStateChangeBehavior-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 1572164 WARN (TEST-OverseerTest.testExternalClusterStateChangeBehavior-seed#[FE8B3A34B4F7525A]) [ ] o.e.j.u.s.S.config Trusting all certificates configured for Client@3ab5dfdb[provider=null,keyStore=null,trustStore=null] [junit4] 2> 1572165 WARN (TEST-OverseerTest.testExternalClusterStateChangeBehavior-seed#[FE8B3A34B4F7525A]) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@3ab5dfdb[provider=null,keyStore=null,trustStore=null] [junit4] 2> 1572165 INFO (TEST-OverseerTest.testExternalClusterStateChangeBehavior-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true] [junit4] 2> 1572168 WARN (TEST-OverseerTest.testExternalClusterStateChangeBehavior-seed#[FE8B3A34B4F7525A]) [ ] o.e.j.u.s.S.config Trusting all certificates configured for Client@79fedee4[provider=null,keyStore=null,trustStore=null] [junit4] 2> 1572168 WARN (TEST-OverseerTest.testExternalClusterStateChangeBehavior-seed#[FE8B3A34B4F7525A]) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@79fedee4[provider=null,keyStore=null,trustStore=null] [junit4] 2> 1572176 INFO (TEST-OverseerTest.testExternalClusterStateChangeBehavior-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 1572179 INFO (zkConnectionManagerCallback-7272-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 1572179 INFO (TEST-OverseerTest.testExternalClusterStateChangeBehavior-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 1572204 INFO (TEST-OverseerTest.testExternalClusterStateChangeBehavior-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 1572208 INFO (zkConnectionManagerCallback-7277-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 1572208 INFO (TEST-OverseerTest.testExternalClusterStateChangeBehavior-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 1572235 INFO (TEST-OverseerTest.testExternalClusterStateChangeBehavior-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:56332/solr ready [junit4] 2> 1572246 INFO (TEST-OverseerTest.testExternalClusterStateChangeBehavior-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:56332_solr [junit4] 2> 1572249 INFO (TEST-OverseerTest.testExternalClusterStateChangeBehavior-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.Overseer Overseer (id=72187932591652866-127.0.0.1:56332_solr-n_0000000000) starting [junit4] 2> 1572290 WARN (TEST-OverseerTest.testExternalClusterStateChangeBehavior-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.Overseer Unable to check back-compat of .system collection - can't obtain ClusterState. [junit4] 2> 1572291 INFO (OverseerStateUpdate-72187932591652866-127.0.0.1:56332_solr-n_0000000000) [ ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:56332_solr [junit4] 2> 1572326 INFO (TEST-OverseerTest.testExternalClusterStateChangeBehavior-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.AbstractDistribZkTestBase verifyReplicaStatus (c1) shard=shard1 coreNodeName=core_node1 [junit4] 2> 1572426 INFO (zkCallback-7259-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/c1/state.json] for collection [c1] has occurred - updating... (live nodes size: [0]) [junit4] 2> 1572427 INFO (zkCallback-7259-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/c1/state.json] for collection [c1] has occurred - updating... (live nodes size: [0]) [junit4] 2> 1572427 INFO (zkCallback-7259-thread-3) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/c1/state.json] for collection [c1] has occurred - updating... (live nodes size: [0]) [junit4] 2> 1572537 INFO (zkCallback-7259-thread-3) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeCreated path:/collections/test/state.json] for collection [test] has occurred - updating... (live nodes size: [0]) [junit4] 2> 1572562 INFO (TEST-OverseerTest.testExternalClusterStateChangeBehavior-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.AbstractDistribZkTestBase verifyReplicaStatus (test) shard=x coreNodeName=core_node1 [junit4] 2> 1572565 INFO (OverseerStateUpdate-72187932591652866-127.0.0.1:56332_solr-n_0000000000) [ ] o.a.s.c.o.SliceMutator createReplica() { [junit4] 2> "operation":"addreplica", [junit4] 2> "collection":"test", [junit4] 2> "shard":"x", [junit4] 2> "base_url":"http://127.0.0.1/solr";, [junit4] 2> "core_node_name":"core_node1", [junit4] 2> "node_name":"node1", [junit4] 2> "core":"core1", [junit4] 2> "state":"down"} [junit4] 2> 1572668 ERROR (OverseerStateUpdate-72187932591652866-127.0.0.1:56332_solr-n_0000000000) [ ] o.a.s.c.Overseer Exception in Overseer main queue loop [junit4] 2> => org.apache.zookeeper.KeeperException$BadVersionException: KeeperErrorCode = BadVersion for /collections/test/state.json [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:122) [junit4] 2> org.apache.zookeeper.KeeperException$BadVersionException: KeeperErrorCode = BadVersion for /collections/test/state.json [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:122) ~[zookeeper-3.6.1.jar:3.6.1] [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) ~[zookeeper-3.6.1.jar:3.6.1] [junit4] 2> at org.apache.zookeeper.ZooKeeper.setData(ZooKeeper.java:2551) ~[zookeeper-3.6.1.jar:3.6.1] [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$setData$6(SolrZkClient.java:351) ~[java/:?] [junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71) ~[java/:?] [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.setData(SolrZkClient.java:351) ~[java/:?] [junit4] 2> at org.apache.solr.cloud.overseer.ZkStateWriter.writePendingUpdates(ZkStateWriter.java:185) ~[java/:?] [junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:321) [java/:?] [junit4] 2> at java.lang.Thread.run(Thread.java:834) [?:?] [junit4] 2> 1572677 INFO (zkCallback-7259-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/test/state.json] for collection [test] has occurred - updating... (live nodes size: [0]) [junit4] 2> 1572682 INFO (OverseerStateUpdate-72187932591652866-127.0.0.1:56332_solr-n_0000000000) [ ] o.a.s.c.o.SliceMutator createReplica() { [junit4] 2> "operation":"addreplica", [junit4] 2> "collection":"test", [junit4] 2> "shard":"x", [junit4] 2> "base_url":"http://127.0.0.1/solr";, [junit4] 2> "core_node_name":"core_node1", [junit4] 2> "node_name":"node1", [junit4] 2> "core":"core1", [junit4] 2> "state":"down"} [junit4] 2> 1572687 INFO (zkCallback-7259-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/test/state.json] for collection [test] has occurred - updating... (live nodes size: [0]) [junit4] 2> 1572699 INFO (TEST-OverseerTest.testExternalClusterStateChangeBehavior-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.AbstractDistribZkTestBase verifyReplicaStatus (c1) shard=shard1 coreNodeName=core_node1 [junit4] 2> 1572705 WARN (TEST-OverseerTest.testExternalClusterStateChangeBehavior-seed#[FE8B3A34B4F7525A]-SendThread(127.0.0.1:56332)) [ ] o.a.z.ClientCnxn An exception was thrown while closing send thread for session 0x100768acf320002. [junit4] 2> => EndOfStreamException: Unable to read additional data from server sessionid 0x100768acf320002, likely server has closed socket [junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:75) [junit4] 2> org.apache.zookeeper.ClientCnxn$EndOfStreamException: Unable to read additional data from server sessionid 0x100768acf320002, likely server has closed socket [junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:75) ~[zookeeper-3.6.1.jar:3.6.1] [junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:348) ~[zookeeper-3.6.1.jar:3.6.1] [junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1262) [zookeeper-3.6.1.jar:3.6.1] [junit4] 2> 1572818 WARN (TEST-OverseerTest.testExternalClusterStateChangeBehavior-seed#[FE8B3A34B4F7525A]-SendThread(127.0.0.1:56332)) [ ] o.a.z.ClientCnxn An exception was thrown while closing send thread for session 0x100768acf320003. [junit4] 2> => EndOfStreamException: Unable to read additional data from server sessionid 0x100768acf320003, likely server has closed socket [junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:75) [junit4] 2> org.apache.zookeeper.ClientCnxn$EndOfStreamException: Unable to read additional data from server sessionid 0x100768acf320003, likely server has closed socket [junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:75) ~[zookeeper-3.6.1.jar:3.6.1] [junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:348) ~[zookeeper-3.6.1.jar:3.6.1] [junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1262) [zookeeper-3.6.1.jar:3.6.1] [junit4] 2> 1572821 WARN (TEST-OverseerTest.testExternalClusterStateChangeBehavior-seed#[FE8B3A34B4F7525A]-SendThread(127.0.0.1:56332)) [ ] o.a.z.ClientCnxn An exception was thrown while closing send thread for session 0x100768acf320004. [junit4] 2> => EndOfStreamException: Unable to read additional data from server sessionid 0x100768acf320004, likely server has closed socket [junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:75) [junit4] 2> org.apache.zookeeper.ClientCnxn$EndOfStreamException: Unable to read additional data from server sessionid 0x100768acf320004, likely server has closed socket [junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:75) ~[zookeeper-3.6.1.jar:3.6.1] [junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:348) ~[zookeeper-3.6.1.jar:3.6.1] [junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1262) [zookeeper-3.6.1.jar:3.6.1] [junit4] 2> 1572929 INFO (closeThreadPool-7281-thread-1) [ ] o.a.s.c.Overseer Overseer (id=72187932591652866-127.0.0.1:56332_solr-n_0000000000) closing [junit4] 2> 1572930 INFO (OverseerStateUpdate-72187932591652866-127.0.0.1:56332_solr-n_0000000000) [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:56332_solr [junit4] 2> 1572932 WARN (OverseerExitThread) [ ] o.a.s.c.Overseer Error communicating with ZooKeeper [junit4] 2> => org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:134) [junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:134) ~[zookeeper-3.6.1.jar:3.6.1] [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) ~[zookeeper-3.6.1.jar:3.6.1] [junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:2348) ~[zookeeper-3.6.1.jar:3.6.1] [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:339) ~[java/:?] [junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71) ~[java/:?] [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:339) ~[java/:?] [junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:395) [java/:?] [junit4] 2> at java.lang.Thread.run(Thread.java:834) [?:?] [junit4] 2> 1573129 INFO (TEST-OverseerTest.testExternalClusterStateChangeBehavior-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testExternalClusterStateChangeBehavior [junit4] 2> 1573141 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testOverseerStatsReset [junit4] 2> 1573407 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 1573409 INFO (zkConnectionManagerCallback-7285-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 1573409 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 1573421 INFO (zkCallback-7259-thread-3) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 1573421 INFO (zkCallback-7284-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 1573423 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 1573425 INFO (zkConnectionManagerCallback-7292-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 1573425 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 1573427 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 1573427 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:56332/solr ready [junit4] 2> 1573434 WARN (TEST-OverseerTest.testOverseerStatsReset-seed#[FE8B3A34B4F7525A]) [ ] o.e.j.u.s.S.config Trusting all certificates configured for Client@5beab15b[provider=null,keyStore=null,trustStore=null] [junit4] 2> 1573434 WARN (TEST-OverseerTest.testOverseerStatsReset-seed#[FE8B3A34B4F7525A]) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@5beab15b[provider=null,keyStore=null,trustStore=null] [junit4] 2> 1573434 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true] [junit4] 2> 1573435 WARN (TEST-OverseerTest.testOverseerStatsReset-seed#[FE8B3A34B4F7525A]) [ ] o.e.j.u.s.S.config Trusting all certificates configured for Client@7eba30ca[provider=null,keyStore=null,trustStore=null] [junit4] 2> 1573435 WARN (TEST-OverseerTest.testOverseerStatsReset-seed#[FE8B3A34B4F7525A]) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@7eba30ca[provider=null,keyStore=null,trustStore=null] [junit4] 2> 1573441 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:56332_solr [junit4] 2> 1573443 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.Overseer Overseer (id=72187932591652865-127.0.0.1:56332_solr-n_0000000000) starting [junit4] 2> 1573459 WARN (TEST-OverseerTest.testOverseerStatsReset-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.Overseer Unable to check back-compat of .system collection - can't obtain ClusterState. [junit4] 2> 1573459 INFO (OverseerStateUpdate-72187932591652865-127.0.0.1:56332_solr-n_0000000000) [ ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:56332_solr [junit4] 2> 1573584 INFO (zkCallback-7284-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1]) [junit4] 2> 1573591 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection1/leaders/shard1 [junit4] 2> 1573596 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/collection1/leaders/shard1/leader after winning as /collections/collection1/leader_elect/shard1/election/72187932591652869-node1_core1-n_0000000000 [junit4] 2> 1573603 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.Overseer Overseer (id=72187932591652865-127.0.0.1:56332_solr-n_0000000000) closing [junit4] 2> 1573603 INFO (OverseerStateUpdate-72187932591652865-127.0.0.1:56332_solr-n_0000000000) [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:56332_solr [junit4] 2> 1573606 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.Overseer Overseer (id=72187932591652865-127.0.0.1:56332_solr-n_0000000000) closing [junit4] 2> 1573606 WARN (OverseerExitThread) [ ] o.a.s.c.Overseer I (id=72187932591652865-127.0.0.1:56332_solr-n_0000000000) am exiting, but I'm still the leader [junit4] 2> 1573608 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.Overseer Overseer (id=72187932591652865-127.0.0.1:56332_solr-n_0000000000) closing [junit4] 2> 1573609 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.Overseer Overseer (id=72187932591652865-127.0.0.1:56332_solr-n_0000000000) closing [junit4] 2> 1573610 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:56332_solr [junit4] 2> 1573616 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.Overseer Overseer (id=72187932591652865-127.0.0.1:56332_solr-n_0000000001) starting [junit4] 2> 1573634 WARN (TEST-OverseerTest.testOverseerStatsReset-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.Overseer Unable to check back-compat of .system collection - can't obtain ClusterState. [junit4] 2> 1573637 INFO (OverseerStateUpdate-72187932591652865-127.0.0.1:56332_solr-n_0000000001) [ ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:56332_solr [junit4] 2> 1573647 INFO (zkCallback-7284-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0) [junit4] 2> 1573648 INFO (zkCallback-7259-thread-3) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0) [junit4] 2> 1573649 INFO (zkCallback-7291-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0) [junit4] 2> 1573649 WARN (TEST-OverseerTest.testOverseerStatsReset-seed#[FE8B3A34B4F7525A]-SendThread(127.0.0.1:56332)) [ ] o.a.z.ClientCnxn An exception was thrown while closing send thread for session 0x100768acf320005. [junit4] 2> => EndOfStreamException: Unable to read additional data from server sessionid 0x100768acf320005, likely server has closed socket [junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:75) [junit4] 2> org.apache.zookeeper.ClientCnxn$EndOfStreamException: Unable to read additional data from server sessionid 0x100768acf320005, likely server has closed socket [junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:75) ~[zookeeper-3.6.1.jar:3.6.1] [junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:348) ~[zookeeper-3.6.1.jar:3.6.1] [junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1262) [zookeeper-3.6.1.jar:3.6.1] [junit4] 2> 1573755 WARN (TEST-OverseerTest.testOverseerStatsReset-seed#[FE8B3A34B4F7525A]-SendThread(127.0.0.1:56332)) [ ] o.a.z.ClientCnxn An exception was thrown while closing send thread for session 0x100768acf320006. [junit4] 2> => EndOfStreamException: Unable to read additional data from server sessionid 0x100768acf320006, likely server has closed socket [junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:75) [junit4] 2> org.apache.zookeeper.ClientCnxn$EndOfStreamException: Unable to read additional data from server sessionid 0x100768acf320006, likely server has closed socket [junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:75) ~[zookeeper-3.6.1.jar:3.6.1] [junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:348) ~[zookeeper-3.6.1.jar:3.6.1] [junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1262) [zookeeper-3.6.1.jar:3.6.1] [junit4] 2> 1573865 INFO (closeThreadPool-7302-thread-1) [ ] o.a.s.c.Overseer Overseer (id=72187932591652865-127.0.0.1:56332_solr-n_0000000001) closing [junit4] 2> 1573865 INFO (OverseerStateUpdate-72187932591652865-127.0.0.1:56332_solr-n_0000000001) [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:56332_solr [junit4] 2> 1573868 WARN (OverseerExitThread) [ ] o.a.s.c.Overseer I (id=72187932591652865-127.0.0.1:56332_solr-n_0000000001) am exiting, but I'm still the leader [junit4] 2> 1573981 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testOverseerStatsReset [junit4] 2> 1573990 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testDoubleAssignment [junit4] 2> 1574216 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 1574220 INFO (zkConnectionManagerCallback-7306-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 1574220 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 1574238 INFO (zkCallback-7305-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 1574240 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 1574241 INFO (zkCallback-7259-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 1574241 INFO (zkConnectionManagerCallback-7310-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 1574241 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 1574248 WARN (TEST-OverseerTest.testDoubleAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.e.j.u.s.S.config Trusting all certificates configured for Client@6b5071e5[provider=null,keyStore=null,trustStore=null] [junit4] 2> 1574248 WARN (TEST-OverseerTest.testDoubleAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@6b5071e5[provider=null,keyStore=null,trustStore=null] [junit4] 2> 1574248 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true] [junit4] 2> 1574249 WARN (TEST-OverseerTest.testDoubleAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.e.j.u.s.S.config Trusting all certificates configured for Client@6beb046d[provider=null,keyStore=null,trustStore=null] [junit4] 2> 1574249 WARN (TEST-OverseerTest.testDoubleAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@6beb046d[provider=null,keyStore=null,trustStore=null] [junit4] 2> 1574250 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 1574253 INFO (zkConnectionManagerCallback-7318-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 1574253 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 1574256 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 1574259 INFO (zkConnectionManagerCallback-7323-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 1574259 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 1574262 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 1574264 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:56332/solr ready [junit4] 2> 1574270 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:56332_solr [junit4] 2> 1574271 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.Overseer Overseer (id=72187932591652872-127.0.0.1:56332_solr-n_0000000000) starting [junit4] 2> 1574291 WARN (TEST-OverseerTest.testDoubleAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.Overseer Unable to check back-compat of .system collection - can't obtain ClusterState. [junit4] 2> 1574293 INFO (OverseerStateUpdate-72187932591652872-127.0.0.1:56332_solr-n_0000000000) [ ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:56332_solr [junit4] 2> 1574298 INFO (OverseerStateUpdate-72187932591652872-127.0.0.1:56332_solr-n_0000000000) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 1574302 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 1574306 INFO (zkConnectionManagerCallback-7330-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 1574306 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 1574309 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 1574311 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:56332/solr ready [junit4] 2> 1574419 INFO (zkCallback-7305-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1]) [junit4] 2> 1574428 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection1/leaders/shard1 [junit4] 2> 1574435 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/collection1/leaders/shard1/leader after winning as /collections/collection1/leader_elect/shard1/election/72187932591652871-node1_core1-n_0000000000 [junit4] 2> 1574445 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.AbstractDistribZkTestBase verifyReplicaStatus (collection1) shard=shard1 coreNodeName=core_node1 [junit4] 2> 1574451 INFO (zkCallback-7309-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0) [junit4] 2> 1574451 INFO (zkCallback-7329-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0) [junit4] 2> 1574451 INFO (zkCallback-7322-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0) [junit4] 2> 1574452 INFO (zkCallback-7305-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0) [junit4] 2> 1574453 INFO (zkCallback-7259-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0) [junit4] 2> 1574453 WARN (TEST-OverseerTest.testDoubleAssignment-seed#[FE8B3A34B4F7525A]-SendThread(127.0.0.1:56332)) [ ] o.a.z.ClientCnxn An exception was thrown while closing send thread for session 0x100768acf320007. [junit4] 2> => EndOfStreamException: Unable to read additional data from server sessionid 0x100768acf320007, likely server has closed socket [junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:75) [junit4] 2> org.apache.zookeeper.ClientCnxn$EndOfStreamException: Unable to read additional data from server sessionid 0x100768acf320007, likely server has closed socket [junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:75) ~[zookeeper-3.6.1.jar:3.6.1] [junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:348) ~[zookeeper-3.6.1.jar:3.6.1] [junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1262) [zookeeper-3.6.1.jar:3.6.1] [junit4] 2> 1574555 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 1574558 INFO (zkConnectionManagerCallback-7332-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 1574558 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 1574571 INFO (zkCallback-7329-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 1574571 INFO (zkCallback-7322-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 1574572 INFO (zkCallback-7331-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 1574572 INFO (zkCallback-7309-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 1574572 INFO (zkCallback-7259-thread-3) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 1574580 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection1/leaders/shard1 [junit4] 2> 1574583 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/collection1/leaders/shard1/leader after winning as /collections/collection1/leader_elect/shard1/election/72187932591652876-node1_core1-n_0000000001 [junit4] 2> 1574689 WARN (OverseerStateUpdate-72187932591652872-127.0.0.1:56332_solr-n_0000000000) [ ] o.a.s.c.Overseer Solr cannot talk to ZK, exiting Overseer main queue loop [junit4] 2> => org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/state.json [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:134) [junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/state.json [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:134) ~[zookeeper-3.6.1.jar:3.6.1] [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) ~[zookeeper-3.6.1.jar:3.6.1] [junit4] 2> at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:2246) ~[zookeeper-3.6.1.jar:3.6.1] [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$exists$3(SolrZkClient.java:315) ~[java/:?] [junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71) ~[java/:?] [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:315) ~[java/:?] [junit4] 2> at org.apache.solr.cloud.overseer.ZkStateWriter.writePendingUpdates(ZkStateWriter.java:181) ~[java/:?] [junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:321) [java/:?] [junit4] 2> at java.lang.Thread.run(Thread.java:834) [?:?] [junit4] 2> 1574690 INFO (OverseerStateUpdate-72187932591652872-127.0.0.1:56332_solr-n_0000000000) [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:56332_solr [junit4] 2> 1574691 WARN (OverseerExitThread) [ ] o.a.s.c.Overseer Error communicating with ZooKeeper [junit4] 2> => org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:134) [junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:134) ~[zookeeper-3.6.1.jar:3.6.1] [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) ~[zookeeper-3.6.1.jar:3.6.1] [junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:2348) ~[zookeeper-3.6.1.jar:3.6.1] [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:339) ~[java/:?] [junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71) ~[java/:?] [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:339) ~[java/:?] [junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:395) [java/:?] [junit4] 2> at java.lang.Thread.run(Thread.java:834) [?:?] [junit4] 2> 1574701 INFO (zkCallback-7329-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0) [junit4] 2> 1574701 INFO (zkCallback-7259-thread-3) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0) [junit4] 2> 1574701 INFO (zkCallback-7331-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0) [junit4] 2> 1574701 INFO (zkCallback-7322-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0) [junit4] 2> 1574703 WARN (TEST-OverseerTest.testDoubleAssignment-seed#[FE8B3A34B4F7525A]-SendThread(127.0.0.1:56332)) [ ] o.a.z.ClientCnxn An exception was thrown while closing send thread for session 0x100768acf32000c. [junit4] 2> => EndOfStreamException: Unable to read additional data from server sessionid 0x100768acf32000c, likely server has closed socket [junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:75) [junit4] 2> org.apache.zookeeper.ClientCnxn$EndOfStreamException: Unable to read additional data from server sessionid 0x100768acf32000c, likely server has closed socket [junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:75) ~[zookeeper-3.6.1.jar:3.6.1] [junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:348) ~[zookeeper-3.6.1.jar:3.6.1] [junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1262) [zookeeper-3.6.1.jar:3.6.1] [junit4] 2> 1574811 WARN (TEST-OverseerTest.testDoubleAssignment-seed#[FE8B3A34B4F7525A]-SendThread(127.0.0.1:56332)) [ ] o.a.z.ClientCnxn An exception was thrown while closing send thread for session 0x100768acf320009. [junit4] 2> => EndOfStreamException: Unable to read additional data from server sessionid 0x100768acf320009, likely server has closed socket [junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:75) [junit4] 2> org.apache.zookeeper.ClientCnxn$EndOfStreamException: Unable to read additional data from server sessionid 0x100768acf320009, likely server has closed socket [junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:75) ~[zookeeper-3.6.1.jar:3.6.1] [junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:348) ~[zookeeper-3.6.1.jar:3.6.1] [junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1262) [zookeeper-3.6.1.jar:3.6.1] [junit4] 2> 1574812 WARN (TEST-OverseerTest.testDoubleAssignment-seed#[FE8B3A34B4F7525A]-SendThread(127.0.0.1:56332)) [ ] o.a.z.ClientCnxn An exception was thrown while closing send thread for session 0x100768acf32000b. [junit4] 2> => EndOfStreamException: Unable to read additional data from server sessionid 0x100768acf32000b, likely server has closed socket [junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:75) [junit4] 2> org.apache.zookeeper.ClientCnxn$EndOfStreamException: Unable to read additional data from server sessionid 0x100768acf32000b, likely server has closed socket [junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:75) ~[zookeeper-3.6.1.jar:3.6.1] [junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:348) ~[zookeeper-3.6.1.jar:3.6.1] [junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1262) [zookeeper-3.6.1.jar:3.6.1] [junit4] 2> 1575130 INFO (closeThreadPool-7336-thread-1) [ ] o.a.s.c.Overseer Overseer (id=72187932591652872-127.0.0.1:56332_solr-n_0000000000) closing [junit4] 2> 1575241 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testDoubleAssignment [junit4] 2> 1575247 INFO (TEST-OverseerTest.testStateChange-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testStateChange [junit4] 2> 1575436 INFO (TEST-OverseerTest.testStateChange-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 1575439 INFO (zkConnectionManagerCallback-7340-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 1575439 INFO (TEST-OverseerTest.testStateChange-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 1575441 WARN (TEST-OverseerTest.testStateChange-seed#[FE8B3A34B4F7525A]) [ ] o.e.j.u.s.S.config Trusting all certificates configured for Client@51c32c08[provider=null,keyStore=null,trustStore=null] [junit4] 2> 1575441 WARN (TEST-OverseerTest.testStateChange-seed#[FE8B3A34B4F7525A]) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@51c32c08[provider=null,keyStore=null,trustStore=null] [junit4] 2> 1575442 INFO (TEST-OverseerTest.testStateChange-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true] [junit4] 2> 1575443 WARN (TEST-OverseerTest.testStateChange-seed#[FE8B3A34B4F7525A]) [ ] o.e.j.u.s.S.config Trusting all certificates configured for Client@7a53b0bd[provider=null,keyStore=null,trustStore=null] [junit4] 2> 1575443 WARN (TEST-OverseerTest.testStateChange-seed#[FE8B3A34B4F7525A]) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@7a53b0bd[provider=null,keyStore=null,trustStore=null] [junit4] 2> 1575444 INFO (TEST-OverseerTest.testStateChange-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 1575447 INFO (zkConnectionManagerCallback-7348-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 1575448 INFO (TEST-OverseerTest.testStateChange-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 1575451 INFO (TEST-OverseerTest.testStateChange-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 1575454 INFO (zkConnectionManagerCallback-7353-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 1575454 INFO (TEST-OverseerTest.testStateChange-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 1575460 INFO (TEST-OverseerTest.testStateChange-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:56332/solr ready [junit4] 2> 1575469 INFO (TEST-OverseerTest.testStateChange-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:56332_solr [junit4] 2> 1575471 INFO (TEST-OverseerTest.testStateChange-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.Overseer Overseer (id=72187932591652877-127.0.0.1:56332_solr-n_0000000000) starting [junit4] 2> 1575498 WARN (TEST-OverseerTest.testStateChange-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.Overseer Unable to check back-compat of .system collection - can't obtain ClusterState. [junit4] 2> 1575500 INFO (OverseerStateUpdate-72187932591652877-127.0.0.1:56332_solr-n_0000000000) [ ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:56332_solr [junit4] 2> 1575530 INFO (TEST-OverseerTest.testStateChange-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.AbstractDistribZkTestBase verifyReplicaStatus (collection1) shard=shard1 coreNodeName=core_node1 [junit4] 2> 1575631 INFO (zkCallback-7259-thread-3) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [0]) [junit4] 2> 1575631 INFO (zkCallback-7259-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [0]) [junit4] 2> 1575634 INFO (TEST-OverseerTest.testStateChange-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.AbstractDistribZkTestBase verifyReplicaStatus (collection1) shard=shard1 coreNodeName=core_node1 [junit4] 2> 1575739 INFO (zkCallback-7259-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [0]) [junit4] 2> 1575739 INFO (zkCallback-7259-thread-3) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [0]) [junit4] 2> 1575739 INFO (zkCallback-7259-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [0]) [junit4] 2> 1575743 WARN (TEST-OverseerTest.testStateChange-seed#[FE8B3A34B4F7525A]-SendThread(127.0.0.1:56332)) [ ] o.a.z.ClientCnxn An exception was thrown while closing send thread for session 0x100768acf32000d. [junit4] 2> => EndOfStreamException: Unable to read additional data from server sessionid 0x100768acf32000d, likely server has closed socket [junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:75) [junit4] 2> org.apache.zookeeper.ClientCnxn$EndOfStreamException: Unable to read additional data from server sessionid 0x100768acf32000d, likely server has closed socket [junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:75) ~[zookeeper-3.6.1.jar:3.6.1] [junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:348) ~[zookeeper-3.6.1.jar:3.6.1] [junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1262) [zookeeper-3.6.1.jar:3.6.1] [junit4] 2> 1576071 INFO (closeThreadPool-7357-thread-1) [ ] o.a.s.c.Overseer Overseer (id=72187932591652877-127.0.0.1:56332_solr-n_0000000000) closing [junit4] 2> 1576071 INFO (OverseerStateUpdate-72187932591652877-127.0.0.1:56332_solr-n_0000000000) [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:56332_solr [junit4] 2> 1576073 WARN (OverseerExitThread) [ ] o.a.s.c.Overseer Error communicating with ZooKeeper [junit4] 2> => org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:134) [junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:134) ~[zookeeper-3.6.1.jar:3.6.1] [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) ~[zookeeper-3.6.1.jar:3.6.1] [junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:2348) ~[zookeeper-3.6.1.jar:3.6.1] [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:339) ~[java/:?] [junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71) ~[java/:?] [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:339) ~[java/:?] [junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:395) [java/:?] [junit4] 2> at java.lang.Thread.run(Thread.java:834) [?:?] [junit4] 2> 1576170 INFO (TEST-OverseerTest.testStateChange-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testStateChange [junit4] 2> 1576181 INFO (TEST-OverseerTest.testShardAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testShardAssignment [junit4] 2> 1576389 INFO (TEST-OverseerTest.testShardAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 1576392 INFO (zkConnectionManagerCallback-7359-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 1576392 INFO (TEST-OverseerTest.testShardAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 1576394 WARN (TEST-OverseerTest.testShardAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.e.j.u.s.S.config Trusting all certificates configured for Client@13561afa[provider=null,keyStore=null,trustStore=null] [junit4] 2> 1576394 WARN (TEST-OverseerTest.testShardAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@13561afa[provider=null,keyStore=null,trustStore=null] [junit4] 2> 1576394 INFO (TEST-OverseerTest.testShardAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true] [junit4] 2> 1576395 WARN (TEST-OverseerTest.testShardAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.e.j.u.s.S.config Trusting all certificates configured for Client@54d56d06[provider=null,keyStore=null,trustStore=null] [junit4] 2> 1576395 WARN (TEST-OverseerTest.testShardAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@54d56d06[provider=null,keyStore=null,trustStore=null] [junit4] 2> 1576396 INFO (TEST-OverseerTest.testShardAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 1576399 INFO (zkConnectionManagerCallback-7367-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 1576399 INFO (TEST-OverseerTest.testShardAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 1576401 INFO (TEST-OverseerTest.testShardAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 1576404 INFO (zkConnectionManagerCallback-7372-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 1576405 INFO (TEST-OverseerTest.testShardAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 1576410 INFO (TEST-OverseerTest.testShardAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:56332/solr ready [junit4] 2> 1576415 INFO (TEST-OverseerTest.testShardAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:56332_solr [junit4] 2> 1576417 INFO (TEST-OverseerTest.testShardAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.Overseer Overseer (id=72187932591652880-127.0.0.1:56332_solr-n_0000000000) starting [junit4] 2> 1576434 WARN (TEST-OverseerTest.testShardAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.Overseer Unable to check back-compat of .system collection - can't obtain ClusterState. [junit4] 2> 1576435 INFO (OverseerStateUpdate-72187932591652880-127.0.0.1:56332_solr-n_0000000000) [ ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:56332_solr [junit4] 2> 1576443 INFO (TEST-OverseerTest.testShardAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 1576446 INFO (zkConnectionManagerCallback-7378-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 1576446 INFO (TEST-OverseerTest.testShardAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 1576458 INFO (zkCallback-7358-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 1576459 INFO (zkCallback-7259-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 1576459 INFO (zkCallback-7377-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 1576459 INFO (zkCallback-7371-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 1576580 INFO (zkCallback-7377-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1]) [junit4] 2> 1576588 INFO (TEST-OverseerTest.testShardAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection1/leaders/shard1 [junit4] 2> 1576595 INFO (TEST-OverseerTest.testShardAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/collection1/leaders/shard1/leader after winning as /collections/collection1/leader_elect/shard1/election/72187932591652883-127.0.0.1_core1-n_0000000000 [junit4] 2> 1576712 INFO (zkCallback-7377-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1]) [junit4] 2> 1576712 INFO (zkCallback-7377-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1]) [junit4] 2> 1576721 INFO (TEST-OverseerTest.testShardAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection1/leaders/shard2 [junit4] 2> 1576730 INFO (TEST-OverseerTest.testShardAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/collection1/leaders/shard2/leader after winning as /collections/collection1/leader_elect/shard2/election/72187932591652883-127.0.0.1_core2-n_0000000000 [junit4] 2> 1576850 INFO (zkCallback-7377-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1]) [junit4] 2> 1576850 INFO (zkCallback-7377-thread-3) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1]) [junit4] 2> 1576850 INFO (zkCallback-7377-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1]) [junit4] 2> 1576860 INFO (TEST-OverseerTest.testShardAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection1/leaders/shard3 [junit4] 2> 1576872 INFO (TEST-OverseerTest.testShardAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/collection1/leaders/shard3/leader after winning as /collections/collection1/leader_elect/shard3/election/72187932591652883-127.0.0.1_core3-n_0000000000 [junit4] 2> 1576994 INFO (zkCallback-7377-thread-3) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1]) [junit4] 2> 1576994 INFO (zkCallback-7377-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1]) [junit4] 2> 1576994 INFO (zkCallback-7377-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1]) [junit4] 2> 1576995 INFO (zkCallback-7377-thread-4) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1]) [junit4] 2> 1577007 INFO (TEST-OverseerTest.testShardAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection1/leaders/shard1 [junit4] 2> 1577123 INFO (zkCallback-7377-thread-4) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1]) [junit4] 2> 1577123 INFO (zkCallback-7377-thread-3) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1]) [junit4] 2> 1577123 INFO (zkCallback-7377-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1]) [junit4] 2> 1577123 INFO (zkCallback-7377-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1]) [junit4] 2> 1577124 INFO (zkCallback-7377-thread-5) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1]) [junit4] 2> 1577137 INFO (TEST-OverseerTest.testShardAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection1/leaders/shard2 [junit4] 2> 1577254 INFO (zkCallback-7377-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1]) [junit4] 2> 1577254 INFO (zkCallback-7377-thread-5) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1]) [junit4] 2> 1577254 INFO (zkCallback-7377-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1]) [junit4] 2> 1577254 INFO (zkCallback-7377-thread-3) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1]) [junit4] 2> 1577255 INFO (zkCallback-7377-thread-4) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1]) [junit4] 2> 1577255 INFO (zkCallback-7377-thread-6) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1]) [junit4] 2> 1577265 INFO (TEST-OverseerTest.testShardAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection1/leaders/shard3 [junit4] 2> 1577283 INFO (TEST-OverseerTest.testShardAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.ShardLeaderElectionContextBase No version found for ephemeral leader parent node, won't remove previous leader registration. [junit4] 2> 1577288 INFO (TEST-OverseerTest.testShardAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.ShardLeaderElectionContextBase No version found for ephemeral leader parent node, won't remove previous leader registration. [junit4] 2> 1577288 INFO (TEST-OverseerTest.testShardAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.ShardLeaderElectionContextBase No version found for ephemeral leader parent node, won't remove previous leader registration. [junit4] 2> 1577290 WARN (zkCallback-7377-thread-3) [ ] o.a.s.c.LeaderElector Our node is no longer in line to be leader [junit4] 2> 1577292 WARN (zkCallback-7377-thread-3) [ ] o.a.s.c.LeaderElector Our node is no longer in line to be leader [junit4] 2> 1577294 WARN (zkCallback-7377-thread-3) [ ] o.a.s.c.LeaderElector Our node is no longer in line to be leader [junit4] 2> 1577297 INFO (zkCallback-7371-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0) [junit4] 2> 1577297 INFO (zkCallback-7358-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0) [junit4] 2> 1577298 INFO (zkCallback-7377-thread-3) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0) [junit4] 2> 1577298 INFO (zkCallback-7259-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0) [junit4] 2> 1577405 WARN (TEST-OverseerTest.testShardAssignment-seed#[FE8B3A34B4F7525A]-SendThread(127.0.0.1:56332)) [ ] o.a.z.ClientCnxn An exception was thrown while closing send thread for session 0x100768acf320010. [junit4] 2> => EndOfStreamException: Unable to read additional data from server sessionid 0x100768acf320010, likely server has closed socket [junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:75) [junit4] 2> org.apache.zookeeper.ClientCnxn$EndOfStreamException: Unable to read additional data from server sessionid 0x100768acf320010, likely server has closed socket [junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:75) ~[zookeeper-3.6.1.jar:3.6.1] [junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:348) ~[zookeeper-3.6.1.jar:3.6.1] [junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1262) [zookeeper-3.6.1.jar:3.6.1] [junit4] 2> 1577522 WARN (TEST-OverseerTest.testShardAssignment-seed#[FE8B3A34B4F7525A]-SendThread(127.0.0.1:56332)) [ ] o.a.z.ClientCnxn An exception was thrown while closing send thread for session 0x100768acf320012. [junit4] 2> => EndOfStreamException: Unable to read additional data from server sessionid 0x100768acf320012, likely server has closed socket [junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:75) [junit4] 2> org.apache.zookeeper.ClientCnxn$EndOfStreamException: Unable to read additional data from server sessionid 0x100768acf320012, likely server has closed socket [junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:75) ~[zookeeper-3.6.1.jar:3.6.1] [junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:348) ~[zookeeper-3.6.1.jar:3.6.1] [junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1262) [zookeeper-3.6.1.jar:3.6.1] [junit4] 2> 1577725 INFO (closeThreadPool-7382-thread-1) [ ] o.a.s.c.Overseer Overseer (id=72187932591652880-127.0.0.1:56332_solr-n_0000000000) closing [junit4] 2> 1577725 INFO (OverseerStateUpdate-72187932591652880-127.0.0.1:56332_solr-n_0000000000) [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:56332_solr [junit4] 2> 1577727 WARN (OverseerExitThread) [ ] o.a.s.c.Overseer Error communicating with ZooKeeper [junit4] 2> => org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:134) [junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:134) ~[zookeeper-3.6.1.jar:3.6.1] [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) ~[zookeeper-3.6.1.jar:3.6.1] [junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:2348) ~[zookeeper-3.6.1.jar:3.6.1] [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:339) ~[java/:?] [junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71) ~[java/:?] [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:339) ~[java/:?] [junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:395) [java/:?] [junit4] 2> at java.lang.Thread.run(Thread.java:834) [?:?] [junit4] 2> 1577901 INFO (TEST-OverseerTest.testShardAssignment-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testShardAssignment [junit4] 2> 1577910 INFO (TEST-OverseerTest.testLatchWatcher-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testLatchWatcher [junit4] 2> 1578019 INFO (TEST-OverseerTest.testLatchWatcher-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testLatchWatcher [junit4] 2> 1578028 INFO (TEST-OverseerTest.testRemovalOfLastReplica-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testRemovalOfLastReplica [junit4] 2> 1578246 INFO (TEST-OverseerTest.testRemovalOfLastReplica-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 1578249 INFO (zkConnectionManagerCallback-7388-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 1578249 INFO (TEST-OverseerTest.testRemovalOfLastReplica-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 1578251 WARN (TEST-OverseerTest.testRemovalOfLastReplica-seed#[FE8B3A34B4F7525A]) [ ] o.e.j.u.s.S.config Trusting all certificates configured for Client@6c1a21d1[provider=null,keyStore=null,trustStore=null] [junit4] 2> 1578251 WARN (TEST-OverseerTest.testRemovalOfLastReplica-seed#[FE8B3A34B4F7525A]) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@6c1a21d1[provider=null,keyStore=null,trustStore=null] [junit4] 2> 1578251 INFO (TEST-OverseerTest.testRemovalOfLastReplica-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true] [junit4] 2> 1578252 WARN (TEST-OverseerTest.testRemovalOfLastReplica-seed#[FE8B3A34B4F7525A]) [ ] o.e.j.u.s.S.config Trusting all certificates configured for Client@31181891[provider=null,keyStore=null,trustStore=null] [junit4] 2> 1578253 WARN (TEST-OverseerTest.testRemovalOfLastReplica-seed#[FE8B3A34B4F7525A]) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@31181891[provider=null,keyStore=null,trustStore=null] [junit4] 2> 1578255 INFO (TEST-OverseerTest.testRemovalOfLastReplica-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 1578259 INFO (zkConnectionManagerCallback-7396-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 1578259 INFO (TEST-OverseerTest.testRemovalOfLastReplica-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 1578264 INFO (TEST-OverseerTest.testRemovalOfLastReplica-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 1578267 INFO (zkConnectionManagerCallback-7401-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 1578267 INFO (TEST-OverseerTest.testRemovalOfLastReplica-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 1578271 INFO (TEST-OverseerTest.testRemovalOfLastReplica-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:56332/solr ready [junit4] 2> 1578277 INFO (TEST-OverseerTest.testRemovalOfLastReplica-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:56332_solr [junit4] 2> 1578279 INFO (TEST-OverseerTest.testRemovalOfLastReplica-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.Overseer Overseer (id=72187932591652884-127.0.0.1:56332_solr-n_0000000000) starting [junit4] 2> 1578296 WARN (TEST-OverseerTest.testRemovalOfLastReplica-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.Overseer Unable to check back-compat of .system collection - can't obtain ClusterState. [junit4] 2> 1578296 INFO (OverseerStateUpdate-72187932591652884-127.0.0.1:56332_solr-n_0000000000) [ ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:56332_solr [junit4] 2> 1578308 INFO (zkCallback-7259-thread-3) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeCreated path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [0]) [junit4] 2> 1578319 INFO (TEST-OverseerTest.testRemovalOfLastReplica-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.AbstractDistribZkTestBase verifyReplicaStatus (collection1) shard=shard1 coreNodeName=core_node5 [junit4] 2> 1578433 INFO (zkCallback-7259-thread-3) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [0]) [junit4] 2> 1578433 INFO (zkCallback-7259-thread-4) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [0]) [junit4] 2> 1578433 INFO (zkCallback-7259-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [0]) [junit4] 2> 1578436 INFO (TEST-OverseerTest.testRemovalOfLastReplica-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.AbstractDistribZkTestBase verifyReplicaStatus (collection1) shard=shard2 coreNodeName=core_node6 [junit4] 2> 1578440 INFO (TEST-OverseerTest.testRemovalOfLastReplica-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.AbstractDistribZkTestBase verifyReplicaStatus (collection1) shard=shard1 coreNodeName=core_node3 [junit4] 2> 1578442 INFO (TEST-OverseerTest.testRemovalOfLastReplica-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.AbstractDistribZkTestBase verifyReplicaStatus (collection1) shard=shard2 coreNodeName=core_node4 [junit4] 2> 1578443 INFO (TEST-OverseerTest.testRemovalOfLastReplica-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.AbstractDistribZkTestBase verifyReplicaStatus (collection1) shard=shard1 coreNodeName=core_node1 [junit4] 2> 1578444 INFO (TEST-OverseerTest.testRemovalOfLastReplica-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.AbstractDistribZkTestBase verifyReplicaStatus (collection1) shard=shard2 coreNodeName=core_node2 [junit4] 2> 1578458 INFO (TEST-OverseerTest.testRemovalOfLastReplica-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.AbstractDistribZkTestBase verifyReplicaStatus (collection1) shard=shard1 coreNodeName=core_node5 [junit4] 2> 1578566 INFO (zkCallback-7259-thread-3) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [0]) [junit4] 2> 1578566 INFO (zkCallback-7259-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [0]) [junit4] 2> 1578566 INFO (zkCallback-7259-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [0]) [junit4] 2> 1578566 INFO (zkCallback-7259-thread-4) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [0]) [junit4] 2> 1578566 INFO (zkCallback-7259-thread-5) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [0]) [junit4] 2> 1578567 INFO (zkCallback-7259-thread-6) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [0]) [junit4] 2> 1578567 INFO (zkCallback-7259-thread-7) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [0]) [junit4] 2> 1578568 INFO (zkCallback-7259-thread-8) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [0]) [junit4] 2> 1578568 INFO (TEST-OverseerTest.testRemovalOfLastReplica-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.AbstractDistribZkTestBase verifyReplicaStatus (collection1) shard=shard2 coreNodeName=core_node6 [junit4] 2> 1578572 INFO (TEST-OverseerTest.testRemovalOfLastReplica-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.AbstractDistribZkTestBase verifyReplicaStatus (collection1) shard=shard1 coreNodeName=core_node3 [junit4] 2> 1578577 INFO (TEST-OverseerTest.testRemovalOfLastReplica-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.AbstractDistribZkTestBase verifyReplicaStatus (collection1) shard=shard2 coreNodeName=core_node4 [junit4] 2> 1578578 INFO (TEST-OverseerTest.testRemovalOfLastReplica-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.AbstractDistribZkTestBase verifyReplicaStatus (collection1) shard=shard1 coreNodeName=core_node1 [junit4] 2> 1578578 INFO (TEST-OverseerTest.testRemovalOfLastReplica-seed#[FE8B3A34B4F7525A]) [ ] o.a.s.c.AbstractDistribZkTestBase verifyReplicaStatus (collection1) shard=shard2 coreNodeName=core_node2 [junit4] 2> 1578686 INFO (zkCallback-7259-thread-5) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [0]) [junit4] 2> 1578686 INFO (zkCallback-7259-thread-6) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [0]) [junit4] 2> 1578686 INFO (zkCallback-7259-thread-7) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [0]) [junit4] 2> 1578686 INFO (zkCallback-7259-thread-8) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [0]) [junit4] 2> 1578686 INFO (zkCallback-7259-thread-4) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [0]) [junit4] 2> 1578686 INFO (zkCallback-7259-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [0]) [junit4] 2> 1578686 INFO (zkCallback-7259-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [0]) [junit4] 2> 1578686 INFO (zkCallback-7259-thread-9) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [0]) [junit4] 2> 1578686 INFO (zkCallback-7259-thread-3) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [0]) [junit4] 2> 1578687 INFO (zkCallback-7259-thread-10) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [0]) [junit4] 2> 1578688 INFO (zkCallback-7259-thread-11) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [0]) [junit4] 2> 1578689 INFO (zkCallback-7259-thread-12) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [0]) [junit4] 2> 1578690 INFO (zkCallback-7259-thread-13) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [0]) [junit4] 2> 1578812 INFO (zkCallback-7259-thread-4) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [0]) [junit4] 2> 1578812 INFO (zkCallback-7259-thread-8) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [0]) [junit4] 2> 1578812 INFO (zkCallback-7259-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [0]) [junit4] 2> 1578812 INFO (zkCallback-7259-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [0]) [junit4] 2> 1578812 INFO (zkCallback-7259-thread-7) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [0]) [junit4] 2> 1578812 INFO (zkCallback-7259-thread-9) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [0]) [junit4] 2> 1578812 INFO (zkCallback-7259-thread-12) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [0]) [junit4] 2> 1578812 INFO (zkCallback-7259-thread-10) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [0]) [junit4] 2> 1578812 INFO (zkCallback-7259-thread-13) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [0]) [junit4] 2> 1578812 INFO (zkCallback-7259-thread-14) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [0] [...truncated too long message...] nagerCLITest-seed#[FE8B3A34B4F7525A]-worker) [ ] o.a.s.c.ZkTestServer Shutting down ZkTestServer. [junit4] 2> 1595473 WARN (SUITE-PackageManagerCLITest-seed#[FE8B3A34B4F7525A]-worker-SendThread(127.0.0.1:56379)) [ ] o.a.z.ClientCnxn An exception was thrown while closing send thread for session 0x100768ae20e0001. [junit4] 2> => EndOfStreamException: Unable to read additional data from server sessionid 0x100768ae20e0001, likely server has closed socket [junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:75) [junit4] 2> org.apache.zookeeper.ClientCnxn$EndOfStreamException: Unable to read additional data from server sessionid 0x100768ae20e0001, likely server has closed socket [junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:75) ~[zookeeper-3.6.1.jar:3.6.1] [junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:348) ~[zookeeper-3.6.1.jar:3.6.1] [junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1262) [zookeeper-3.6.1.jar:3.6.1] [junit4] 2> 1595579 WARN (ZkTestServer Run Thread) [ ] o.a.s.c.ZkTestServer Watch limit violations: [junit4] 2> Maximum concurrent create/delete watches above limit: [junit4] 2> [junit4] 2> 10 /solr/configs/conf1 [junit4] 2> 2 /solr/aliases.json [junit4] 2> 2 /solr/collections/def/terms/shard1 [junit4] 2> 2 /solr/clusterprops.json [junit4] 2> 2 /solr/collections/abc/terms/shard1 [junit4] 2> [junit4] 2> Maximum concurrent data watches above limit: [junit4] 2> [junit4] 2> 12 /solr/collections/abc/state.json [junit4] 2> 12 /solr/collections/def/state.json [junit4] 2> 2 /solr/packages.json [junit4] 2> [junit4] 2> Maximum concurrent children watches above limit: [junit4] 2> [junit4] 2> 23 /solr/overseer/queue [junit4] 2> 8 /solr/overseer/collection-queue-work [junit4] 2> 6 /solr/collections [junit4] 2> 4 /solr/live_nodes [junit4] 2> [junit4] 2> 1595583 INFO (SUITE-PackageManagerCLITest-seed#[FE8B3A34B4F7525A]-worker) [ ] o.a.s.c.ZkTestServer waitForServerDown: 127.0.0.1:56379 [junit4] 2> 1595583 INFO (SUITE-PackageManagerCLITest-seed#[FE8B3A34B4F7525A]-worker) [ ] o.a.s.c.ZkTestServer parse host and port list: 127.0.0.1:56379 [junit4] 2> 1595583 INFO (SUITE-PackageManagerCLITest-seed#[FE8B3A34B4F7525A]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1 56379 [junit4] 2> 1595583 INFO (SUITE-PackageManagerCLITest-seed#[FE8B3A34B4F7525A]-worker) [ ] o.a.s.SolrTestCaseJ4 ------------------------------------------------------- Done waiting for tracked resources to be released [junit4] 2> Aug 12, 2020 4:57:39 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks [junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s). [junit4] 2> Aug 12, 2020 4:57:49 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks [junit4] 2> SEVERE: 1 thread leaked from SUITE scope at org.apache.solr.cloud.PackageManagerCLITest: [junit4] 2> 1) Thread[id=16545, name=SessionTracker, state=TIMED_WAITING, group=TGRP-PackageManagerCLITest] [junit4] 2> at [email protected]/java.lang.Thread.sleep(Native Method) [junit4] 2> at app//org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:157) [junit4] 2> Aug 12, 2020 4:57:49 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll [junit4] 2> INFO: Starting to interrupt leaked threads: [junit4] 2> 1) Thread[id=16545, name=SessionTracker, state=TIMED_WAITING, group=TGRP-PackageManagerCLITest] [junit4] 2> 1605699 ERROR (SessionTracker) [ ] o.a.z.s.ZooKeeperCriticalThread Severe unrecoverable error, from thread : SessionTracker [junit4] 2> => java.lang.InterruptedException: sleep interrupted [junit4] 2> at java.base/java.lang.Thread.sleep(Native Method) [junit4] 2> java.lang.InterruptedException: sleep interrupted [junit4] 2> at java.lang.Thread.sleep(Native Method) ~[?:?] [junit4] 2> at org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:157) [zookeeper-3.6.1.jar:3.6.1] [junit4] 2> Aug 12, 2020 4:57:50 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll [junit4] 2> INFO: All leaked threads terminated. [junit4] 2> NOTE: test params are: codec=Asserting(Lucene86): {}, docValues:{}, maxPointsInLeafNode=1960, maxMBSortInHeap=7.845498835766988, sim=Asserting(RandomSimilarity(queryNorm=false): {}), locale=cgg, timezone=GMT0 [junit4] 2> NOTE: Mac OS X 10.14.6 x86_64/AdoptOpenJDK 11.0.6 (64-bit)/cpus=6,threads=1,free=172528576,total=357761024 [junit4] 2> NOTE: All tests run in this JVM: [IndexSchemaRuntimeFieldTest, TestFieldSortValues, ZkDynamicConfigTest, MultiSolrCloudTestCaseTest, TestMultiValuedNumericRangeQuery, TestSearcherReuse, TestJsonFacetRefinement, TestExternalRoleRuleBasedAuthorizationPlugin, TestComponentsName, TestDownShardTolerantSearch, BigEndianAscendingWordDeserializerTest, TestSystemIdResolver, ReplaceNodeTest, TestEmbeddedSolrServerConstructors, TestRTGBase, TestLegacyFieldCache, TestDistributedTracing, V2ApiIntegrationTest, ResourceLoaderTest, TestSchemaSimilarityResource, SuggestComponentContextFilterQueryTest, SuggesterFSTTest, TestLRUStatsCache, LeaderFailureAfterFreshStartTest, CSVRequestHandlerTest, TaggerTest, TestCoreAdminApis, MigrateRouteKeyTest, SortByFunctionTest, OutputWriterTest, TestDFISimilarityFactory, ResponseLogComponentTest, TestAddFieldRealTimeGet, TestStressUserVersions, TestRuleBasedAuthorizationWithKerberos, TestLegacyBM25SimilarityFactory, ForceLeaderTest, OverriddenZkACLAndCredentialsProvidersTest, TestCustomStream, TestCursorMarkWithoutUniqueKey, TestCloudManagedSchema, TestSubQueryTransformerCrossCore, SpellCheckCollatorWithCollapseTest, IgnoreCommitOptimizeUpdateProcessorFactoryTest, ConjunctionSolrSpellCheckerTest, CoreAdminCreateDiscoverTest, TestLockTree, HLLSerializationTest, BasicAuthStandaloneTest, ReturnFieldsTest, TestHighFrequencyDictionaryFactory, TestCollectionsAPIViaSolrCloudCluster, TestTolerantUpdateProcessorCloud, TestRecovery, SSLMigrationTest, PingRequestHandlerTest, TestSizeLimitedDistributedMap, OverseerSolrResponseUnsafeSerializationTest, MergeStrategyTest, HighlighterMaxOffsetTest, TestReplicationHandlerBackup, SpatialHeatmapFacetsTest, TestStressCloudBlindAtomicUpdates, DocValuesMissingTest, SyncSliceTest, RankFieldTest, TestAnalyzedSuggestions, DefaultValueUpdateProcessorTest, DeleteNodeTest, TestConfigSetsAPIExclusivity, InfixSuggestersTest, CachingDirectoryFactoryTest, CategoryRoutedAliasUpdateProcessorTest, TestMacroExpander, TestObjectReleaseTracker, TestJmxIntegration, TestManagedSchema, CursorPagingTest, CurrencyRangeFacetCloudTest, TestExportWriter, SecurityConfHandlerTest, VersionInfoTest, TestDocumentBuilder, TestGroupingSearch, TestJoin, TestRandomDVFaceting, TestSimpleTrackingShardHandler, TestTolerantSearch, PathHierarchyTokenizerFactoryTest, TestDeprecatedFilters, TestLuceneMatchVersion, TestWordDelimiterFilterFactory, TestJettySolrRunner, ActionThrottleTest, AddReplicaTest, AssignBackwardCompatibilityTest, BasicDistributedZk2Test, ChaosMonkeySafeLeaderTest, CloudExitableDirectoryReaderTest, CollectionsAPISolrJTest, PackageManagerCLITest] [junit4] 2> NOTE: reproduce with: ant test -Dtestcase=PackageManagerCLITest -Dtests.seed=FE8B3A34B4F7525A -Dtests.slow=true -Dtests.locale=cgg -Dtests.timezone=GMT0 -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1 [junit4] ERROR 0.00s J3 | PackageManagerCLITest (suite) <<< [junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.PackageManagerCLITest: [junit4] > 1) Thread[id=16545, name=SessionTracker, state=TIMED_WAITING, group=TGRP-PackageManagerCLITest] [junit4] > at [email protected]/java.lang.Thread.sleep(Native Method) [junit4] > at app//org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:157) [junit4] > at __randomizedtesting.SeedInfo.seed([FE8B3A34B4F7525A]:0) [junit4] Completed [477/865 (2!)] on J3 in 29.64s, 1 test, 1 error <<< FAILURES! [...truncated 53255 lines...] [repro] Jenkins log URL: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/6408/consoleText [repro] Revision: e72a0d66c666dec7d69f1f2e15306957656d2e70 [repro] Ant options: "-Dargs=-XX:-UseCompressedOops -XX:+UseSerialGC" [repro] Repro line: ant test -Dtestcase=OverseerTest -Dtests.seed=FE8B3A34B4F7525A -Dtests.slow=true -Dtests.locale=fr-DZ -Dtests.timezone=America/Cayenne -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1 [repro] Repro line: ant test -Dtestcase=PackageManagerCLITest -Dtests.seed=FE8B3A34B4F7525A -Dtests.slow=true -Dtests.locale=cgg -Dtests.timezone=GMT0 -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1 [repro] JUnit rest result XML files will be moved to: ./repro-reports [repro] ant clean [...truncated 6 lines...] [repro] Test suites by module: [repro] solr/core [repro] PackageManagerCLITest [repro] OverseerTest [repro] ant compile-test [...truncated 2446 lines...] [repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=10 -Dtests.class="*.PackageManagerCLITest|*.OverseerTest" -Dtests.showOutput=onerror "-Dargs=-XX:-UseCompressedOops -XX:+UseSerialGC" -Dtests.seed=FE8B3A34B4F7525A -Dtests.slow=true -Dtests.locale=cgg -Dtests.timezone=GMT0 -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1 [...truncated 161 lines...] [repro] Failures w/original seeds: [repro] 0/5 failed: org.apache.solr.cloud.OverseerTest [repro] 0/5 failed: org.apache.solr.cloud.PackageManagerCLITest [repro] Exiting with code 0 [...truncated 79 lines...]
--------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
