Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/1189/
1 tests failed.
FAILED:
org.apache.solr.cloud.OverseerTest.testExternalClusterStateChangeBehavior
Error Message:
Illegal state, was: down expected:active clusterState:live
nodes:[]collections:{c1=DocCollection(c1)={ "shards":{"shard1":{
"parent":null, "range":null, "state":"active",
"replicas":{"core_node1":{ "base_url":"http://127.0.0.1/solr";,
"node_name":"node1", "core":"core1", "roles":"",
"state":"down"}}}}, "router":{"name":"implicit"}},
test=LazyCollectionRef(test)}
Stack Trace:
java.lang.AssertionError: Illegal state, was: down expected:active
clusterState:live nodes:[]collections:{c1=DocCollection(c1)={
"shards":{"shard1":{
"parent":null,
"range":null,
"state":"active",
"replicas":{"core_node1":{
"base_url":"http://127.0.0.1/solr";,
"node_name":"node1",
"core":"core1",
"roles":"",
"state":"down"}}}},
"router":{"name":"implicit"}}, test=LazyCollectionRef(test)}
at
__randomizedtesting.SeedInfo.seed([CEC4632E0A269E2:64F245DE023233AC]:0)
at org.junit.Assert.fail(Assert.java:93)
at
org.apache.solr.cloud.AbstractDistribZkTestBase.verifyReplicaStatus(AbstractDistribZkTestBase.java:243)
at
org.apache.solr.cloud.OverseerTest.testExternalClusterStateChangeBehavior(OverseerTest.java:1273)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 11234 lines...]
[junit4] Suite: org.apache.solr.cloud.OverseerTest
[junit4] 2> Creating dataDir:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.OverseerTest_CEC4632E0A269E2-001/init-core-data-001
[junit4] 2> 922171 INFO
(SUITE-OverseerTest-seed#[CEC4632E0A269E2]-worker) [ ] o.a.s.SolrTestCaseJ4
Randomized ssl (true) and clientAuth (false) via:
@org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 922171 INFO
(SUITE-OverseerTest-seed#[CEC4632E0A269E2]-worker) [ ] o.a.s.SolrTestCaseJ4
####initCore
[junit4] 2> 922171 INFO
(SUITE-OverseerTest-seed#[CEC4632E0A269E2]-worker) [ ] o.a.s.SolrTestCaseJ4
####initCore end
[junit4] 2> 922172 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.SolrTestCaseJ4 ###Starting testShardLeaderChange
[junit4] 2> 922173 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 922173 INFO (Thread-2697) [ ] o.a.s.c.ZkTestServer client
port:0.0.0.0/0.0.0.0:0
[junit4] 2> 922173 INFO (Thread-2697) [ ] o.a.s.c.ZkTestServer
Starting server
[junit4] 2> 922273 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.ZkTestServer start zk server on port:45676
[junit4] 2> 922273 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 922274 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 922275 INFO (zkCallback-15332-thread-1) [ ]
o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@f3096f7 name:ZooKeeperConnection
Watcher:127.0.0.1:45676/solr got event WatchedEvent state:SyncConnected
type:None path:null path:null type:None
[junit4] 2> 922275 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 922275 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 922276 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 922276 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 922276 INFO (zkCallback-15333-thread-1) [ ]
o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@12e618d name:ZooKeeperConnection
Watcher:127.0.0.1:45676 got event WatchedEvent state:SyncConnected type:None
path:null path:null type:None
[junit4] 2> 922276 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 922277 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 922277 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 922277 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 922278 INFO (zkCallback-15334-thread-1) [ ]
o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@7a82a1aa
name:ZooKeeperConnection Watcher:127.0.0.1:45676 got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 922278 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 922278 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 922278 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.SolrZkClient makePath: /solr
[junit4] 2> 922280 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.SolrZkClient makePath: /live_nodes
[junit4] 2> 922281 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.SolrZkClient makePath: /collections
[junit4] 2> 922281 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.SolrZkClient makePath: /aliases.json
[junit4] 2> 922282 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
[junit4] 2> 922283 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.SolrZkClient makePath: /security.json
[junit4] 2> 922284 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 922284 INFO (Thread-2698) [ ] o.a.s.c.c.SolrZkClient Using
default ZkCredentialsProvider
[junit4] 2> 922284 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.ZkStateReader Loaded empty cluster properties
[junit4] 2> 922284 INFO (Thread-2698) [ ] o.a.s.c.c.ConnectionManager
Waiting for client to connect to ZooKeeper
[junit4] 2> 922285 INFO (zkCallback-15336-thread-1) [ ]
o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@5a1b88b2
name:ZooKeeperConnection Watcher:127.0.0.1:45676/solr got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 922285 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
[junit4] 2> 922285 INFO (Thread-2698) [ ] o.a.s.c.c.ConnectionManager
Client is connected to ZooKeeper
[junit4] 2> 922285 INFO (Thread-2698) [ ] o.a.s.c.c.SolrZkClient Using
default ZkACLProvider
[junit4] 2> 922286 INFO (Thread-2698) [ ] o.a.s.u.UpdateShardHandler
Creating UpdateShardHandler HTTP client with params:
[junit4] 2> 922287 INFO (Thread-2698) [ ] o.a.s.c.c.SolrZkClient
makePath: /overseer_elect
[junit4] 2> 922287 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 922288 INFO (Thread-2698) [ ] o.a.s.c.c.SolrZkClient
makePath: /overseer_elect/election
[junit4] 2> 922288 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 922288 INFO (Thread-2698) [ ] o.a.s.c.Overseer Overseer
(id=null) closing
[junit4] 2> 922289 INFO (zkCallback-15340-thread-1) [ ]
o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@7d5e4c18
name:ZooKeeperConnection Watcher:127.0.0.1:45676/solr got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 922289 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 922289 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 922289 INFO (Thread-2698) [ ] o.a.s.c.LeaderElector Joined
leadership election with path:
/overseer_elect/election/96003445639282691-127.0.0.1:45676_solr-n_0000000000
[junit4] 2> 922289 INFO (Thread-2698) [ ]
o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:45676_solr
[junit4] 2> 922289 INFO (Thread-2698) [ ] o.a.s.c.c.SolrZkClient
makePath: /overseer_elect/leader
[junit4] 2> 922290 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 922290 INFO (Thread-2698) [ ] o.a.s.c.Overseer Overseer
(id=96003445639282691-127.0.0.1:45676_solr-n_0000000000) starting
[junit4] 2> 922290 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.ZkStateReader Loaded empty cluster properties
[junit4] 2> 922290 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
[junit4] 2> 922290 INFO (Thread-2698) [ ] o.a.s.c.c.SolrZkClient
makePath: /overseer/queue
[junit4] 2> 922291 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.SolrZkClient makePath: /live_nodes/node1
[junit4] 2> 922291 INFO (Thread-2698) [ ] o.a.s.c.c.SolrZkClient
makePath: /overseer/queue-work
[junit4] 2> 922291 INFO (zkCallback-15332-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live
nodes size: [0])
[junit4] 2> 922291 INFO (zkCallback-15340-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live
nodes size: [0])
[junit4] 2> 922296 INFO (zkCallback-15332-thread-1) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 922298 INFO (zkCallback-15340-thread-1) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 922298 INFO (Thread-2698) [ ] o.a.s.c.c.SolrZkClient
makePath: /overseer/collection-map-failure
[junit4] 2> 922299 INFO (Thread-2698) [ ] o.a.s.c.c.SolrZkClient
makePath: /overseer/collection-map-running
[junit4] 2> 922300 INFO (Thread-2698) [ ] o.a.s.c.c.SolrZkClient
makePath: /overseer/collection-map-completed
[junit4] 2> 922301 INFO (Thread-2698) [ ] o.a.s.c.c.SolrZkClient
makePath: /overseer/collection-queue-work
[junit4] 2> 922302 INFO (Thread-2698) [ ]
o.a.s.c.OverseerAutoReplicaFailoverThread Starting
OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000
autoReplicaFailoverWaitAfterExpiration=30000
autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 922303 INFO
(OverseerCollectionConfigSetProcessor-96003445639282691-127.0.0.1:45676_solr-n_0000000000)
[ ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer
operations
[junit4] 2> 922303 INFO (Thread-2698) [ ] o.a.s.c.OverseerTest Killing
overseer.
[junit4] 2> 922303 INFO
(OverseerStateUpdate-96003445639282691-127.0.0.1:45676_solr-n_0000000000) [
] o.a.s.c.Overseer Starting to work on the main queue
[junit4] 2> 922303 INFO (Thread-2698) [ ] o.a.s.c.c.SolrZkClient Using
default ZkCredentialsProvider
[junit4] 2> 922303 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ]
o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from
client sessionid 0x15512a00a8b0003, likely client has closed socket
[junit4] 2> at
org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at
org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 922304 INFO (Thread-2698) [ ] o.a.s.c.c.ConnectionManager
Waiting for client to connect to ZooKeeper
[junit4] 2> 922304 INFO (zkCallback-15342-thread-1) [ ]
o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@6d9468af
name:ZooKeeperConnection Watcher:127.0.0.1:45676/solr got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 922304 INFO (Thread-2698) [ ] o.a.s.c.c.ConnectionManager
Client is connected to ZooKeeper
[junit4] 2> 922304 INFO (Thread-2698) [ ] o.a.s.c.c.SolrZkClient Using
default ZkACLProvider
[junit4] 2> 922304 INFO (Thread-2698) [ ] o.a.s.c.Overseer Overseer
(id=96003445639282691-127.0.0.1:45676_solr-n_0000000000) closing
[junit4] 2> 922304 INFO (Thread-2698) [ ] o.a.s.u.UpdateShardHandler
Creating UpdateShardHandler HTTP client with params:
[junit4] 2> 922305 INFO
(OverseerStateUpdate-96003445639282691-127.0.0.1:45676_solr-n_0000000000) [
] o.a.s.c.Overseer According to ZK I
(id=96003445639282691-127.0.0.1:45676_solr-n_0000000000) am no longer a leader.
[junit4] 2> 922305 INFO
(OverseerStateUpdate-96003445639282691-127.0.0.1:45676_solr-n_0000000000) [
] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:45676_solr
[junit4] 2> 922305 ERROR
(OverseerCollectionConfigSetProcessor-96003445639282691-127.0.0.1:45676_solr-n_0000000000)
[ ] o.a.s.c.OverseerTaskProcessor Unable to prioritize overseer
[junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException:
KeeperErrorCode = Session expired for /roles.json
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at
org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1045)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:314)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:311)
[junit4] 2> at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:311)
[junit4] 2> at
org.apache.solr.cloud.OverseerNodePrioritizer.prioritizeOverseerNodes(OverseerNodePrioritizer.java:60)
[junit4] 2> at
org.apache.solr.cloud.OverseerTaskProcessor.run(OverseerTaskProcessor.java:154)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 922314 INFO (Thread-2698) [ ] o.a.s.c.Overseer Overseer
(id=null) closing
[junit4] 2> 922314 INFO (Thread-2698) [ ] o.a.s.c.LeaderElector Joined
leadership election with path:
/overseer_elect/election/96003445639282693-127.0.0.1:45676_solr-n_0000000001
[junit4] 2> 922314 INFO (Thread-2698) [ ]
o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:45676_solr
[junit4] 2> 922314 ERROR (OverseerExitThread) [ ] o.a.s.c.Overseer
could not read the data
[junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException:
KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at
org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:348)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:345)
[junit4] 2> at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:345)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:309)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:89)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater$2.run(Overseer.java:268)
[junit4] 2> 922314 INFO (Thread-2698) [ ] o.a.s.c.c.SolrZkClient
makePath: /overseer_elect/leader
[junit4] 2> 922316 INFO (Thread-2698) [ ] o.a.s.c.Overseer Overseer
(id=96003445639282693-127.0.0.1:45676_solr-n_0000000001) starting
[junit4] 2> 922319 INFO (Thread-2698) [ ]
o.a.s.c.OverseerAutoReplicaFailoverThread Starting
OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000
autoReplicaFailoverWaitAfterExpiration=30000
autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 922319 INFO
(OverseerCollectionConfigSetProcessor-96003445639282693-127.0.0.1:45676_solr-n_0000000001)
[ ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer
operations
[junit4] 2> 922320 INFO
(OverseerStateUpdate-96003445639282693-127.0.0.1:45676_solr-n_0000000001) [
] o.a.s.c.Overseer Starting to work on the main queue
[junit4] 2> 922320 INFO
(OverseerStateUpdate-96003445639282693-127.0.0.1:45676_solr-n_0000000001) [
] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 922320 INFO
(OverseerStateUpdate-96003445639282693-127.0.0.1:45676_solr-n_0000000001) [
] o.a.s.c.c.ZkStateReader Loaded empty cluster properties
[junit4] 2> 922321 INFO
(OverseerStateUpdate-96003445639282693-127.0.0.1:45676_solr-n_0000000001) [
] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 922322 INFO
(OverseerStateUpdate-96003445639282693-127.0.0.1:45676_solr-n_0000000001) [
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"} current state version: 0
[junit4] 2> 922322 INFO
(OverseerStateUpdate-96003445639282693-127.0.0.1:45676_solr-n_0000000001) [
] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 922322 INFO
(OverseerStateUpdate-96003445639282693-127.0.0.1:45676_solr-n_0000000001) [
] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
[junit4] 2> 922322 INFO
(OverseerStateUpdate-96003445639282693-127.0.0.1:45676_solr-n_0000000001) [
] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 922322 INFO (zkCallback-15332-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred
- updating... (live nodes size: [1])
[junit4] 2> 922323 INFO (zkCallback-15342-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred
- updating... (live nodes size: [1])
[junit4] 2> 922324 INFO (zkCallback-15340-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred
- updating... (live nodes size: [1])
[junit4] 2> 922798 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.SolrZkClient makePath:
/collections/collection1/leader_elect/shard1/election
[junit4] 2> 922799 INFO (zkCallback-15342-thread-1) [ ]
o.a.s.c.c.ZkStateReader A collections change: [WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/collections], has occurred - updating...
[junit4] 2> 922799 INFO (zkCallback-15340-thread-1) [ ]
o.a.s.c.c.ZkStateReader A collections change: [WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/collections], has occurred - updating...
[junit4] 2> 922799 INFO (zkCallback-15332-thread-1) [ ]
o.a.s.c.c.ZkStateReader A collections change: [WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/collections], has occurred - updating...
[junit4] 2> 922801 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.LeaderElector Joined leadership election with path:
/collections/collection1/leader_elect/shard1/election/96003445639282692-node1_core1-n_0000000000
[junit4] 2> 922801 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.SolrZkClient makePath: /collections/collection1/leaders/shard1
[junit4] 2> 922802 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
/collections/collection1/leaders/shard1/leader after winning as
/collections/collection1/leader_elect/shard1/election/96003445639282692-node1_core1-n_0000000000
[junit4] 2> 922804 INFO
(OverseerStateUpdate-96003445639282693-127.0.0.1:45676_solr-n_0000000001) [
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "base_url":"http://node1/solr/";,
[junit4] 2> "core":"core1",
[junit4] 2> "state":"active"} current state version: 1
[junit4] 2> 922804 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.ElectionContext Canceling election
/collections/collection1/leader_elect/shard1/election/96003445639282692-node1_core1-n_0000000000
[junit4] 2> 922805 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.ShardLeaderElectionContextBase Removing leader registration node on
cancel: /collections/collection1/leaders/shard1/leader 1
[junit4] 2> 922805 INFO
(OverseerStateUpdate-96003445639282693-127.0.0.1:45676_solr-n_0000000001) [
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"} current state version: 1
[junit4] 2> 922805 INFO
(OverseerStateUpdate-96003445639282693-127.0.0.1:45676_solr-n_0000000001) [
] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 922805 INFO
(OverseerStateUpdate-96003445639282693-127.0.0.1:45676_solr-n_0000000001) [
] o.a.s.c.o.ReplicaMutator shard=shard1 is already registered
[junit4] 2> 922806 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.SolrZkClient makePath:
/collections/collection1/leader_elect/shard1/election
[junit4] 2> 922807 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.LeaderElector Joined leadership election with path:
/collections/collection1/leader_elect/shard1/election/96003445639282692-node1_core1-n_0000000001
[junit4] 2> 922813 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
/collections/collection1/leaders/shard1/leader after winning as
/collections/collection1/leader_elect/shard1/election/96003445639282692-node1_core1-n_0000000001
[junit4] 2> 922815 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 922815 INFO
(OverseerStateUpdate-96003445639282693-127.0.0.1:45676_solr-n_0000000001) [
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "base_url":"http://node1/solr/";,
[junit4] 2> "core":"core1",
[junit4] 2> "state":"active"} current state version: 1
[junit4] 2> 922815 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 922816 INFO (zkCallback-15346-thread-1) [ ]
o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@3f3b442f
name:ZooKeeperConnection Watcher:127.0.0.1:45676/solr got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 922816 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 922816 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 922816 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 922817 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.ZkStateReader Loaded empty cluster properties
[junit4] 2> 922817 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 922818 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.SolrZkClient makePath: /live_nodes/node2
[junit4] 2> 922818 INFO (zkCallback-15346-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live
nodes size: [1])
[junit4] 2> 922818 INFO (zkCallback-15332-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live
nodes size: [1])
[junit4] 2> 922818 INFO (zkCallback-15342-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live
nodes size: [1])
[junit4] 2> 922818 INFO (zkCallback-15332-thread-1) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 922818 INFO (zkCallback-15340-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live
nodes size: [1])
[junit4] 2> 922819 INFO (zkCallback-15342-thread-1) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 922819 INFO (zkCallback-15346-thread-1) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 922819 INFO (zkCallback-15340-thread-1) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 922820 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.ElectionContext Canceling election
/collections/collection1/leader_elect/shard1/election/96003445639282692-node1_core1-n_0000000001
[junit4] 2> 922820 INFO
(OverseerStateUpdate-96003445639282693-127.0.0.1:45676_solr-n_0000000001) [
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"} current state version: 1
[junit4] 2> 922820 INFO
(OverseerStateUpdate-96003445639282693-127.0.0.1:45676_solr-n_0000000001) [
] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 922820 INFO
(OverseerStateUpdate-96003445639282693-127.0.0.1:45676_solr-n_0000000001) [
] o.a.s.c.o.ReplicaMutator shard=shard1 is already registered
[junit4] 2> 922820 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.ShardLeaderElectionContextBase Removing leader registration node on
cancel: /collections/collection1/leaders/shard1/leader 2
[junit4] 2> 922821 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.SolrZkClient makePath:
/collections/collection1/leader_elect/shard1/election
[junit4] 2> 922822 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.LeaderElector Joined leadership election with path:
/collections/collection1/leader_elect/shard1/election/96003445639282692-node1_core1-n_0000000002
[junit4] 2> 922822 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
/collections/collection1/leaders/shard1/leader after winning as
/collections/collection1/leader_elect/shard1/election/96003445639282692-node1_core1-n_0000000002
[junit4] 2> 922826 INFO
(OverseerStateUpdate-96003445639282693-127.0.0.1:45676_solr-n_0000000001) [
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "base_url":"http://node1/solr/";,
[junit4] 2> "core":"core1",
[junit4] 2> "state":"active"} current state version: 1
[junit4] 2> 922928 INFO (zkCallback-15332-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred
- updating... (live nodes size: [2])
[junit4] 2> 922928 INFO (zkCallback-15342-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred
- updating... (live nodes size: [2])
[junit4] 2> 922928 INFO (zkCallback-15340-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred
- updating... (live nodes size: [2])
[junit4] 2> 922929 INFO (zkCallback-15346-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred
- updating... (live nodes size: [2])
[junit4] 2> 923027 INFO
(OverseerStateUpdate-96003445639282693-127.0.0.1:45676_solr-n_0000000001) [
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node2",
[junit4] 2> "core":"core4",
[junit4] 2> "core_node_name":"node2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node2/solr/"} current state version: 2
[junit4] 2> 923027 INFO
(OverseerStateUpdate-96003445639282693-127.0.0.1:45676_solr-n_0000000001) [
] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node2",
[junit4] 2> "core":"core4",
[junit4] 2> "core_node_name":"node2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node2/solr/"}
[junit4] 2> 923027 INFO
(OverseerStateUpdate-96003445639282693-127.0.0.1:45676_solr-n_0000000001) [
] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=1
[junit4] 2> 923027 INFO
(OverseerStateUpdate-96003445639282693-127.0.0.1:45676_solr-n_0000000001) [
] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 923128 INFO (zkCallback-15346-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred
- updating... (live nodes size: [2])
[junit4] 2> 923128 INFO (zkCallback-15340-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred
- updating... (live nodes size: [2])
[junit4] 2> 923128 INFO (zkCallback-15332-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred
- updating... (live nodes size: [2])
[junit4] 2> 923129 INFO (zkCallback-15342-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred
- updating... (live nodes size: [2])
[junit4] 2> 923527 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.SolrZkClient makePath:
/collections/collection1/leader_elect/shard1/election
[junit4] 2> 923528 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.LeaderElector Joined leadership election with path:
/collections/collection1/leader_elect/shard1/election/96003445639282694-node2_core4-n_0000000003
[junit4] 2> 923529 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.LeaderElector Watching path
/collections/collection1/leader_elect/shard1/election/96003445639282692-node1_core1-n_0000000002
to know if I could be the leader
[junit4] 2> 923529 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.ElectionContext Canceling election
/collections/collection1/leader_elect/shard1/election/96003445639282692-node1_core1-n_0000000002
[junit4] 2> 923529 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.ShardLeaderElectionContextBase Removing leader registration node on
cancel: /collections/collection1/leaders/shard1/leader 3
[junit4] 2> 923530 INFO (zkCallback-15346-thread-1) [ ]
o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
/collections/collection1/leaders/shard1/leader after winning as
/collections/collection1/leader_elect/shard1/election/96003445639282694-node2_core4-n_0000000003
[junit4] 2> 923530 INFO (zkCallback-15342-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live
nodes size: [2])
[junit4] 2> 923530 INFO (zkCallback-15332-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live
nodes size: [2])
[junit4] 2> 923530 INFO (zkCallback-15340-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live
nodes size: [2])
[junit4] 2> 923530 INFO (zkCallback-15346-thread-2) [ ]
o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live
nodes size: [2])
[junit4] 2> 923531 INFO (zkCallback-15342-thread-1) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 923531 INFO (zkCallback-15332-thread-1) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 923531 INFO (zkCallback-15346-thread-2) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 923533 INFO
(OverseerStateUpdate-96003445639282693-127.0.0.1:45676_solr-n_0000000001) [
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "base_url":"http://node2/solr/";,
[junit4] 2> "core":"core4",
[junit4] 2> "state":"active"} current state version: 3
[junit4] 2> 923634 INFO (zkCallback-15342-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred
- updating... (live nodes size: [1])
[junit4] 2> 923634 INFO (zkCallback-15332-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred
- updating... (live nodes size: [1])
[junit4] 2> 923634 INFO (zkCallback-15346-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred
- updating... (live nodes size: [1])
[junit4] 2> 925033 WARN (zkCallback-15340-thread-1) [ ]
o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK:
[KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 925033 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 925034 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 925034 INFO (zkCallback-15348-thread-1) [ ]
o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@7f67c011
name:ZooKeeperConnection Watcher:127.0.0.1:45676/solr got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 925035 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 925035 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 925035 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 925036 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.ZkStateReader Loaded empty cluster properties
[junit4] 2> 925036 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 925037 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.SolrZkClient makePath: /live_nodes/node1
[junit4] 2> 925037 INFO (zkCallback-15342-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live
nodes size: [1])
[junit4] 2> 925037 INFO (zkCallback-15332-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live
nodes size: [1])
[junit4] 2> 925037 INFO (zkCallback-15346-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live
nodes size: [1])
[junit4] 2> 925038 INFO (zkCallback-15342-thread-1) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 925038 INFO (zkCallback-15332-thread-1) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 925038 INFO (zkCallback-15346-thread-1) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 925038 INFO (zkCallback-15348-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live
nodes size: [1])
[junit4] 2> 925039 INFO (zkCallback-15348-thread-1) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 925039 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.SolrZkClient makePath:
/collections/collection1/leader_elect/shard1/election
[junit4] 2> 925039 INFO
(OverseerStateUpdate-96003445639282693-127.0.0.1:45676_solr-n_0000000001) [
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"} current state version: 4
[junit4] 2> 925039 INFO
(OverseerStateUpdate-96003445639282693-127.0.0.1:45676_solr-n_0000000001) [
] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 925039 INFO
(OverseerStateUpdate-96003445639282693-127.0.0.1:45676_solr-n_0000000001) [
] o.a.s.c.o.ReplicaMutator shard=shard1 is already registered
[junit4] 2> 925041 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.LeaderElector Joined leadership election with path:
/collections/collection1/leader_elect/shard1/election/96003445639282695-node1_core1-n_0000000004
[junit4] 2> 925041 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.LeaderElector Watching path
/collections/collection1/leader_elect/shard1/election/96003445639282694-node2_core4-n_0000000003
to know if I could be the leader
[junit4] 2> 925041 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.ElectionContext Canceling election
/collections/collection1/leader_elect/shard1/election/96003445639282694-node2_core4-n_0000000003
[junit4] 2> 925041 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.ShardLeaderElectionContextBase Removing leader registration node on
cancel: /collections/collection1/leaders/shard1/leader 4
[junit4] 2> 925045 INFO (zkCallback-15342-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live
nodes size: [2])
[junit4] 2> 925045 INFO (zkCallback-15332-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live
nodes size: [2])
[junit4] 2> 925046 INFO (zkCallback-15346-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live
nodes size: [2])
[junit4] 2> 925046 INFO (zkCallback-15348-thread-2) [ ]
o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live
nodes size: [2])
[junit4] 2> 925046 INFO (zkCallback-15332-thread-1) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 925046 INFO (zkCallback-15346-thread-1) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 925046 INFO (zkCallback-15342-thread-1) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 925054 INFO (zkCallback-15348-thread-1) [ ]
o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
/collections/collection1/leaders/shard1/leader after winning as
/collections/collection1/leader_elect/shard1/election/96003445639282695-node1_core1-n_0000000004
[junit4] 2> 925054 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.ElectionContext Canceling election
/collections/collection1/leader_elect/shard1/election/96003445639282695-node1_core1-n_0000000004
[junit4] 2> 925055 INFO (zkCallback-15348-thread-2) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 925055 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.ShardLeaderElectionContextBase Removing leader registration node on
cancel: /collections/collection1/leaders/shard1/leader 5
[junit4] 2> 925055 INFO
(OverseerStateUpdate-96003445639282693-127.0.0.1:45676_solr-n_0000000001) [
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"} current state version: 4
[junit4] 2> 925055 INFO
(OverseerStateUpdate-96003445639282693-127.0.0.1:45676_solr-n_0000000001) [
] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 925055 INFO
(OverseerStateUpdate-96003445639282693-127.0.0.1:45676_solr-n_0000000001) [
] o.a.s.c.o.ReplicaMutator shard=shard1 is already registered
[junit4] 2> 925055 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.SolrZkClient makePath:
/collections/collection1/leader_elect/shard1/election
[junit4] 2> 925056 INFO
(OverseerStateUpdate-96003445639282693-127.0.0.1:45676_solr-n_0000000001) [
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "base_url":"http://node1/solr/";,
[junit4] 2> "core":"core1",
[junit4] 2> "state":"active"} current state version: 4
[junit4] 2> 925057 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.LeaderElector Joined leadership election with path:
/collections/collection1/leader_elect/shard1/election/96003445639282695-node1_core1-n_0000000005
[junit4] 2> 925057 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
/collections/collection1/leaders/shard1/leader after winning as
/collections/collection1/leader_elect/shard1/election/96003445639282695-node1_core1-n_0000000005
[junit4] 2> 925058 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 925059 INFO
(OverseerStateUpdate-96003445639282693-127.0.0.1:45676_solr-n_0000000001) [
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "base_url":"http://node1/solr/";,
[junit4] 2> "core":"core1",
[junit4] 2> "state":"active"} current state version: 4
[junit4] 2> 925062 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 925065 INFO (zkCallback-15350-thread-1) [ ]
o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@4f82cdef
name:ZooKeeperConnection Watcher:127.0.0.1:45676/solr got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 925065 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 925066 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 925066 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 925066 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.ZkStateReader Loaded empty cluster properties
[junit4] 2> 925067 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 925067 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.SolrZkClient makePath: /live_nodes/node2
[junit4] 2> 925068 INFO (zkCallback-15350-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live
nodes size: [1])
[junit4] 2> 925068 INFO (zkCallback-15332-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live
nodes size: [1])
[junit4] 2> 925068 INFO (zkCallback-15348-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live
nodes size: [1])
[junit4] 2> 925068 INFO (zkCallback-15342-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live
nodes size: [1])
[junit4] 2> 925071 INFO (zkCallback-15332-thread-1) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 925071 INFO (zkCallback-15342-thread-1) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 925071 INFO (zkCallback-15348-thread-1) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 925071 INFO (zkCallback-15350-thread-1) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 925072 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.ElectionContext Canceling election
/collections/collection1/leader_elect/shard1/election/96003445639282695-node1_core1-n_0000000005
[junit4] 2> 925072 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.ShardLeaderElectionContextBase Removing leader registration node on
cancel: /collections/collection1/leaders/shard1/leader 6
[junit4] 2> 925072 INFO
(OverseerStateUpdate-96003445639282693-127.0.0.1:45676_solr-n_0000000001) [
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"} current state version: 4
[junit4] 2> 925072 INFO
(OverseerStateUpdate-96003445639282693-127.0.0.1:45676_solr-n_0000000001) [
] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 925072 INFO
(OverseerStateUpdate-96003445639282693-127.0.0.1:45676_solr-n_0000000001) [
] o.a.s.c.o.ReplicaMutator shard=shard1 is already registered
[junit4] 2> 925073 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.SolrZkClient makePath:
/collections/collection1/leader_elect/shard1/election
[junit4] 2> 925074 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.LeaderElector Joined leadership election with path:
/collections/collection1/leader_elect/shard1/election/96003445639282695-node1_core1-n_0000000006
[junit4] 2> 925074 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
/collections/collection1/leaders/shard1/leader after winning as
/collections/collection1/leader_elect/shard1/election/96003445639282695-node1_core1-n_0000000006
[junit4] 2> 925075 INFO
(OverseerStateUpdate-96003445639282693-127.0.0.1:45676_solr-n_0000000001) [
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "base_url":"http://node1/solr/";,
[junit4] 2> "core":"core1",
[junit4] 2> "state":"active"} current state version: 4
[junit4] 2> 925131 INFO (Thread-2698) [ ] o.a.s.c.OverseerTest Killing
overseer.
[junit4] 2> 925131 INFO (Thread-2698) [ ] o.a.s.c.c.SolrZkClient Using
default ZkCredentialsProvider
[junit4] 2> 925132 INFO (Thread-2698) [ ] o.a.s.c.c.ConnectionManager
Waiting for client to connect to ZooKeeper
[junit4] 2> 925133 INFO (zkCallback-15352-thread-1) [ ]
o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@3c17b7ab
name:ZooKeeperConnection Watcher:127.0.0.1:45676/solr got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 925133 INFO (Thread-2698) [ ] o.a.s.c.c.ConnectionManager
Client is connected to ZooKeeper
[junit4] 2> 925133 INFO (Thread-2698) [ ] o.a.s.c.c.SolrZkClient Using
default ZkACLProvider
[junit4] 2> 925133 INFO (Thread-2698) [ ] o.a.s.c.Overseer Overseer
(id=96003445639282693-127.0.0.1:45676_solr-n_0000000001) closing
[junit4] 2> 925133 INFO
(OverseerStateUpdate-96003445639282693-127.0.0.1:45676_solr-n_0000000001) [
] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:45676_solr
[junit4] 2> 925133 INFO (Thread-2698) [ ] o.a.s.u.UpdateShardHandler
Creating UpdateShardHandler HTTP client with params:
[junit4] 2> 925140 INFO (Thread-2698) [ ] o.a.s.c.Overseer Overseer
(id=null) closing
[junit4] 2> 925141 INFO (Thread-2698) [ ] o.a.s.c.LeaderElector Joined
leadership election with path:
/overseer_elect/election/96003445639282697-127.0.0.1:45676_solr-n_0000000002
[junit4] 2> 925141 INFO (Thread-2698) [ ]
o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:45676_solr
[junit4] 2> 925141 INFO (Thread-2698) [ ] o.a.s.c.c.SolrZkClient
makePath: /overseer_elect/leader
[junit4] 2> 925142 INFO (Thread-2698) [ ] o.a.s.c.Overseer Overseer
(id=96003445639282697-127.0.0.1:45676_solr-n_0000000002) starting
[junit4] 2> 925144 INFO (Thread-2698) [ ]
o.a.s.c.OverseerAutoReplicaFailoverThread Starting
OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000
autoReplicaFailoverWaitAfterExpiration=30000
autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 925145 INFO
(OverseerCollectionConfigSetProcessor-96003445639282697-127.0.0.1:45676_solr-n_0000000002)
[ ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer
operations
[junit4] 2> 925145 INFO
(OverseerStateUpdate-96003445639282697-127.0.0.1:45676_solr-n_0000000002) [
] o.a.s.c.Overseer Starting to work on the main queue
[junit4] 2> 925145 INFO
(OverseerStateUpdate-96003445639282697-127.0.0.1:45676_solr-n_0000000002) [
] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 925145 INFO
(OverseerStateUpdate-96003445639282697-127.0.0.1:45676_solr-n_0000000002) [
] o.a.s.c.c.ZkStateReader Loaded empty cluster properties
[junit4] 2> 925146 INFO
(OverseerStateUpdate-96003445639282697-127.0.0.1:45676_solr-n_0000000002) [
] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 925147 INFO
(OverseerStateUpdate-96003445639282697-127.0.0.1:45676_solr-n_0000000002) [
] o.a.s.c.Overseer processMessage: workQueueSize: 6, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 925147 INFO
(OverseerStateUpdate-96003445639282697-127.0.0.1:45676_solr-n_0000000002) [
] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 925147 INFO
(OverseerStateUpdate-96003445639282697-127.0.0.1:45676_solr-n_0000000002) [
] o.a.s.c.o.ReplicaMutator shard=shard1 is already registered
[junit4] 2> 925147 INFO (zkCallback-15350-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred
- updating... (live nodes size: [2])
[junit4] 2> 925147 ERROR (OverseerExitThread) [ ] o.a.s.c.Overseer
could not read the data
[junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException:
KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at
org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:348)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:345)
[junit4] 2> at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:345)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:309)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:89)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater$2.run(Overseer.java:268)
[junit4] 2> 925147 INFO (zkCallback-15332-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred
- updating... (live nodes size: [2])
[junit4] 2> 925147 INFO (zkCallback-15348-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred
- updating... (live nodes size: [2])
[junit4] 2> 925149 INFO (zkCallback-15352-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred
- updating... (live nodes size: [2])
[junit4] 2> 925150 INFO
(OverseerStateUpdate-96003445639282697-127.0.0.1:45676_solr-n_0000000002) [
] o.a.s.c.Overseer processMessage: workQueueSize: 6, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 925150 INFO
(OverseerStateUpdate-96003445639282697-127.0.0.1:45676_solr-n_0000000002) [
] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 925150 INFO
(OverseerStateUpdate-96003445639282697-127.0.0.1:45676_solr-n_0000000002) [
] o.a.s.c.o.ReplicaMutator shard=shard1 is already registered
[junit4] 2> 925158 INFO (zkCallback-15348-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred
- updating... (live nodes size: [2])
[junit4] 2> 925158 INFO (zkCallback-15352-thread-2) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred
- updating... (live nodes size: [2])
[junit4] 2> 925158 INFO (zkCallback-15350-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred
- updating... (live nodes size: [2])
[junit4] 2> 925158 INFO (zkCallback-15332-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred
- updating... (live nodes size: [2])
[junit4] 2> 925160 INFO
(OverseerStateUpdate-96003445639282697-127.0.0.1:45676_solr-n_0000000002) [
] o.a.s.c.Overseer processMessage: workQueueSize: 6, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "base_url":"http://node1/solr/";,
[junit4] 2> "core":"core1",
[junit4] 2> "state":"active"}
[junit4] 2> 925160 INFO (zkCallback-15352-thread-2) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred
- updating... (live nodes size: [2])
[junit4] 2> 925160 INFO (zkCallback-15348-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred
- updating... (live nodes size: [2])
[junit4] 2> 925160 INFO (zkCallback-15350-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred
- updating... (live nodes size: [2])
[junit4] 2> 925161 INFO (zkCallback-15332-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred
- updating... (live nodes size: [2])
[junit4] 2> 925162 INFO
(OverseerStateUpdate-96003445639282697-127.0.0.1:45676_solr-n_0000000002) [
] o.a.s.c.Overseer processMessage: workQueueSize: 6, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "base_url":"http://node1/solr/";,
[junit4] 2> "core":"core1",
[junit4] 2> "state":"active"}
[junit4] 2> 925164 INFO (zkCallback-15352-thread-2) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred
- updating... (live nodes size: [2])
[junit4] 2> 925164 INFO (zkCallback-15332-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred
- updating... (live nodes size: [2])
[junit4] 2> 925164 INFO (zkCallback-15350-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred
- updating... (live nodes size: [2])
[junit4] 2> 925165 INFO (zkCallback-15348-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred
- updating... (live nodes size: [2])
[junit4] 2> 925166 INFO
(OverseerStateUpdate-96003445639282697-127.0.0.1:45676_solr-n_0000000002) [
] o.a.s.c.Overseer processMessage: workQueueSize: 6, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 925166 INFO
(OverseerStateUpdate-96003445639282697-127.0.0.1:45676_solr-n_0000000002) [
] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 925166 INFO
(OverseerStateUpdate-96003445639282697-127.0.0.1:45676_solr-n_0000000002) [
] o.a.s.c.o.ReplicaMutator shard=shard1 is already registered
[junit4] 2> 925166 INFO (zkCallback-15352-thread-2) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred
- updating... (live nodes size: [2])
[junit4] 2> 925166 INFO (zkCallback-15348-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred
- updating... (live nodes size: [2])
[junit4] 2> 925166 INFO (zkCallback-15350-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred
- updating... (live nodes size: [2])
[junit4] 2> 925166 INFO (zkCallback-15332-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred
- updating... (live nodes size: [2])
[junit4] 2> 925167 INFO
(OverseerStateUpdate-96003445639282697-127.0.0.1:45676_solr-n_0000000002) [
] o.a.s.c.Overseer processMessage: workQueueSize: 6, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "base_url":"http://node1/solr/";,
[junit4] 2> "core":"core1",
[junit4] 2> "state":"active"}
[junit4] 2> 925168 INFO (zkCallback-15352-thread-2) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred
- updating... (live nodes size: [2])
[junit4] 2> 925168 INFO (zkCallback-15348-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred
- updating... (live nodes size: [2])
[junit4] 2> 925168 INFO (zkCallback-15350-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred
- updating... (live nodes size: [2])
[junit4] 2> 925168 INFO (zkCallback-15332-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred
- updating... (live nodes size: [2])
[junit4] 2> 925276 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.SolrZkClient makePath:
/collections/collection1/leader_elect/shard1/election
[junit4] 2> 925276 INFO
(OverseerStateUpdate-96003445639282697-127.0.0.1:45676_solr-n_0000000002) [
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node2",
[junit4] 2> "core":"core4",
[junit4] 2> "core_node_name":"node2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node2/solr/"} current state version: 10
[junit4] 2> 925276 INFO
(OverseerStateUpdate-96003445639282697-127.0.0.1:45676_solr-n_0000000002) [
] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node2",
[junit4] 2> "core":"core4",
[junit4] 2> "core_node_name":"node2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node2/solr/"}
[junit4] 2> 925277 INFO
(OverseerStateUpdate-96003445639282697-127.0.0.1:45676_solr-n_0000000002) [
] o.a.s.c.o.ReplicaMutator shard=shard1 is already registered
[junit4] 2> 925278 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.LeaderElector Joined leadership election with path:
/collections/collection1/leader_elect/shard1/election/96003445639282696-node2_core4-n_0000000007
[junit4] 2> 925278 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.LeaderElector Watching path
/collections/collection1/leader_elect/shard1/election/96003445639282695-node1_core1-n_0000000006
to know if I could be the leader
[junit4] 2> 925278 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.ElectionContext Canceling election
/collections/collection1/leader_elect/shard1/election/96003445639282695-node1_core1-n_0000000006
[junit4] 2> 925278 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.ShardLeaderElectionContextBase Removing leader registration node on
cancel: /collections/collection1/leaders/shard1/leader 7
[junit4] 2> 925279 INFO (zkCallback-15332-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live
nodes size: [2])
[junit4] 2> 925279 INFO (zkCallback-15348-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live
nodes size: [2])
[junit4] 2> 925279 INFO (zkCallback-15350-thread-1) [ ]
o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
/collections/collection1/leaders/shard1/leader after winning as
/collections/collection1/leader_elect/shard1/election/96003445639282696-node2_core4-n_0000000007
[junit4] 2> 925279 INFO (zkCallback-15332-thread-1) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 925280 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ]
o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from
client sessionid 0x15512a00a8b0007, likely client has closed socket
[junit4] 2> at
org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at
org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 925280 INFO
(OverseerStateUpdate-96003445639282697-127.0.0.1:45676_solr-n_0000000002) [
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "base_url":"http://node2/solr/";,
[junit4] 2> "core":"core4",
[junit4] 2> "state":"active"} current state version: 10
[junit4] 2> 925282 INFO (zkCallback-15352-thread-2) [ ]
o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live
nodes size: [2])
[junit4] 2> 925282 INFO (zkCallback-15350-thread-2) [ ]
o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live
nodes size: [2])
[junit4] 2> 925282 INFO (zkCallback-15352-thread-2) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 925283 INFO (zkCallback-15350-thread-2) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 925382 INFO (zkCallback-15352-thread-2) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred
- updating... (live nodes size: [1])
[junit4] 2> 925382 INFO (zkCallback-15350-thread-2) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred
- updating... (live nodes size: [1])
[junit4] 2> 925382 INFO (zkCallback-15332-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred
- updating... (live nodes size: [1])
[junit4] 2> 926779 WARN (zkCallback-15348-thread-1) [ ]
o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK:
[KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 926780 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 926781 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 926781 INFO (zkCallback-15356-thread-1) [ ]
o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@44d796e4
name:ZooKeeperConnection Watcher:127.0.0.1:45676/solr got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 926781 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 926782 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 926786 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 926787 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.ZkStateReader Loaded empty cluster properties
[junit4] 2> 926787 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 926788 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.SolrZkClient makePath: /live_nodes/node1
[junit4] 2> 926789 INFO (zkCallback-15352-thread-2) [ ]
o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live
nodes size: [1])
[junit4] 2> 926789 INFO (zkCallback-15350-thread-2) [ ]
o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live
nodes size: [1])
[junit4] 2> 926789 INFO (zkCallback-15332-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live
nodes size: [1])
[junit4] 2> 926789 INFO (zkCallback-15352-thread-2) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 926789 INFO (zkCallback-15350-thread-2) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 926790 INFO (zkCallback-15356-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live
nodes size: [1])
[junit4] 2> 926790 INFO (zkCallback-15332-thread-1) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 926791 INFO (zkCallback-15356-thread-1) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 926791 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.c.SolrZkClient makePath:
/collections/collection1/leader_elect/shard1/election
[junit4] 2> 926792 INFO
(OverseerStateUpdate-96003445639282697-127.0.0.1:45676_solr-n_0000000002) [
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"} current state version: 11
[junit4] 2> 926792 INFO
(OverseerStateUpdate-96003445639282697-127.0.0.1:45676_solr-n_0000000002) [
] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 926792 INFO
(OverseerStateUpdate-96003445639282697-127.0.0.1:45676_solr-n_0000000002) [
] o.a.s.c.o.ReplicaMutator shard=shard1 is already registered
[junit4] 2> 926796 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.LeaderElector Joined leadership election with path:
/collections/collection1/leader_elect/shard1/election/96003445639282698-node1_core1-n_0000000008
[junit4] 2> 926797 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.LeaderElector Watching path
/collections/collection1/leader_elect/shard1/election/96003445639282696-node2_core4-n_0000000007
to know if I could be the leader
[junit4] 2> 926797 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.ElectionContext Canceling election
/collections/collection1/leader_elect/shard1/election/96003445639282696-node2_core4-n_0000000007
[junit4] 2> 926797 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.ShardLeaderElectionContextBase Removing leader registration node on
cancel: /collections/collection1/leaders/shard1/leader 8
[junit4] 2> 926798 INFO (zkCallback-15350-thread-2) [ ]
o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live
nodes size: [2])
[junit4] 2> 926798 INFO (zkCallback-15332-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live
nodes size: [2])
[junit4] 2> 926798 INFO (zkCallback-15356-thread-2) [ ]
o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live
nodes size: [2])
[junit4] 2> 926798 INFO (zkCallback-15332-thread-1) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 926798 INFO (zkCallback-15356-thread-1) [ ]
o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
/collections/collection1/leaders/shard1/leader after winning as
/collections/collection1/leader_elect/shard1/election/96003445639282698-node1_core1-n_0000000008
[junit4] 2> 926798 INFO (zkCallback-15352-thread-2) [ ]
o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live
nodes size: [2])
[junit4] 2> 926799 INFO (zkCallback-15352-thread-2) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 926799 INFO (zkCallback-15356-thread-2) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 926801 INFO
(OverseerStateUpdate-96003445639282697-127.0.0.1:45676_solr-n_0000000002) [
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "base_url":"http://node1/solr/";,
[junit4] 2> "core":"core1",
[junit4] 2> "state":"active"} current state version: 11
[junit4] 2> 926847 INFO (Thread-2698) [ ] o.a.s.c.OverseerTest Killing
overseer.
[junit4] 2> 926848 INFO (Thread-2698) [ ] o.a.s.c.c.SolrZkClient Using
default ZkCredentialsProvider
[junit4] 2> 926849 INFO (Thread-2698) [ ] o.a.s.c.c.ConnectionManager
Waiting for client to connect to ZooKeeper
[junit4] 2> 926850 INFO (zkCallback-15358-thread-1) [ ]
o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@47569e39
name:ZooKeeperConnection Watcher:127.0.0.1:45676/solr got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 926850 INFO (Thread-2698) [ ] o.a.s.c.c.ConnectionManager
Client is connected to ZooKeeper
[junit4] 2> 926850 INFO (Thread-2698) [ ] o.a.s.c.c.SolrZkClient Using
default ZkACLProvider
[junit4] 2> 926850 INFO (Thread-2698) [ ] o.a.s.c.Overseer Overseer
(id=96003445639282697-127.0.0.1:45676_solr-n_0000000002) closing
[junit4] 2> 926851 INFO
(OverseerStateUpdate-96003445639282697-127.0.0.1:45676_solr-n_0000000002) [
] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:45676_solr
[junit4] 2> 926852 INFO (Thread-2698) [ ] o.a.s.u.UpdateShardHandler
Creating UpdateShardHandler HTTP client with params:
[junit4] 2> 926857 ERROR (OverseerExitThread) [ ] o.a.s.c.Overseer
could not read the data
[junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException:
KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.c
[...truncated too long message...]
3)
[junit4] 2> 969062 INFO (zkCallback-15538-thread-1) [ ]
o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
/collections/collection1/leaders/shard1/leader after winning as
/collections/collection1/leader_elect/shard1/election/96003448551309327-node11_core11-n_0000000002
[junit4] 2> 969062 INFO
(TEST-OverseerTest.testShardAssignmentBigger-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.ShardLeaderElectionContextBase Removing leader registration node on
cancel: /collections/collection1/leaders/shard3/leader 3
[junit4] 2> 969062 INFO (zkCallback-15538-thread-2) [ ]
o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live
nodes size: [4])
[junit4] 2> 969062 INFO (zkCallback-15536-thread-2) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 969069 INFO (zkCallback-15534-thread-2) [ ]
o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live
nodes size: [3])
[junit4] 2> 969069 INFO (zkCallback-15536-thread-2) [ ]
o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live
nodes size: [3])
[junit4] 2> 969069 INFO (zkCallback-15534-thread-2) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 969070 INFO (zkCallback-15538-thread-2) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 969070 INFO (zkCallback-15536-thread-2) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 969075 INFO
(TEST-OverseerTest.testShardAssignmentBigger-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.ElectionContext Canceling election
/collections/collection1/leader_elect/shard4/election/96003448551309326-node10_core10-n_0000000002
[junit4] 2> 969075 INFO
(TEST-OverseerTest.testShardAssignmentBigger-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.ShardLeaderElectionContextBase Removing leader registration node on
cancel: /collections/collection1/leaders/shard4/leader 3
[junit4] 2> 969076 INFO (zkCallback-15536-thread-2) [ ]
o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live
nodes size: [2])
[junit4] 2> 969076 INFO (zkCallback-15536-thread-2) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 969076 INFO (zkCallback-15538-thread-3) [ ]
o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live
nodes size: [3])
[junit4] 2> 969076 INFO (zkCallback-15538-thread-3) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (1)
[junit4] 2> 969076 INFO
(TEST-OverseerTest.testShardAssignmentBigger-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.ElectionContext Canceling election
/collections/collection1/leader_elect/shard1/election/96003448551309327-node11_core11-n_0000000002
[junit4] 2> 969077 INFO
(TEST-OverseerTest.testShardAssignmentBigger-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.ShardLeaderElectionContextBase Removing leader registration node on
cancel: /collections/collection1/leaders/shard1/leader 3
[junit4] 2> 969078 INFO (zkCallback-15538-thread-3) [ ]
o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live
nodes size: [1])
[junit4] 2> 970578 WARN (zkCallback-15538-thread-3) [ ]
o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK:
[KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 970580 INFO
(TEST-OverseerTest.testShardAssignmentBigger-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.ZkTestServer connecting to 127.0.0.1:48550 48550
[junit4] 2> 970750 INFO (Thread-2709) [ ] o.a.s.c.ZkTestServer
connecting to 127.0.0.1:48550 48550
[junit4] 2> 970752 WARN (Thread-2709) [ ] o.a.s.c.ZkTestServer Watch
limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 14 /solr/aliases.json
[junit4] 2> 14 /solr/clusterprops.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 14 /solr/clusterstate.json
[junit4] 2> 2
/solr/collections/collection1/leader_elect/shard2/election/96003448551309320-node4_core4-n_0000000000
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 14 /solr/live_nodes
[junit4] 2> 14 /solr/collections
[junit4] 2>
[junit4] 2> 970753 INFO
(TEST-OverseerTest.testShardAssignmentBigger-seed#[CEC4632E0A269E2]) [ ]
o.a.s.SolrTestCaseJ4 ###Ending testShardAssignmentBigger
[junit4] 2> 970753 INFO
(TEST-OverseerTest.testShardAssignmentBigger-seed#[CEC4632E0A269E2]) [ ]
o.a.s.c.Overseer Overseer
(id=96003448551309315-127.0.0.1:48550_solr-n_0000000000) closing
[junit4] 2> 973756 INFO
(SUITE-OverseerTest-seed#[CEC4632E0A269E2]-worker) [ ] o.a.s.SolrTestCaseJ4
###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.OverseerTest_CEC4632E0A269E2-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene62): {},
docValues:{}, maxPointsInLeafNode=1824, maxMBSortInHeap=7.205488883585745,
sim=ClassicSimilarity, locale=ar-SA, timezone=Etc/GMT+12
[junit4] 2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation
1.8.0_74 (64-bit)/cpus=4,threads=1,free=124805312,total=523763712
[junit4] 2> NOTE: All tests run in this JVM: [TestFunctionQuery,
CdcrReplicationDistributedZkTest, OverseerCollectionConfigSetProcessorTest,
CloudExitableDirectoryReaderTest, TestLegacyFieldCache, PolyFieldTest,
SolrCoreTest, ResourceLoaderTest, SharedFSAutoReplicaFailoverTest,
TestSchemaNameResource, TestStressLiveNodes, SpatialRPTFieldTypeTest,
BlobRepositoryCloudTest, TestLMDirichletSimilarityFactory,
TestRequestStatusCollectionAPI, TestFieldResource, TestDistribDocBasedVersion,
TestHighlightDedupGrouping, CoreAdminCreateDiscoverTest, TestManagedSchema,
TestConfigSetProperties, DistributedDebugComponentTest, TestStandardQParsers,
LeaderInitiatedRecoveryOnCommitTest, TestDistribIDF, TestDocSet,
TestIBSimilarityFactory, BlockJoinFacetDistribTest,
CreateCollectionCleanupTest, TestFoldingMultitermQuery,
AsyncCallRequestStatusResponseTest, ReplicationFactorTest,
TestUninvertingReader, TestRTGBase, TestObjectReleaseTracker,
TestAnalyzeInfixSuggestions, HdfsSyncSliceTest, IndexSchemaRuntimeFieldTest,
HdfsBasicDistributedZk2Test, ReturnFieldsTest, DistribJoinFromCollectionTest,
TestConfig, TlogReplayBufferedWhileIndexingTest,
AddSchemaFieldsUpdateProcessorFactoryTest, OutputWriterTest,
SuggestComponentContextFilterQueryTest, TestCSVLoader, BasicDistributedZkTest,
BitVectorTest, RankQueryTest, CloneFieldUpdateProcessorFactoryTest,
TestMiniSolrCloudCluster, TestSchemaVersionResource, ExternalFileFieldSortTest,
SyncSliceTest, TestOmitPositions, IndexSchemaTest,
TestReplicationHandlerBackup, TestSQLHandler, ResponseLogComponentTest,
HdfsLockFactoryTest, HdfsRecoverLeaseTest, SolrRequestParserTest,
FileBasedSpellCheckerTest, TestGraphTermsQParserPlugin,
DeleteLastCustomShardedReplicaTest, RequestLoggingTest, TestOrdValues,
BasicZkTest, CursorPagingTest, TestSha256AuthenticationProvider,
QueryElevationComponentTest, SystemInfoHandlerTest,
ConcurrentDeleteAndCreateCollectionTest, TestLRUStatsCache,
TestSubQueryTransformerCrossCore, TestManagedSynonymFilterFactory,
DistributedFacetPivotLongTailTest, CacheHeaderTest, TestDistributedSearch,
SSLMigrationTest, SmileWriterTest, TestClassicSimilarityFactory,
TestXmlQParser, ParsingFieldUpdateProcessorsTest,
PathHierarchyTokenizerFactoryTest, SynonymTokenizerTest,
TestSizeLimitedDistributedMap, InfoHandlerTest, ZkNodePropsTest,
TestRemoteStreaming, JSONWriterTest, TestRealTimeGet, CopyFieldTest,
TestClusterStateMutator, CurrencyFieldOpenExchangeTest, JsonLoaderTest,
MultiThreadedOCPTest, StandardRequestHandlerTest,
TestPerFieldSimilarityWithDefaultOverride, TestFieldCacheSortRandom,
BasicDistributedZk2Test, OpenCloseCoreStressTest, OverseerTest]
[junit4] Completed [225/612 (1!)] on J0 in 51.60s, 13 tests, 1 failure, 1
skipped <<< FAILURES!
[...truncated 1241 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/build.xml:740: The
following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/build.xml:684: The
following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/build.xml:59: The
following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build.xml:233:
The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/common-build.xml:531:
The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/common-build.xml:1427:
The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/common-build.xml:984:
There were test failures: 612 suites (10 ignored), 2560 tests, 1 failure, 87
ignored (72 assumptions) [seed: CEC4632E0A269E2]
Total time: 66 minutes 41 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
