Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-Java8/345/
1 tests failed.
REGRESSION:
org.apache.solr.cloud.OverseerTest.testExternalClusterStateChangeBehavior
Error Message:
Illegal state, was: down expected:active clusterState:live
nodes:[]collections:{c1=DocCollection(c1)={ "shards":{"shard1":{
"parent":null, "range":null, "state":"active",
"replicas":{"core_node1":{ "base_url":"http://127.0.0.1/solr";,
"node_name":"node1", "core":"core1", "roles":"",
"state":"down"}}}}, "router":{"name":"implicit"}},
test=LazyCollectionRef(test)}
Stack Trace:
java.lang.AssertionError: Illegal state, was: down expected:active
clusterState:live nodes:[]collections:{c1=DocCollection(c1)={
"shards":{"shard1":{
"parent":null,
"range":null,
"state":"active",
"replicas":{"core_node1":{
"base_url":"http://127.0.0.1/solr";,
"node_name":"node1",
"core":"core1",
"roles":"",
"state":"down"}}}},
"router":{"name":"implicit"}}, test=LazyCollectionRef(test)}
at
__randomizedtesting.SeedInfo.seed([7B71D6EA298C7E2A:136FD506CB1C2464]:0)
at org.junit.Assert.fail(Assert.java:93)
at
org.apache.solr.cloud.OverseerTest.verifyStatus(OverseerTest.java:601)
at
org.apache.solr.cloud.OverseerTest.testExternalClusterStateChangeBehavior(OverseerTest.java:1261)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 9423 lines...]
[junit4] Suite: org.apache.solr.cloud.OverseerTest
[junit4] 2> Creating dataDir:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.OverseerTest_7B71D6EA298C7E2A-001/init-core-data-001
[junit4] 2> 257381 INFO
(SUITE-OverseerTest-seed#[7B71D6EA298C7E2A]-worker) [ ] o.a.s.SolrTestCaseJ4
Randomized ssl (true) and clientAuth (false)
[junit4] 2> 257381 INFO
(SUITE-OverseerTest-seed#[7B71D6EA298C7E2A]-worker) [ ] o.a.s.SolrTestCaseJ4
####initCore
[junit4] 2> 257381 INFO
(SUITE-OverseerTest-seed#[7B71D6EA298C7E2A]-worker) [ ] o.a.s.SolrTestCaseJ4
####initCore end
[junit4] 2> 257382 INFO
(TEST-OverseerTest.testOverseerStatsReset-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.SolrTestCaseJ4 ###Starting testOverseerStatsReset
[junit4] 2> 257383 INFO
(TEST-OverseerTest.testOverseerStatsReset-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 257383 INFO (Thread-635) [ ] o.a.s.c.ZkTestServer client
port:0.0.0.0/0.0.0.0:0
[junit4] 2> 257383 INFO (Thread-635) [ ] o.a.s.c.ZkTestServer Starting
server
[junit4] 2> 257483 INFO
(TEST-OverseerTest.testOverseerStatsReset-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.ZkTestServer start zk server on port:41768
[junit4] 2> 257484 INFO
(TEST-OverseerTest.testOverseerStatsReset-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 257484 INFO
(TEST-OverseerTest.testOverseerStatsReset-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 257487 INFO (zkCallback-124-thread-1) [ ]
o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@5e6f6281
name:ZooKeeperConnection Watcher:127.0.0.1:41768 got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 257487 INFO
(TEST-OverseerTest.testOverseerStatsReset-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 257487 INFO
(TEST-OverseerTest.testOverseerStatsReset-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 257489 INFO
(TEST-OverseerTest.testOverseerStatsReset-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 257489 INFO
(TEST-OverseerTest.testOverseerStatsReset-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 257491 INFO (zkCallback-125-thread-1) [ ]
o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@2df5f3db
name:ZooKeeperConnection Watcher:127.0.0.1:41768 got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 257491 INFO
(TEST-OverseerTest.testOverseerStatsReset-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 257491 INFO
(TEST-OverseerTest.testOverseerStatsReset-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 257491 INFO
(TEST-OverseerTest.testOverseerStatsReset-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient makePath: /solr
[junit4] 2> 257494 INFO
(TEST-OverseerTest.testOverseerStatsReset-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 257494 INFO
(TEST-OverseerTest.testOverseerStatsReset-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 257500 INFO (zkCallback-126-thread-1) [ ]
o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@40d80f3c
name:ZooKeeperConnection Watcher:127.0.0.1:41768/solr got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 257500 INFO
(TEST-OverseerTest.testOverseerStatsReset-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 257500 INFO
(TEST-OverseerTest.testOverseerStatsReset-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 257501 INFO
(TEST-OverseerTest.testOverseerStatsReset-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient makePath: /live_nodes
[junit4] 2> 257502 INFO
(TEST-OverseerTest.testOverseerStatsReset-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient makePath: /collections
[junit4] 2> 257507 INFO
(TEST-OverseerTest.testOverseerStatsReset-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient makePath: /aliases.json
[junit4] 2> 257508 INFO
(TEST-OverseerTest.testOverseerStatsReset-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
[junit4] 2> 257509 INFO
(TEST-OverseerTest.testOverseerStatsReset-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient makePath: /security.json
[junit4] 2> 257510 INFO
(TEST-OverseerTest.testOverseerStatsReset-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 257512 INFO
(TEST-OverseerTest.testOverseerStatsReset-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 257519 INFO
(TEST-OverseerTest.testOverseerStatsReset-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 257521 INFO (zkCallback-127-thread-1) [ ]
o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@872030a name:ZooKeeperConnection
Watcher:127.0.0.1:41768/solr got event WatchedEvent state:SyncConnected
type:None path:null path:null type:None
[junit4] 2> 257521 INFO
(TEST-OverseerTest.testOverseerStatsReset-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 257521 INFO
(TEST-OverseerTest.testOverseerStatsReset-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 257523 INFO
(TEST-OverseerTest.testOverseerStatsReset-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 257526 INFO
(TEST-OverseerTest.testOverseerStatsReset-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient makePath: /live_nodes/node1
[junit4] 2> 257527 INFO
(TEST-OverseerTest.testOverseerStatsReset-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params:
socketTimeout=600000&connTimeout=60000&retry=true
[junit4] 2> 257529 INFO (zkCallback-127-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 0)
[junit4] 2> 257530 INFO (zkCallback-126-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 0)
[junit4] 2> 257531 INFO
(TEST-OverseerTest.testOverseerStatsReset-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient makePath: /overseer_elect
[junit4] 2> 257533 INFO
(TEST-OverseerTest.testOverseerStatsReset-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
[junit4] 2> 257534 INFO
(TEST-OverseerTest.testOverseerStatsReset-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 257535 INFO
(TEST-OverseerTest.testOverseerStatsReset-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.LeaderElector Joined leadership election with path:
/overseer_elect/election/94440548181016578-127.0.0.1:41768_solr-n_0000000000
[junit4] 2> 257536 INFO
(TEST-OverseerTest.testOverseerStatsReset-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:41768_solr
[junit4] 2> 257536 INFO
(TEST-OverseerTest.testOverseerStatsReset-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
[junit4] 2> 257537 INFO
(TEST-OverseerTest.testOverseerStatsReset-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.Overseer Overseer
(id=94440548181016578-127.0.0.1:41768_solr-n_0000000000) starting
[junit4] 2> 257540 INFO
(TEST-OverseerTest.testOverseerStatsReset-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient makePath: /overseer/queue
[junit4] 2> 257541 INFO
(TEST-OverseerTest.testOverseerStatsReset-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
[junit4] 2> 257543 INFO
(TEST-OverseerTest.testOverseerStatsReset-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
[junit4] 2> 257544 INFO
(TEST-OverseerTest.testOverseerStatsReset-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
[junit4] 2> 257546 INFO
(TEST-OverseerTest.testOverseerStatsReset-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
[junit4] 2> 257547 INFO
(TEST-OverseerTest.testOverseerStatsReset-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
[junit4] 2> 257550 INFO
(TEST-OverseerTest.testOverseerStatsReset-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.OverseerAutoReplicaFailoverThread Starting
OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000
autoReplicaFailoverWaitAfterExpiration=30000
autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 257553 INFO
(OverseerCollectionConfigSetProcessor-94440548181016578-127.0.0.1:41768_solr-n_0000000000)
[ ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer
operations
[junit4] 2> 257554 INFO
(OverseerStateUpdate-94440548181016578-127.0.0.1:41768_solr-n_0000000000) [
] o.a.s.c.Overseer Starting to work on the main queue
[junit4] 2> 257562 INFO
(OverseerStateUpdate-94440548181016578-127.0.0.1:41768_solr-n_0000000000) [
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"} current state version: 0
[junit4] 2> 257563 INFO
(OverseerStateUpdate-94440548181016578-127.0.0.1:41768_solr-n_0000000000) [
] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 257563 INFO
(OverseerStateUpdate-94440548181016578-127.0.0.1:41768_solr-n_0000000000) [
] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
[junit4] 2> 257563 INFO
(OverseerStateUpdate-94440548181016578-127.0.0.1:41768_solr-n_0000000000) [
] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 257564 INFO (zkCallback-126-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 1)
[junit4] 2> 257564 INFO (zkCallback-127-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 1)
[junit4] 2> 258062 INFO
(TEST-OverseerTest.testOverseerStatsReset-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient makePath:
/collections/collection1/leader_elect/shard1/election
[junit4] 2> 258064 INFO (zkCallback-126-thread-1) [ ]
o.a.s.c.c.ZkStateReader A collections change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/collections, has occurred - updating...
[junit4] 2> 258064 INFO (zkCallback-127-thread-1) [ ]
o.a.s.c.c.ZkStateReader A collections change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/collections, has occurred - updating...
[junit4] 2> 258067 INFO
(TEST-OverseerTest.testOverseerStatsReset-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.LeaderElector Joined leadership election with path:
/collections/collection1/leader_elect/shard1/election/94440548181016579-node1_core1-n_0000000000
[junit4] 2> 258068 INFO
(TEST-OverseerTest.testOverseerStatsReset-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient makePath: /collections/collection1/leaders/shard1
[junit4] 2> 258071 INFO
(TEST-OverseerTest.testOverseerStatsReset-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.Overseer Overseer
(id=94440548181016578-127.0.0.1:41768_solr-n_0000000000) closing
[junit4] 2> 258071 INFO
(TEST-OverseerTest.testOverseerStatsReset-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.ElectionContext canceling election
/overseer_elect/election/94440548181016578-127.0.0.1:41768_solr-n_0000000000
[junit4] 2> 258071 INFO
(OverseerStateUpdate-94440548181016578-127.0.0.1:41768_solr-n_0000000000) [
] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:41768_solr
[junit4] 2> 258073 INFO (OverseerExitThread) [ ] o.a.s.c.Overseer I'm
exiting , but I'm still the leader
[junit4] 2> 258073 INFO
(TEST-OverseerTest.testOverseerStatsReset-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.LeaderElector Joined leadership election with path:
/overseer_elect/election/94440548181016578-127.0.0.1:41768_solr-n_0000000001
[junit4] 2> 258074 INFO
(TEST-OverseerTest.testOverseerStatsReset-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:41768_solr
[junit4] 2> 258075 INFO
(TEST-OverseerTest.testOverseerStatsReset-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
[junit4] 2> 258076 INFO
(TEST-OverseerTest.testOverseerStatsReset-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.Overseer Overseer
(id=94440548181016578-127.0.0.1:41768_solr-n_0000000001) starting
[junit4] 2> 258085 INFO
(TEST-OverseerTest.testOverseerStatsReset-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.OverseerAutoReplicaFailoverThread Starting
OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000
autoReplicaFailoverWaitAfterExpiration=30000
autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 258085 INFO
(OverseerCollectionConfigSetProcessor-94440548181016578-127.0.0.1:41768_solr-n_0000000001)
[ ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer
operations
[junit4] 2> 258086 INFO
(OverseerStateUpdate-94440548181016578-127.0.0.1:41768_solr-n_0000000001) [
] o.a.s.c.Overseer Starting to work on the main queue
[junit4] 2> 258086 INFO (zkCallback-127-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 1)
[junit4] 2> 258086 INFO (zkCallback-126-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 1)
[junit4] 2> 258089 INFO
(TEST-OverseerTest.testOverseerStatsReset-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.ZkTestServer connecting to 127.0.0.1:41768 41768
[junit4] 2> 258360 INFO (Thread-635) [ ] o.a.s.c.ZkTestServer
connecting to 127.0.0.1:41768 41768
[junit4] 2> 258361 WARN (Thread-635) [ ] o.a.s.c.ZkTestServer Watch
limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 2 /solr/aliases.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 2 /solr/clusterstate.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 2 /solr/live_nodes
[junit4] 2> 2 /solr/collections
[junit4] 2>
[junit4] 2> 258362 INFO
(TEST-OverseerTest.testOverseerStatsReset-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.SolrTestCaseJ4 ###Ending testOverseerStatsReset
[junit4] 2> 258362 INFO
(TEST-OverseerTest.testOverseerStatsReset-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.Overseer Overseer
(id=94440548181016578-127.0.0.1:41768_solr-n_0000000001) closing
[junit4] 2> 258362 INFO
(OverseerStateUpdate-94440548181016578-127.0.0.1:41768_solr-n_0000000001) [
] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:41768_solr
[junit4] 2> 258371 ERROR (OverseerExitThread) [ ] o.a.s.c.Overseer
could not read the data
[junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException:
KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at
org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:353)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:350)
[junit4] 2> at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:350)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:304)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:87)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater$2.run(Overseer.java:265)
[junit4] 2> 258371 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.SolrTestCaseJ4 ###Starting testShardLeaderChange
[junit4] 2> 258372 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 258372 INFO (Thread-636) [ ] o.a.s.c.ZkTestServer client
port:0.0.0.0/0.0.0.0:0
[junit4] 2> 258372 INFO (Thread-636) [ ] o.a.s.c.ZkTestServer Starting
server
[junit4] 2> 258472 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.ZkTestServer start zk server on port:52175
[junit4] 2> 258472 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 258473 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 258476 INFO (zkCallback-129-thread-1) [ ]
o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@51fe82f3
name:ZooKeeperConnection Watcher:127.0.0.1:52175/solr got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 258476 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 258476 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 258476 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 258477 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 258478 INFO (zkCallback-130-thread-1) [ ]
o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@ea65736 name:ZooKeeperConnection
Watcher:127.0.0.1:52175 got event WatchedEvent state:SyncConnected type:None
path:null path:null type:None
[junit4] 2> 258478 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 258478 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 258480 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 258481 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 258483 INFO (zkCallback-131-thread-1) [ ]
o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@64910f6c
name:ZooKeeperConnection Watcher:127.0.0.1:52175 got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 258484 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 258484 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 258484 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient makePath: /solr
[junit4] 2> 258495 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient makePath: /live_nodes
[junit4] 2> 258497 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient makePath: /collections
[junit4] 2> 258498 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient makePath: /aliases.json
[junit4] 2> 258499 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
[junit4] 2> 258500 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient makePath: /security.json
[junit4] 2> 258502 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 258502 INFO (Thread-637) [ ] o.a.s.c.c.SolrZkClient Using
default ZkCredentialsProvider
[junit4] 2> 258503 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 258503 INFO (Thread-637) [ ] o.a.s.c.c.ConnectionManager
Waiting for client to connect to ZooKeeper
[junit4] 2> 258505 INFO (zkCallback-133-thread-1) [ ]
o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@546cbe8e
name:ZooKeeperConnection Watcher:127.0.0.1:52175/solr got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 258505 INFO (Thread-637) [ ] o.a.s.c.c.ConnectionManager
Client is connected to ZooKeeper
[junit4] 2> 258506 INFO (Thread-637) [ ] o.a.s.c.c.SolrZkClient Using
default ZkACLProvider
[junit4] 2> 258506 INFO (Thread-637) [ ] o.a.s.u.UpdateShardHandler
Creating UpdateShardHandler HTTP client with params:
socketTimeout=600000&connTimeout=60000&retry=true
[junit4] 2> 258519 INFO (zkCallback-132-thread-1) [ ]
o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@3b4f01b7
name:ZooKeeperConnection Watcher:127.0.0.1:52175/solr got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 258519 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 258520 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 258520 INFO (Thread-637) [ ] o.a.s.c.c.SolrZkClient
makePath: /overseer_elect
[junit4] 2> 258522 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 258522 INFO (Thread-637) [ ] o.a.s.c.c.SolrZkClient
makePath: /overseer_elect/election
[junit4] 2> 258523 INFO (Thread-637) [ ] o.a.s.c.Overseer Overseer
(id=null) closing
[junit4] 2> 258523 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient makePath: /live_nodes/node1
[junit4] 2> 258523 INFO (Thread-637) [ ] o.a.s.c.LeaderElector Joined
leadership election with path:
/overseer_elect/election/94440548245831683-127.0.0.1:52175_solr-n_0000000000
[junit4] 2> 258524 INFO (zkCallback-132-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 0)
[junit4] 2> 258524 INFO (Thread-637) [ ]
o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:52175_solr
[junit4] 2> 258524 INFO (Thread-637) [ ] o.a.s.c.c.SolrZkClient
makePath: /overseer_elect/leader
[junit4] 2> 258526 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient makePath: /overseer/queue
[junit4] 2> 258526 INFO (Thread-637) [ ] o.a.s.c.Overseer Overseer
(id=94440548245831683-127.0.0.1:52175_solr-n_0000000000) starting
[junit4] 2> 258528 INFO (Thread-637) [ ] o.a.s.c.c.SolrZkClient
makePath: /overseer/queue-work
[junit4] 2> 258529 INFO (Thread-637) [ ] o.a.s.c.c.SolrZkClient
makePath: /overseer/collection-map-failure
[junit4] 2> 258531 INFO (Thread-637) [ ] o.a.s.c.c.SolrZkClient
makePath: /overseer/collection-map-running
[junit4] 2> 258533 INFO (Thread-637) [ ] o.a.s.c.c.SolrZkClient
makePath: /overseer/collection-map-completed
[junit4] 2> 258542 INFO (Thread-637) [ ] o.a.s.c.c.SolrZkClient
makePath: /overseer/collection-queue-work
[junit4] 2> 258546 INFO (Thread-637) [ ]
o.a.s.c.OverseerAutoReplicaFailoverThread Starting
OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000
autoReplicaFailoverWaitAfterExpiration=30000
autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 258546 INFO
(OverseerCollectionConfigSetProcessor-94440548245831683-127.0.0.1:52175_solr-n_0000000000)
[ ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer
operations
[junit4] 2> 258546 INFO (Thread-637) [ ] o.a.s.SolrTestCaseJ4 Killing
overseer.
[junit4] 2> 258547 INFO
(OverseerStateUpdate-94440548245831683-127.0.0.1:52175_solr-n_0000000000) [
] o.a.s.c.Overseer Starting to work on the main queue
[junit4] 2> 258547 INFO
(OverseerStateUpdate-94440548245831683-127.0.0.1:52175_solr-n_0000000000) [
] o.a.s.c.Overseer
[junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException:
KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at
org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:353)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:350)
[junit4] 2> at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:350)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater.amILeader(Overseer.java:411)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:143)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 258547 ERROR
(OverseerCollectionConfigSetProcessor-94440548245831683-127.0.0.1:52175_solr-n_0000000000)
[ ] o.a.s.c.OverseerTaskProcessor
:org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode
= Session expired for /overseer/collection-queue-work
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at
org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:336)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:333)
[junit4] 2> at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:333)
[junit4] 2> at
org.apache.solr.cloud.DistributedQueue.fetchZkChildren(DistributedQueue.java:310)
[junit4] 2> at
org.apache.solr.cloud.OverseerTaskQueue.getTailId(OverseerTaskQueue.java:257)
[junit4] 2> at
org.apache.solr.cloud.OverseerTaskProcessor.run(OverseerTaskProcessor.java:144)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 258547 INFO
(OverseerStateUpdate-94440548245831683-127.0.0.1:52175_solr-n_0000000000) [
] o.a.s.c.Overseer According to ZK I
(id=94440548245831683-127.0.0.1:52175_solr-n_0000000000) am no longer a leader.
[junit4] 2> 258548 ERROR
(OverseerCollectionConfigSetProcessor-94440548245831683-127.0.0.1:52175_solr-n_0000000000)
[ ] o.a.s.c.OverseerTaskProcessor Unable to prioritize overseer
[junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException:
KeeperErrorCode = Session expired for /roles.json
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at
org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1045)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:319)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:316)
[junit4] 2> at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:316)
[junit4] 2> at
org.apache.solr.cloud.OverseerNodePrioritizer.prioritizeOverseerNodes(OverseerNodePrioritizer.java:59)
[junit4] 2> at
org.apache.solr.cloud.OverseerTaskProcessor.run(OverseerTaskProcessor.java:159)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 258548 INFO (Thread-637) [ ] o.a.s.c.c.SolrZkClient Using
default ZkCredentialsProvider
[junit4] 2> 258548 INFO
(OverseerCollectionConfigSetProcessor-94440548245831683-127.0.0.1:52175_solr-n_0000000000)
[ ] o.a.s.c.OverseerTaskProcessor
[junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException:
KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at
org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:353)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:350)
[junit4] 2> at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:350)
[junit4] 2> at
org.apache.solr.cloud.OverseerTaskProcessor.amILeader(OverseerTaskProcessor.java:355)
[junit4] 2> at
org.apache.solr.cloud.OverseerTaskProcessor.run(OverseerTaskProcessor.java:172)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 258548 INFO
(OverseerStateUpdate-94440548245831683-127.0.0.1:52175_solr-n_0000000000) [
] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:52175_solr
[junit4] 2> 258549 INFO
(OverseerCollectionConfigSetProcessor-94440548245831683-127.0.0.1:52175_solr-n_0000000000)
[ ] o.a.s.c.OverseerTaskProcessor According to ZK I
(id=94440548245831683-127.0.0.1:52175_solr-n_0000000000) am no longer a leader.
[junit4] 2> 258549 INFO (Thread-637) [ ] o.a.s.c.c.ConnectionManager
Waiting for client to connect to ZooKeeper
[junit4] 2> 258557 ERROR (OverseerExitThread) [ ] o.a.s.c.Overseer
could not read the data
[junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException:
KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at
org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:353)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:350)
[junit4] 2> at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:350)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:304)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:87)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater$2.run(Overseer.java:265)
[junit4] 2> 258559 INFO (zkCallback-135-thread-1) [ ]
o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@6d73e4bf
name:ZooKeeperConnection Watcher:127.0.0.1:52175/solr got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 258561 INFO (Thread-637) [ ] o.a.s.c.c.ConnectionManager
Client is connected to ZooKeeper
[junit4] 2> 258561 INFO (Thread-637) [ ] o.a.s.c.c.SolrZkClient Using
default ZkACLProvider
[junit4] 2> 258561 INFO (Thread-637) [ ] o.a.s.c.Overseer Overseer
(id=94440548245831683-127.0.0.1:52175_solr-n_0000000000) closing
[junit4] 2> 258561 INFO (Thread-637) [ ] o.a.s.u.UpdateShardHandler
Creating UpdateShardHandler HTTP client with params:
socketTimeout=600000&connTimeout=60000&retry=true
[junit4] 2> 258562 ERROR
(OverseerHdfsCoreFailoverThread-94440548245831683-127.0.0.1:52175_solr-n_0000000000)
[ ] o.a.s.c.OverseerAutoReplicaFailoverThread
OverseerAutoReplicaFailoverThread had an error in its thread work
loop.:org.apache.solr.common.SolrException: Error reading cluster properties
[junit4] 2> at
org.apache.solr.common.cloud.ZkStateReader.getClusterProps(ZkStateReader.java:731)
[junit4] 2> at
org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.doWork(OverseerAutoReplicaFailoverThread.java:152)
[junit4] 2> at
org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:131)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> Caused by: java.lang.InterruptedException: sleep interrupted
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at
org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:108)
[junit4] 2> at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:76)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:316)
[junit4] 2> at
org.apache.solr.common.cloud.ZkStateReader.getClusterProps(ZkStateReader.java:724)
[junit4] 2> ... 3 more
[junit4] 2>
[junit4] 2> 258568 INFO (Thread-637) [ ] o.a.s.c.Overseer Overseer
(id=null) closing
[junit4] 2> 258568 INFO (Thread-637) [ ] o.a.s.c.LeaderElector Joined
leadership election with path:
/overseer_elect/election/94440548245831685-127.0.0.1:52175_solr-n_0000000001
[junit4] 2> 258569 INFO (Thread-637) [ ]
o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:52175_solr
[junit4] 2> 258569 INFO (Thread-637) [ ] o.a.s.c.c.SolrZkClient
makePath: /overseer_elect/leader
[junit4] 2> 258570 INFO (Thread-637) [ ] o.a.s.c.Overseer Overseer
(id=94440548245831685-127.0.0.1:52175_solr-n_0000000001) starting
[junit4] 2> 258575 INFO (Thread-637) [ ]
o.a.s.c.OverseerAutoReplicaFailoverThread Starting
OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000
autoReplicaFailoverWaitAfterExpiration=30000
autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 258576 INFO
(OverseerCollectionConfigSetProcessor-94440548245831685-127.0.0.1:52175_solr-n_0000000001)
[ ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer
operations
[junit4] 2> 258576 INFO
(OverseerStateUpdate-94440548245831685-127.0.0.1:52175_solr-n_0000000001) [
] o.a.s.c.Overseer Starting to work on the main queue
[junit4] 2> 258577 INFO
(OverseerStateUpdate-94440548245831685-127.0.0.1:52175_solr-n_0000000001) [
] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 258579 INFO
(OverseerStateUpdate-94440548245831685-127.0.0.1:52175_solr-n_0000000001) [
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"} current state version: 0
[junit4] 2> 258580 INFO
(OverseerStateUpdate-94440548245831685-127.0.0.1:52175_solr-n_0000000001) [
] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 258580 INFO
(OverseerStateUpdate-94440548245831685-127.0.0.1:52175_solr-n_0000000001) [
] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
[junit4] 2> 258580 INFO
(OverseerStateUpdate-94440548245831685-127.0.0.1:52175_solr-n_0000000001) [
] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 258581 INFO (zkCallback-135-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 1)
[junit4] 2> 258581 INFO (zkCallback-132-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 1)
[junit4] 2> 259027 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient makePath:
/collections/collection1/leader_elect/shard1/election
[junit4] 2> 259029 INFO (zkCallback-132-thread-1) [ ]
o.a.s.c.c.ZkStateReader A collections change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/collections, has occurred - updating...
[junit4] 2> 259029 INFO (zkCallback-135-thread-1) [ ]
o.a.s.c.c.ZkStateReader A collections change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/collections, has occurred - updating...
[junit4] 2> 259032 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.LeaderElector Joined leadership election with path:
/collections/collection1/leader_elect/shard1/election/94440548245831684-node1_core1-n_0000000000
[junit4] 2> 259033 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient makePath: /collections/collection1/leaders/shard1
[junit4] 2> 259037 INFO
(OverseerStateUpdate-94440548245831685-127.0.0.1:52175_solr-n_0000000001) [
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "base_url":"http://node1/solr/";,
[junit4] 2> "core":"core1",
[junit4] 2> "state":"active"} current state version: 1
[junit4] 2> 259037 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient makePath:
/collections/collection1/leader_elect/shard1/election
[junit4] 2> 259040 INFO
(OverseerStateUpdate-94440548245831685-127.0.0.1:52175_solr-n_0000000001) [
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"} current state version: 1
[junit4] 2> 259040 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.LeaderElector Joined leadership election with path:
/collections/collection1/leader_elect/shard1/election/94440548245831684-node1_core1-n_0000000001
[junit4] 2> 259041 INFO
(OverseerStateUpdate-94440548245831685-127.0.0.1:52175_solr-n_0000000001) [
] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 259041 INFO
(OverseerStateUpdate-94440548245831685-127.0.0.1:52175_solr-n_0000000001) [
] o.a.s.c.o.ReplicaMutator shard=shard1 is already registered
[junit4] 2> 259041 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.LeaderElector was going to be leader
/collections/collection1/leader_elect/shard1/election/94440548245831684-node1_core1-n_0000000001
, seq(0)
/collections/collection1/leader_elect/shard1/election/94440548245831684-node1_core1-n_0000000000
[junit4] 2> 259041 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 259042 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 259043 INFO (zkCallback-137-thread-1) [ ]
o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@4b91f184
name:ZooKeeperConnection Watcher:127.0.0.1:52175/solr got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 259044 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 259044 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 259046 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 259047 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient makePath: /live_nodes/node2
[junit4] 2> 259048 INFO (zkCallback-132-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 1)
[junit4] 2> 259048 INFO (zkCallback-135-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 1)
[junit4] 2> 259048 INFO (zkCallback-137-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 1)
[junit4] 2> 259050 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient makePath:
/collections/collection1/leader_elect/shard1/election
[junit4] 2> 259051 INFO
(OverseerStateUpdate-94440548245831685-127.0.0.1:52175_solr-n_0000000001) [
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"} current state version: 1
[junit4] 2> 259053 INFO
(OverseerStateUpdate-94440548245831685-127.0.0.1:52175_solr-n_0000000001) [
] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 259053 INFO
(OverseerStateUpdate-94440548245831685-127.0.0.1:52175_solr-n_0000000001) [
] o.a.s.c.o.ReplicaMutator shard=shard1 is already registered
[junit4] 2> 259054 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.LeaderElector Joined leadership election with path:
/collections/collection1/leader_elect/shard1/election/94440548245831684-node1_core1-n_0000000002
[junit4] 2> 259055 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.LeaderElector was going to be leader
/collections/collection1/leader_elect/shard1/election/94440548245831684-node1_core1-n_0000000002
, seq(0)
/collections/collection1/leader_elect/shard1/election/94440548245831684-node1_core1-n_0000000000
[junit4] 2> 259055 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 259155 INFO (zkCallback-137-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 2)
[junit4] 2> 259156 INFO (zkCallback-135-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 2)
[junit4] 2> 259155 INFO (zkCallback-129-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 2)
[junit4] 2> 259155 INFO (zkCallback-132-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 2)
[junit4] 2> 259260 INFO
(OverseerStateUpdate-94440548245831685-127.0.0.1:52175_solr-n_0000000001) [
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node2",
[junit4] 2> "core":"core4",
[junit4] 2> "core_node_name":"node2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node2/solr/"} current state version: 2
[junit4] 2> 259261 INFO
(OverseerStateUpdate-94440548245831685-127.0.0.1:52175_solr-n_0000000001) [
] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node2",
[junit4] 2> "core":"core4",
[junit4] 2> "core_node_name":"node2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node2/solr/"}
[junit4] 2> 259261 INFO
(OverseerStateUpdate-94440548245831685-127.0.0.1:52175_solr-n_0000000001) [
] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=1
[junit4] 2> 259261 INFO
(OverseerStateUpdate-94440548245831685-127.0.0.1:52175_solr-n_0000000001) [
] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 259364 INFO (zkCallback-132-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 2)
[junit4] 2> 259364 INFO (zkCallback-129-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 2)
[junit4] 2> 259364 INFO (zkCallback-137-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 2)
[junit4] 2> 259365 INFO (zkCallback-135-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 2)
[junit4] 2> 259760 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient makePath:
/collections/collection1/leader_elect/shard1/election
[junit4] 2> 259763 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.LeaderElector Joined leadership election with path:
/collections/collection1/leader_elect/shard1/election/94440548245831686-node2_core4-n_0000000003
[junit4] 2> 259764 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.LeaderElector Watching path
/collections/collection1/leader_elect/shard1/election/94440548245831684-node1_core1-n_0000000002
to know if I could be the leader
[junit4] 2> 259765 INFO (zkCallback-129-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 2)
[junit4] 2> 259765 INFO (zkCallback-135-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 2)
[junit4] 2> 259765 INFO (zkCallback-132-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 2)
[junit4] 2> 259765 INFO (zkCallback-137-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 2)
[junit4] 2> 259767 INFO (zkCallback-137-thread-2) [ ]
o.a.s.c.c.SolrZkClient makePath: /collections/collection1/leaders/shard1
[junit4] 2> 259770 INFO
(OverseerStateUpdate-94440548245831685-127.0.0.1:52175_solr-n_0000000001) [
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "base_url":"http://node2/solr/";,
[junit4] 2> "core":"core4",
[junit4] 2> "state":"active"} current state version: 3
[junit4] 2> 259872 INFO (zkCallback-137-thread-2) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 1)
[junit4] 2> 259872 INFO (zkCallback-129-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 1)
[junit4] 2> 259872 INFO (zkCallback-135-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 1)
[junit4] 2> 261265 WARN (zkCallback-132-thread-1) [ ]
o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 261267 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 261268 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 261272 INFO (zkCallback-138-thread-1) [ ]
o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@58a8a476
name:ZooKeeperConnection Watcher:127.0.0.1:52175/solr got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 261272 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 261272 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 261274 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 261275 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient makePath: /live_nodes/node1
[junit4] 2> 261277 INFO (zkCallback-129-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 1)
[junit4] 2> 261278 INFO (zkCallback-135-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 1)
[junit4] 2> 261278 INFO (zkCallback-137-thread-2) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 1)
[junit4] 2> 261279 INFO (zkCallback-138-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 1)
[junit4] 2> 261280 INFO (Thread-637) [ ] o.a.s.SolrTestCaseJ4 Killing
overseer.
[junit4] 2> 261281 INFO (Thread-637) [ ] o.a.s.c.c.SolrZkClient Using
default ZkCredentialsProvider
[junit4] 2> 261285 INFO (Thread-637) [ ] o.a.s.c.c.ConnectionManager
Waiting for client to connect to ZooKeeper
[junit4] 2> 261285 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient makePath:
/collections/collection1/leader_elect/shard1/election
[junit4] 2> 261289 INFO (zkCallback-139-thread-1) [ ]
o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@137c14c6
name:ZooKeeperConnection Watcher:127.0.0.1:52175/solr got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 261289 INFO (Thread-637) [ ] o.a.s.c.c.ConnectionManager
Client is connected to ZooKeeper
[junit4] 2> 261289 INFO (Thread-637) [ ] o.a.s.c.c.SolrZkClient Using
default ZkACLProvider
[junit4] 2> 261289 INFO (Thread-637) [ ] o.a.s.c.Overseer Overseer
(id=94440548245831685-127.0.0.1:52175_solr-n_0000000001) closing
[junit4] 2> 261290 INFO
(OverseerStateUpdate-94440548245831685-127.0.0.1:52175_solr-n_0000000001) [
] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:52175_solr
[junit4] 2> 261290 INFO (Thread-637) [ ] o.a.s.u.UpdateShardHandler
Creating UpdateShardHandler HTTP client with params:
socketTimeout=600000&connTimeout=60000&retry=true
[junit4] 2> 261290 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.LeaderElector Joined leadership election with path:
/collections/collection1/leader_elect/shard1/election/94440548245831687-node1_core1-n_0000000004
[junit4] 2> 261299 ERROR (OverseerExitThread) [ ] o.a.s.c.Overseer
could not read the data
[junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException:
KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at
org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:353)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:350)
[junit4] 2> at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:350)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:304)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:87)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater$2.run(Overseer.java:265)
[junit4] 2> 261303 INFO (Thread-637) [ ] o.a.s.c.Overseer Overseer
(id=null) closing
[junit4] 2> 261303 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.LeaderElector Watching path
/collections/collection1/leader_elect/shard1/election/94440548245831686-node2_core4-n_0000000003
to know if I could be the leader
[junit4] 2> 261303 INFO (Thread-637) [ ] o.a.s.c.LeaderElector Joined
leadership election with path:
/overseer_elect/election/94440548245831688-127.0.0.1:52175_solr-n_0000000002
[junit4] 2> 261303 INFO (zkCallback-137-thread-2) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 2)
[junit4] 2> 261303 INFO (zkCallback-138-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 2)
[junit4] 2> 261303 INFO (zkCallback-129-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 2)
[junit4] 2> 261304 INFO (Thread-637) [ ]
o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:52175_solr
[junit4] 2> 261304 INFO (Thread-637) [ ] o.a.s.c.c.SolrZkClient
makePath: /overseer_elect/leader
[junit4] 2> 261308 INFO (zkCallback-138-thread-2) [ ]
o.a.s.c.c.SolrZkClient makePath: /collections/collection1/leaders/shard1
[junit4] 2> 261308 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient makePath:
/collections/collection1/leader_elect/shard1/election
[junit4] 2> 261308 INFO (Thread-637) [ ] o.a.s.c.Overseer Overseer
(id=94440548245831688-127.0.0.1:52175_solr-n_0000000002) starting
[junit4] 2> 261311 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.LeaderElector Joined leadership election with path:
/collections/collection1/leader_elect/shard1/election/94440548245831687-node1_core1-n_0000000005
[junit4] 2> 261312 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.LeaderElector was going to be leader
/collections/collection1/leader_elect/shard1/election/94440548245831687-node1_core1-n_0000000005
, seq(0)
/collections/collection1/leader_elect/shard1/election/94440548245831687-node1_core1-n_0000000004
[junit4] 2> 261312 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 261312 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 261314 INFO (zkCallback-141-thread-1) [ ]
o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@73b0c23 name:ZooKeeperConnection
Watcher:127.0.0.1:52175/solr got event WatchedEvent state:SyncConnected
type:None path:null path:null type:None
[junit4] 2> 261314 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 261314 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 261315 INFO (Thread-637) [ ]
o.a.s.c.OverseerAutoReplicaFailoverThread Starting
OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000
autoReplicaFailoverWaitAfterExpiration=30000
autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 261316 INFO
(OverseerCollectionConfigSetProcessor-94440548245831688-127.0.0.1:52175_solr-n_0000000002)
[ ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer
operations
[junit4] 2> 261316 INFO
(OverseerStateUpdate-94440548245831688-127.0.0.1:52175_solr-n_0000000002) [
] o.a.s.c.Overseer Starting to work on the main queue
[junit4] 2> 261316 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 261316 INFO
(OverseerStateUpdate-94440548245831688-127.0.0.1:52175_solr-n_0000000002) [
] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 261318 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient makePath: /live_nodes/node2
[junit4] 2> 261319 INFO (zkCallback-129-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 1)
[junit4] 2> 261319 INFO (zkCallback-138-thread-2) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 1)
[junit4] 2> 261319 INFO (zkCallback-141-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 1)
[junit4] 2> 261319 INFO (zkCallback-139-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 1)
[junit4] 2> 261320 INFO
(OverseerStateUpdate-94440548245831688-127.0.0.1:52175_solr-n_0000000002) [
] o.a.s.c.Overseer processMessage: queueSize: 3, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"} current state version: 4
[junit4] 2> 261321 INFO
(OverseerStateUpdate-94440548245831688-127.0.0.1:52175_solr-n_0000000002) [
] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 261321 INFO
(OverseerStateUpdate-94440548245831688-127.0.0.1:52175_solr-n_0000000002) [
] o.a.s.c.o.ReplicaMutator shard=shard1 is already registered
[junit4] 2> 261322 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient makePath:
/collections/collection1/leader_elect/shard1/election
[junit4] 2> 261326 INFO (zkCallback-129-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 2)
[junit4] 2> 261326 INFO (zkCallback-141-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 2)
[junit4] 2> 261326 INFO (zkCallback-139-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 2)
[junit4] 2> 261326 INFO (zkCallback-138-thread-2) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 2)
[junit4] 2> 261333 INFO
(OverseerStateUpdate-94440548245831688-127.0.0.1:52175_solr-n_0000000002) [
] o.a.s.c.Overseer processMessage: queueSize: 0, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"} current state version: 5
[junit4] 2> 261334 INFO
(OverseerStateUpdate-94440548245831688-127.0.0.1:52175_solr-n_0000000002) [
] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 261334 INFO
(OverseerStateUpdate-94440548245831688-127.0.0.1:52175_solr-n_0000000002) [
] o.a.s.c.o.ReplicaMutator shard=shard1 is already registered
[junit4] 2> 261334 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.LeaderElector Joined leadership election with path:
/collections/collection1/leader_elect/shard1/election/94440548245831687-node1_core1-n_0000000006
[junit4] 2> 261335 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.LeaderElector was going to be leader
/collections/collection1/leader_elect/shard1/election/94440548245831687-node1_core1-n_0000000006
, seq(0)
/collections/collection1/leader_elect/shard1/election/94440548245831687-node1_core1-n_0000000004
[junit4] 2> 261339 INFO
(OverseerStateUpdate-94440548245831688-127.0.0.1:52175_solr-n_0000000002) [
] o.a.s.c.Overseer processMessage: queueSize: 0, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "base_url":"http://node1/solr/";,
[junit4] 2> "core":"core1",
[junit4] 2> "state":"active"} current state version: 5
[junit4] 2> 261344 INFO
(OverseerStateUpdate-94440548245831688-127.0.0.1:52175_solr-n_0000000002) [
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"} current state version: 5
[junit4] 2> 261344 INFO
(OverseerStateUpdate-94440548245831688-127.0.0.1:52175_solr-n_0000000002) [
] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 261344 INFO
(OverseerStateUpdate-94440548245831688-127.0.0.1:52175_solr-n_0000000002) [
] o.a.s.c.o.ReplicaMutator shard=shard1 is already registered
[junit4] 2> 261446 INFO (zkCallback-141-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 2)
[junit4] 2> 261446 INFO (zkCallback-139-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 2)
[junit4] 2> 261446 INFO (zkCallback-138-thread-2) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 2)
[junit4] 2> 261446 INFO (zkCallback-129-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 2)
[junit4] 2> 261541 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient makePath:
/collections/collection1/leader_elect/shard1/election
[junit4] 2> 261542 INFO
(OverseerStateUpdate-94440548245831688-127.0.0.1:52175_solr-n_0000000002) [
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node2",
[junit4] 2> "core":"core4",
[junit4] 2> "core_node_name":"node2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node2/solr/"} current state version: 6
[junit4] 2> 261543 INFO
(OverseerStateUpdate-94440548245831688-127.0.0.1:52175_solr-n_0000000002) [
] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node2",
[junit4] 2> "core":"core4",
[junit4] 2> "core_node_name":"node2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node2/solr/"}
[junit4] 2> 261543 INFO
(OverseerStateUpdate-94440548245831688-127.0.0.1:52175_solr-n_0000000002) [
] o.a.s.c.o.ReplicaMutator shard=shard1 is already registered
[junit4] 2> 261544 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.LeaderElector Joined leadership election with path:
/collections/collection1/leader_elect/shard1/election/94440548245831689-node2_core4-n_0000000007
[junit4] 2> 261545 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.LeaderElector Watching path
/collections/collection1/leader_elect/shard1/election/94440548245831687-node1_core1-n_0000000006
to know if I could be the leader
[junit4] 2> 261546 INFO (zkCallback-138-thread-2) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 2)
[junit4] 2> 261546 INFO (zkCallback-141-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 2)
[junit4] 2> 261547 INFO (zkCallback-129-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 2)
[junit4] 2> 261548 INFO (zkCallback-139-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 2)
[junit4] 2> 261549 INFO (zkCallback-141-thread-2) [ ]
o.a.s.c.c.SolrZkClient makePath: /collections/collection1/leaders/shard1
[junit4] 2> 261552 INFO
(OverseerStateUpdate-94440548245831688-127.0.0.1:52175_solr-n_0000000002) [
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "base_url":"http://node2/solr/";,
[junit4] 2> "core":"core4",
[junit4] 2> "state":"active"} current state version: 6
[junit4] 2> 261655 INFO (zkCallback-141-thread-2) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 1)
[junit4] 2> 261655 INFO (zkCallback-129-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 1)
[junit4] 2> 261655 INFO (zkCallback-139-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 1)
[junit4] 2> 261751 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 261752 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 261753 INFO (zkCallback-142-thread-1) [ ]
o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@2117b96b
name:ZooKeeperConnection Watcher:127.0.0.1:52175/solr got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 261754 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 261754 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 261755 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 261757 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient makePath: /live_nodes/node1
[junit4] 2> 261758 INFO (zkCallback-129-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 1)
[junit4] 2> 261758 INFO (zkCallback-142-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 1)
[junit4] 2> 261758 INFO (zkCallback-139-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 1)
[junit4] 2> 261759 INFO (zkCallback-141-thread-2) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 1)
[junit4] 2> 261761 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient makePath:
/collections/collection1/leader_elect/shard1/election
[junit4] 2> 261763 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.LeaderElector Joined leadership election with path:
/collections/collection1/leader_elect/shard1/election/94440548245831690-node1_core1-n_0000000008
[junit4] 2> 261763 INFO
(OverseerStateUpdate-94440548245831688-127.0.0.1:52175_solr-n_0000000002) [
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"} current state version: 7
[junit4] 2> 261764 INFO
(OverseerStateUpdate-94440548245831688-127.0.0.1:52175_solr-n_0000000002) [
] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 261764 INFO
(OverseerStateUpdate-94440548245831688-127.0.0.1:52175_solr-n_0000000002) [
] o.a.s.c.o.ReplicaMutator shard=shard1 is already registered
[junit4] 2> 261764 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.LeaderElector Watching path
/collections/collection1/leader_elect/shard1/election/94440548245831689-node2_core4-n_0000000007
to know if I could be the leader
[junit4] 2> 261765 INFO (zkCallback-141-thread-2) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 2)
[junit4] 2> 261765 INFO (zkCallback-139-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 2)
[junit4] 2> 261765 INFO (zkCallback-129-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 2)
[junit4] 2> 261766 WARN (zkCallback-141-thread-2) [ ]
o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 261766 INFO (zkCallback-142-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 2)
[junit4] 2> 261767 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ]
o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from
client sessionid 0x14f852da2960009, likely client has closed socket
[junit4] 2> at
org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at
org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 261768 INFO (zkCallback-142-thread-2) [ ]
o.a.s.c.c.SolrZkClient makePath: /collections/collection1/leaders/shard1
[junit4] 2> 261769 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient makePath:
/collections/collection1/leader_elect/shard1/election
[junit4] 2> 261769 INFO
(OverseerStateUpdate-94440548245831688-127.0.0.1:52175_solr-n_0000000002) [
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"} current state version: 7
[junit4] 2> 261770 INFO
(OverseerStateUpdate-94440548245831688-127.0.0.1:52175_solr-n_0000000002) [
] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 261770 INFO
(OverseerStateUpdate-94440548245831688-127.0.0.1:52175_solr-n_0000000002) [
] o.a.s.c.o.ReplicaMutator shard=shard1 is already registered
[junit4] 2> 261772 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.LeaderElector Joined leadership election with path:
/collections/collection1/leader_elect/shard1/election/94440548245831690-node1_core1-n_0000000009
[junit4] 2> 261772 INFO
(OverseerStateUpdate-94440548245831688-127.0.0.1:52175_solr-n_0000000002) [
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "base_url":"http://node1/solr/";,
[junit4] 2> "core":"core1",
[junit4] 2> "state":"active"} current state version: 7
[junit4] 2> 261772 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.LeaderElector was going to be leader
/collections/collection1/leader_elect/shard1/election/94440548245831690-node1_core1-n_0000000009
, seq(0)
/collections/collection1/leader_elect/shard1/election/94440548245831690-node1_core1-n_0000000008
[junit4] 2> 261773 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 261773 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 261775 INFO (zkCallback-143-thread-1) [ ]
o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@3e03399f
name:ZooKeeperConnection Watcher:127.0.0.1:52175/solr got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 261776 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 261776 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7E2A]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 261778 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[7B71D6EA298C7
[...truncated too long message...]
junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:214)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 312350 INFO
(OverseerStateUpdate-94440551759478787-127.0.0.1:58208_solr-n_0000000000) [
] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
[junit4] 2> "operation":"create",
[junit4] 2> "name":"test",
[junit4] 2> "numShards":"1",
[junit4] 2> "replicationFactor":"1",
[junit4] 2> "stateFormat":"2"} current state version: 2
[junit4] 2> 312351 INFO
(OverseerStateUpdate-94440551759478787-127.0.0.1:58208_solr-n_0000000000) [
] o.a.s.c.o.ClusterStateMutator building a new cName: test
[junit4] 2> 312352 INFO
(OverseerStateUpdate-94440551759478787-127.0.0.1:58208_solr-n_0000000000) [
] o.a.s.c.o.ZkStateWriter going to create_collection
/collections/test/state.json
[junit4] 2> 312360 INFO
(OverseerStateUpdate-94440551759478787-127.0.0.1:58208_solr-n_0000000000) [
] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
[junit4] 2> "operation":"createshard",
[junit4] 2> "collection":"test",
[junit4] 2> "shard":"x",
[junit4] 2> "replicationFactor":"1"} current state version: 2
[junit4] 2> 312362 INFO
(OverseerStateUpdate-94440551759478787-127.0.0.1:58208_solr-n_0000000000) [
] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
[junit4] 2> "operation":"addreplica",
[junit4] 2> "collection":"test",
[junit4] 2> "shard":"x",
[junit4] 2> "base_url":"http://127.0.0.1/solr";,
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "state":"down"} current state version: 2
[junit4] 2> 312362 INFO
(OverseerStateUpdate-94440551759478787-127.0.0.1:58208_solr-n_0000000000) [
] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"addreplica",
[junit4] 2> "collection":"test",
[junit4] 2> "shard":"x",
[junit4] 2> "base_url":"http://127.0.0.1/solr";,
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "state":"down"}
[junit4] 2> 312472 INFO
(OverseerStateUpdate-94440551759478787-127.0.0.1:58208_solr-n_0000000000) [
] o.a.s.c.o.ZkStateWriter going to update_collection
/collections/test/state.json version: 0
[junit4] 2> 317586 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ]
o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from
client sessionid 0x14f852e74040003, likely client has closed socket
[junit4] 2> at
org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at
org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 317586 INFO
(TEST-OverseerTest.testExternalClusterStateChangeBehavior-seed#[7B71D6EA298C7E2A])
[ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:58208 58208
[junit4] 2> 317811 INFO (Thread-646) [ ] o.a.s.c.ZkTestServer
connecting to 127.0.0.1:58208 58208
[junit4] 2> 317812 WARN (Thread-646) [ ] o.a.s.c.ZkTestServer Watch
limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 2 /solr/aliases.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 2 /solr/clusterstate.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 2 /solr/live_nodes
[junit4] 2> 2 /solr/collections
[junit4] 2>
[junit4] 2> 317813 INFO
(TEST-OverseerTest.testExternalClusterStateChangeBehavior-seed#[7B71D6EA298C7E2A])
[ ] o.a.s.SolrTestCaseJ4 ###Ending testExternalClusterStateChangeBehavior
[junit4] 2> 317813 INFO
(TEST-OverseerTest.testExternalClusterStateChangeBehavior-seed#[7B71D6EA298C7E2A])
[ ] o.a.s.c.Overseer Overseer
(id=94440551759478787-127.0.0.1:58208_solr-n_0000000000) closing
[junit4] 2> 317816 INFO
(OverseerStateUpdate-94440551759478787-127.0.0.1:58208_solr-n_0000000000) [
] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:58208_solr
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=OverseerTest
-Dtests.method=testExternalClusterStateChangeBehavior
-Dtests.seed=7B71D6EA298C7E2A -Dtests.multiplier=2 -Dtests.slow=true
-Dtests.locale=es -Dtests.timezone=America/Cambridge_Bay -Dtests.asserts=true
-Dtests.file.encoding=US-ASCII
[junit4] FAILURE 5.88s J2 |
OverseerTest.testExternalClusterStateChangeBehavior <<<
[junit4] > Throwable #1: java.lang.AssertionError: Illegal state, was:
down expected:active clusterState:live
nodes:[]collections:{c1=DocCollection(c1)={
[junit4] > "shards":{"shard1":{
[junit4] > "parent":null,
[junit4] > "range":null,
[junit4] > "state":"active",
[junit4] > "replicas":{"core_node1":{
[junit4] > "base_url":"http://127.0.0.1/solr";,
[junit4] > "node_name":"node1",
[junit4] > "core":"core1",
[junit4] > "roles":"",
[junit4] > "state":"down"}}}},
[junit4] > "router":{"name":"implicit"}}, test=LazyCollectionRef(test)}
[junit4] > at
__randomizedtesting.SeedInfo.seed([7B71D6EA298C7E2A:136FD506CB1C2464]:0)
[junit4] > at
org.apache.solr.cloud.OverseerTest.verifyStatus(OverseerTest.java:601)
[junit4] > at
org.apache.solr.cloud.OverseerTest.testExternalClusterStateChangeBehavior(OverseerTest.java:1261)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 317837 ERROR (OverseerExitThread) [ ] o.a.s.c.Overseer
could not read the data
[junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException:
KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at
org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:353)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:350)
[junit4] 2> at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:350)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:304)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:87)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater$2.run(Overseer.java:265)
[junit4] 2> 320829 INFO
(SUITE-OverseerTest-seed#[7B71D6EA298C7E2A]-worker) [ ] o.a.s.SolrTestCaseJ4
###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.OverseerTest_7B71D6EA298C7E2A-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene53): {},
docValues:{}, sim=DefaultSimilarity, locale=es, timezone=America/Cambridge_Bay
[junit4] 2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation
1.8.0_45 (64-bit)/cpus=4,threads=1,free=180977048,total=448266240
[junit4] 2> NOTE: All tests run in this JVM: [TestSolr4Spatial2,
TestExtendedDismaxParser, UnloadDistributedZkTest, TestStressReorder,
TestUniqueKeyFieldResource, CursorMarkTest, TestExpandComponent,
TestDynamicFieldCollectionResource, TestManagedResourceStorage,
TestManagedResource, SimpleFacetsTest, TestShardHandlerFactory,
HdfsDirectoryFactoryTest, TestSearchPerf, TestAddFieldRealTimeGet,
DirectSolrConnectionTest, TestCursorMarkWithoutUniqueKey,
AsyncMigrateRouteKeyTest, PrimitiveFieldTypeTest, TestTrieFacet,
SystemInfoHandlerTest, TestManagedStopFilterFactory,
FieldAnalysisRequestHandlerTest, TestFieldTypeCollectionResource, TestSolrXml,
HdfsBasicDistributedZkTest, TestPostingsSolrHighlighter,
XmlUpdateRequestHandlerTest, TestRandomRequestDistribution,
TestDefaultStatsCache, SharedFSAutoReplicaFailoverTest,
TestClassNameShortening, OverseerTest]
[junit4] Completed [107/543] on J2 in 63.49s, 12 tests, 1 failure, 1 skipped
<<< FAILURES!
[...truncated 1371 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/build.xml:775:
The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/build.xml:719:
The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/build.xml:59:
The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build.xml:233:
The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/common-build.xml:514:
The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/lucene/common-build.xml:1432:
The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/lucene/common-build.xml:987:
There were test failures: 543 suites, 2145 tests, 1 failure, 47 ignored (35
assumptions)
Total time: 55 minutes 19 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-Tests-trunk-Java8 #343
Archived 4 artifacts
Archive block size is 32768
Received 0 blocks and 298911238 bytes
Compression is 0.0%
Took 57 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
