Build: https://builds.apache.org/job/Lucene-Solr-Tests-5.x-Java7/3852/
1 tests failed.
FAILED:
org.apache.solr.cloud.OverseerTest.testExternalClusterStateChangeBehavior
Error Message:
Illegal state, was: down expected:active clusterState:live
nodes:[]collections:{c1=DocCollection(c1)={ "shards":{"shard1":{
"state":"active", "range":null, "parent":null,
"replicas":{"core_node1":{ "base_url":"http://127.0.0.1/solr";,
"node_name":"node1", "core":"core1", "roles":"",
"state":"down"}}}}, "router":{"name":"implicit"}},
test=LazyCollectionRef(test)}
Stack Trace:
java.lang.AssertionError: Illegal state, was: down expected:active
clusterState:live nodes:[]collections:{c1=DocCollection(c1)={
"shards":{"shard1":{
"state":"active",
"range":null,
"parent":null,
"replicas":{"core_node1":{
"base_url":"http://127.0.0.1/solr";,
"node_name":"node1",
"core":"core1",
"roles":"",
"state":"down"}}}},
"router":{"name":"implicit"}}, test=LazyCollectionRef(test)}
at
__randomizedtesting.SeedInfo.seed([1928F325F304463C:7136F0C911941C72]:0)
at org.junit.Assert.fail(Assert.java:93)
at
org.apache.solr.cloud.AbstractDistribZkTestBase.verifyReplicaStatus(AbstractDistribZkTestBase.java:237)
at
org.apache.solr.cloud.OverseerTest.testExternalClusterStateChangeBehavior(OverseerTest.java:1262)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 9810 lines...]
[junit4] Suite: org.apache.solr.cloud.OverseerTest
[junit4] 2> Creating dataDir:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.OverseerTest_1928F325F304463C-001/init-core-data-001
[junit4] 2> 6909 INFO (SUITE-OverseerTest-seed#[1928F325F304463C]-worker)
[ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
[junit4] 2> 6909 INFO (SUITE-OverseerTest-seed#[1928F325F304463C]-worker)
[ ] o.a.s.SolrTestCaseJ4 ####initCore
[junit4] 2> 6909 INFO (SUITE-OverseerTest-seed#[1928F325F304463C]-worker)
[ ] o.a.s.SolrTestCaseJ4 ####initCore end
[junit4] 2> 6937 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.SolrTestCaseJ4 ###Starting testShardLeaderChange
[junit4] 2> 6956 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 6964 INFO (Thread-4) [ ] o.a.s.c.ZkTestServer client
port:0.0.0.0/0.0.0.0:0
[junit4] 2> 6964 INFO (Thread-4) [ ] o.a.s.c.ZkTestServer Starting
server
[junit4] 2> 7167 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.ZkTestServer start zk server on port:60416
[junit4] 2> 7219 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 7327 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 7373 INFO (zkCallback-2-thread-1) [ ]
o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@19fa7c4 name:ZooKeeperConnection
Watcher:127.0.0.1:60416/solr got event WatchedEvent state:SyncConnected
type:None path:null path:null type:None
[junit4] 2> 7373 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 7374 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 7376 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 7382 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 7393 INFO (zkCallback-3-thread-1) [ ]
o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@65ce0806
name:ZooKeeperConnection Watcher:127.0.0.1:60416 got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 7393 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 7394 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 7419 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 7426 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 7427 INFO (zkCallback-4-thread-1) [ ]
o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@1e79c125
name:ZooKeeperConnection Watcher:127.0.0.1:60416 got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 7427 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 7427 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 7429 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.SolrZkClient makePath: /solr
[junit4] 2> 7458 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.SolrZkClient makePath: /live_nodes
[junit4] 2> 7461 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.SolrZkClient makePath: /collections
[junit4] 2> 7465 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.SolrZkClient makePath: /aliases.json
[junit4] 2> 7468 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
[junit4] 2> 7472 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.SolrZkClient makePath: /security.json
[junit4] 2> 7476 INFO (Thread-5) [ ] o.a.s.c.c.SolrZkClient Using
default ZkCredentialsProvider
[junit4] 2> 7482 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 7483 INFO (Thread-5) [ ] o.a.s.c.c.ConnectionManager
Waiting for client to connect to ZooKeeper
[junit4] 2> 7485 INFO (zkCallback-5-thread-1) [ ]
o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@549154f9
name:ZooKeeperConnection Watcher:127.0.0.1:60416/solr got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 7485 INFO (Thread-5) [ ] o.a.s.c.c.ConnectionManager
Client is connected to ZooKeeper
[junit4] 2> 7485 INFO (Thread-5) [ ] o.a.s.c.c.SolrZkClient Using
default ZkACLProvider
[junit4] 2> 7494 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 7495 INFO (Thread-5) [ ] o.a.s.u.UpdateShardHandler
Creating UpdateShardHandler HTTP client with params:
socketTimeout=600000&connTimeout=60000&retry=true
[junit4] 2> 7503 INFO (zkCallback-6-thread-1) [ ]
o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@578e5065
name:ZooKeeperConnection Watcher:127.0.0.1:60416/solr got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 7504 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 7504 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 7514 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 7516 INFO (Thread-5) [ ] o.a.s.c.c.SolrZkClient makePath:
/overseer_elect
[junit4] 2> 7521 INFO (Thread-5) [ ] o.a.s.c.c.SolrZkClient makePath:
/overseer_elect/election
[junit4] 2> 7526 INFO (Thread-5) [ ] o.a.s.c.Overseer Overseer
(id=null) closing
[junit4] 2> 7528 INFO (Thread-5) [ ] o.a.s.c.LeaderElector Joined
leadership election with path:
/overseer_elect/election/95063054811332611-127.0.0.1:60416_solr-n_0000000000
[junit4] 2> 7537 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.SolrZkClient makePath: /live_nodes/node1
[junit4] 2> 7541 INFO (Thread-5) [ ] o.a.s.c.OverseerElectionContext I
am going to be the leader 127.0.0.1:60416_solr
[junit4] 2> 7547 INFO (Thread-5) [ ] o.a.s.c.c.SolrZkClient makePath:
/overseer_elect/leader
[junit4] 2> 7551 INFO (zkCallback-6-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 0)
[junit4] 2> 7556 INFO (Thread-5) [ ] o.a.s.c.Overseer Overseer
(id=95063054811332611-127.0.0.1:60416_solr-n_0000000000) starting
[junit4] 2> 7563 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.SolrZkClient makePath: /overseer/queue
[junit4] 2> 7571 INFO (Thread-5) [ ] o.a.s.c.c.SolrZkClient makePath:
/overseer/queue-work
[junit4] 2> 7577 INFO (Thread-5) [ ] o.a.s.c.c.SolrZkClient makePath:
/overseer/collection-map-failure
[junit4] 2> 7582 INFO (Thread-5) [ ] o.a.s.c.c.SolrZkClient makePath:
/overseer/collection-map-running
[junit4] 2> 7593 INFO (Thread-5) [ ] o.a.s.c.c.SolrZkClient makePath:
/overseer/collection-map-completed
[junit4] 2> 7617 INFO (Thread-5) [ ] o.a.s.c.c.SolrZkClient makePath:
/overseer/collection-queue-work
[junit4] 2> 7647 INFO (Thread-5) [ ]
o.a.s.c.OverseerAutoReplicaFailoverThread Starting
OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000
autoReplicaFailoverWaitAfterExpiration=30000
autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 7755 INFO
(OverseerCollectionConfigSetProcessor-95063054811332611-127.0.0.1:60416_solr-n_0000000000)
[ ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer
operations
[junit4] 2> 7757 INFO (Thread-5) [ ] o.a.s.c.OverseerTest Killing
overseer.
[junit4] 2> 7755 INFO
(OverseerStateUpdate-95063054811332611-127.0.0.1:60416_solr-n_0000000000) [
] o.a.s.c.Overseer Starting to work on the main queue
[junit4] 2> 7762 INFO (Thread-5) [ ] o.a.s.c.c.SolrZkClient Using
default ZkCredentialsProvider
[junit4] 2> 7768 INFO (Thread-5) [ ] o.a.s.c.c.ConnectionManager
Waiting for client to connect to ZooKeeper
[junit4] 2> 7770 INFO (zkCallback-8-thread-1) [ ]
o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@139bc376
name:ZooKeeperConnection Watcher:127.0.0.1:60416/solr got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 7770 INFO (Thread-5) [ ] o.a.s.c.c.ConnectionManager
Client is connected to ZooKeeper
[junit4] 2> 7770 INFO (Thread-5) [ ] o.a.s.c.c.SolrZkClient Using
default ZkACLProvider
[junit4] 2> 7771 INFO (Thread-5) [ ] o.a.s.c.Overseer Overseer
(id=95063054811332611-127.0.0.1:60416_solr-n_0000000000) closing
[junit4] 2> 7771 INFO (Thread-5) [ ] o.a.s.u.UpdateShardHandler
Creating UpdateShardHandler HTTP client with params:
socketTimeout=600000&connTimeout=60000&retry=true
[junit4] 2> 7771 INFO
(OverseerStateUpdate-95063054811332611-127.0.0.1:60416_solr-n_0000000000) [
] o.a.s.c.Overseer According to ZK I
(id=95063054811332611-127.0.0.1:60416_solr-n_0000000000) am no longer a leader.
[junit4] 2> 7771 INFO
(OverseerStateUpdate-95063054811332611-127.0.0.1:60416_solr-n_0000000000) [
] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:60416_solr
[junit4] 2> 7774 ERROR
(OverseerCollectionConfigSetProcessor-95063054811332611-127.0.0.1:60416_solr-n_0000000000)
[ ] o.a.s.c.OverseerTaskProcessor Unable to prioritize overseer
[junit4] 2> java.lang.InterruptedException: sleep interrupted
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at
org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:108)
[junit4] 2> at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:76)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:308)
[junit4] 2> at
org.apache.solr.cloud.OverseerNodePrioritizer.prioritizeOverseerNodes(OverseerNodePrioritizer.java:61)
[junit4] 2> at
org.apache.solr.cloud.OverseerTaskProcessor.run(OverseerTaskProcessor.java:159)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 7783 ERROR (OverseerExitThread) [ ] o.a.s.c.Overseer could
not read the data
[junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException:
KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at
org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:345)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:342)
[junit4] 2> at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:342)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:305)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:88)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater$2.run(Overseer.java:266)
[junit4] 2> 7783 ERROR
(OverseerHdfsCoreFailoverThread-95063054811332611-127.0.0.1:60416_solr-n_0000000000)
[ ] o.a.s.c.OverseerAutoReplicaFailoverThread
OverseerAutoReplicaFailoverThread had an error in its thread work
loop.:org.apache.solr.common.SolrException: Error reading cluster properties
[junit4] 2> at
org.apache.solr.common.cloud.ZkStateReader.getClusterProps(ZkStateReader.java:738)
[junit4] 2> at
org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.doWork(OverseerAutoReplicaFailoverThread.java:153)
[junit4] 2> at
org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:132)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> Caused by: java.lang.InterruptedException: sleep interrupted
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at
org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:108)
[junit4] 2> at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:76)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:308)
[junit4] 2> at
org.apache.solr.common.cloud.ZkStateReader.getClusterProps(ZkStateReader.java:731)
[junit4] 2> ... 3 more
[junit4] 2>
[junit4] 2> 7787 INFO (Thread-5) [ ] o.a.s.c.Overseer Overseer
(id=null) closing
[junit4] 2> 7788 INFO (Thread-5) [ ] o.a.s.c.LeaderElector Joined
leadership election with path:
/overseer_elect/election/95063054811332613-127.0.0.1:60416_solr-n_0000000001
[junit4] 2> 7789 INFO (Thread-5) [ ] o.a.s.c.OverseerElectionContext I
am going to be the leader 127.0.0.1:60416_solr
[junit4] 2> 7789 INFO (Thread-5) [ ] o.a.s.c.c.SolrZkClient makePath:
/overseer_elect/leader
[junit4] 2> 7792 INFO (Thread-5) [ ] o.a.s.c.Overseer Overseer
(id=95063054811332613-127.0.0.1:60416_solr-n_0000000001) starting
[junit4] 2> 7810 INFO (Thread-5) [ ]
o.a.s.c.OverseerAutoReplicaFailoverThread Starting
OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000
autoReplicaFailoverWaitAfterExpiration=30000
autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 7811 INFO
(OverseerCollectionConfigSetProcessor-95063054811332613-127.0.0.1:60416_solr-n_0000000001)
[ ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer
operations
[junit4] 2> 7811 INFO
(OverseerStateUpdate-95063054811332613-127.0.0.1:60416_solr-n_0000000001) [
] o.a.s.c.Overseer Starting to work on the main queue
[junit4] 2> 7813 INFO
(OverseerStateUpdate-95063054811332613-127.0.0.1:60416_solr-n_0000000001) [
] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 7826 INFO
(OverseerStateUpdate-95063054811332613-127.0.0.1:60416_solr-n_0000000001) [
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"} current state version: 0
[junit4] 2> 7836 INFO
(OverseerStateUpdate-95063054811332613-127.0.0.1:60416_solr-n_0000000001) [
] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 7837 INFO
(OverseerStateUpdate-95063054811332613-127.0.0.1:60416_solr-n_0000000001) [
] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
[junit4] 2> 7848 INFO
(OverseerStateUpdate-95063054811332613-127.0.0.1:60416_solr-n_0000000001) [
] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 7856 INFO (zkCallback-6-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 1)
[junit4] 2> 7856 INFO (zkCallback-8-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 1)
[junit4] 2> 8070 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.SolrZkClient makePath:
/collections/collection1/leader_elect/shard1/election
[junit4] 2> 8073 INFO (zkCallback-8-thread-1) [ ]
o.a.s.c.c.ZkStateReader A collections change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/collections, has occurred - updating...
[junit4] 2> 8073 INFO (zkCallback-6-thread-1) [ ]
o.a.s.c.c.ZkStateReader A collections change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/collections, has occurred - updating...
[junit4] 2> 8084 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.LeaderElector Joined leadership election with path:
/collections/collection1/leader_elect/shard1/election/95063054811332612-node1_core1-n_0000000000
[junit4] 2> 8091 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.SolrZkClient makePath: /collections/collection1/leaders/shard1
[junit4] 2> 8107 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
[junit4] 2> 8134 INFO
(OverseerStateUpdate-95063054811332613-127.0.0.1:60416_solr-n_0000000001) [
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "base_url":"http://node1/solr/";,
[junit4] 2> "core":"core1",
[junit4] 2> "state":"active"} current state version: 1
[junit4] 2> 8140 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.SolrZkClient makePath:
/collections/collection1/leader_elect/shard1/election
[junit4] 2> 8143 INFO
(OverseerStateUpdate-95063054811332613-127.0.0.1:60416_solr-n_0000000001) [
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"} current state version: 1
[junit4] 2> 8144 INFO
(OverseerStateUpdate-95063054811332613-127.0.0.1:60416_solr-n_0000000001) [
] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 8144 INFO
(OverseerStateUpdate-95063054811332613-127.0.0.1:60416_solr-n_0000000001) [
] o.a.s.c.o.ReplicaMutator shard=shard1 is already registered
[junit4] 2> 8152 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.LeaderElector Joined leadership election with path:
/collections/collection1/leader_elect/shard1/election/95063054811332612-node1_core1-n_0000000001
[junit4] 2> 8153 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.LeaderElector was going to be leader
/collections/collection1/leader_elect/shard1/election/95063054811332612-node1_core1-n_0000000001
, seq(0)
/collections/collection1/leader_elect/shard1/election/95063054811332612-node1_core1-n_0000000000
[junit4] 2> 8154 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 8154 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 8157 INFO (zkCallback-10-thread-1) [ ]
o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@6ad28f39
name:ZooKeeperConnection Watcher:127.0.0.1:60416/solr got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 8157 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 8157 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 8161 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 8171 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.SolrZkClient makePath: /live_nodes/node2
[junit4] 2> 8174 INFO (zkCallback-8-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 1)
[junit4] 2> 8174 INFO (zkCallback-6-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 1)
[junit4] 2> 8175 INFO (zkCallback-10-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 1)
[junit4] 2> 8181 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.SolrZkClient makePath:
/collections/collection1/leader_elect/shard1/election
[junit4] 2> 8184 INFO
(OverseerStateUpdate-95063054811332613-127.0.0.1:60416_solr-n_0000000001) [
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"} current state version: 1
[junit4] 2> 8185 INFO
(OverseerStateUpdate-95063054811332613-127.0.0.1:60416_solr-n_0000000001) [
] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 8185 INFO
(OverseerStateUpdate-95063054811332613-127.0.0.1:60416_solr-n_0000000001) [
] o.a.s.c.o.ReplicaMutator shard=shard1 is already registered
[junit4] 2> 8189 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.LeaderElector Joined leadership election with path:
/collections/collection1/leader_elect/shard1/election/95063054811332612-node1_core1-n_0000000002
[junit4] 2> 8191 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.LeaderElector was going to be leader
/collections/collection1/leader_elect/shard1/election/95063054811332612-node1_core1-n_0000000002
, seq(0)
/collections/collection1/leader_elect/shard1/election/95063054811332612-node1_core1-n_0000000000
[junit4] 2> 8191 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 8294 INFO (zkCallback-2-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 2)
[junit4] 2> 8294 INFO (zkCallback-8-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 2)
[junit4] 2> 8294 INFO (zkCallback-6-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 2)
[junit4] 2> 8294 INFO (zkCallback-10-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 2)
[junit4] 2> 8404 INFO
(OverseerStateUpdate-95063054811332613-127.0.0.1:60416_solr-n_0000000001) [
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node2",
[junit4] 2> "core":"core4",
[junit4] 2> "core_node_name":"node2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node2/solr/"} current state version: 2
[junit4] 2> 8405 INFO
(OverseerStateUpdate-95063054811332613-127.0.0.1:60416_solr-n_0000000001) [
] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node2",
[junit4] 2> "core":"core4",
[junit4] 2> "core_node_name":"node2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node2/solr/"}
[junit4] 2> 8405 INFO
(OverseerStateUpdate-95063054811332613-127.0.0.1:60416_solr-n_0000000001) [
] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=1
[junit4] 2> 8405 INFO
(OverseerStateUpdate-95063054811332613-127.0.0.1:60416_solr-n_0000000001) [
] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 8511 INFO (zkCallback-8-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 2)
[junit4] 2> 8511 INFO (zkCallback-2-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 2)
[junit4] 2> 8511 INFO (zkCallback-6-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 2)
[junit4] 2> 8512 INFO (zkCallback-10-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 2)
[junit4] 2> 8902 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.SolrZkClient makePath:
/collections/collection1/leader_elect/shard1/election
[junit4] 2> 8910 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.LeaderElector Joined leadership election with path:
/collections/collection1/leader_elect/shard1/election/95063054811332614-node2_core4-n_0000000003
[junit4] 2> 8917 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.LeaderElector Watching path
/collections/collection1/leader_elect/shard1/election/95063054811332612-node1_core1-n_0000000002
to know if I could be the leader
[junit4] 2> 8918 INFO (zkCallback-2-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 2)
[junit4] 2> 8918 INFO (zkCallback-8-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 2)
[junit4] 2> 8919 INFO (zkCallback-6-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 2)
[junit4] 2> 8922 INFO (zkCallback-10-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 2)
[junit4] 2> 8926 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ]
o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from
client sessionid 0x151bb583d4b0004, likely client has closed socket
[junit4] 2> at
org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at
org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 8930 INFO (zkCallback-10-thread-2) [ ]
o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
[junit4] 2> 8941 INFO
(OverseerStateUpdate-95063054811332613-127.0.0.1:60416_solr-n_0000000001) [
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "base_url":"http://node2/solr/";,
[junit4] 2> "core":"core4",
[junit4] 2> "state":"active"} current state version: 3
[junit4] 2> 9048 INFO (zkCallback-10-thread-2) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 1)
[junit4] 2> 9048 INFO (zkCallback-2-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 1)
[junit4] 2> 9048 INFO (zkCallback-8-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 1)
[junit4] 2> 9132 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 9132 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 9134 INFO (zkCallback-11-thread-1) [ ]
o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@5173b4cf
name:ZooKeeperConnection Watcher:127.0.0.1:60416/solr got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 9135 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 9135 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 9139 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 9146 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.SolrZkClient makePath: /live_nodes/node1
[junit4] 2> 9151 INFO (zkCallback-10-thread-2) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 1)
[junit4] 2> 9151 INFO (zkCallback-8-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 1)
[junit4] 2> 9152 INFO (zkCallback-2-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 1)
[junit4] 2> 9152 INFO (zkCallback-11-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 1)
[junit4] 2> 9157 INFO
(OverseerStateUpdate-95063054811332613-127.0.0.1:60416_solr-n_0000000001) [
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"} current state version: 4
[junit4] 2> 9158 INFO
(OverseerStateUpdate-95063054811332613-127.0.0.1:60416_solr-n_0000000001) [
] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 9158 INFO
(OverseerStateUpdate-95063054811332613-127.0.0.1:60416_solr-n_0000000001) [
] o.a.s.c.o.ReplicaMutator shard=shard1 is already registered
[junit4] 2> 9161 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.SolrZkClient makePath:
/collections/collection1/leader_elect/shard1/election
[junit4] 2> 9170 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.LeaderElector Joined leadership election with path:
/collections/collection1/leader_elect/shard1/election/95063054811332615-node1_core1-n_0000000004
[junit4] 2> 9172 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.LeaderElector Watching path
/collections/collection1/leader_elect/shard1/election/95063054811332614-node2_core4-n_0000000003
to know if I could be the leader
[junit4] 2> 9173 INFO (zkCallback-2-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 2)
[junit4] 2> 9173 INFO (zkCallback-8-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 2)
[junit4] 2> 9174 INFO (zkCallback-10-thread-2) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 2)
[junit4] 2> 9182 INFO (zkCallback-11-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 2)
[junit4] 2> 9184 INFO (zkCallback-11-thread-2) [ ]
o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
[junit4] 2> 9190 INFO
(OverseerStateUpdate-95063054811332613-127.0.0.1:60416_solr-n_0000000001) [
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "base_url":"http://node1/solr/";,
[junit4] 2> "core":"core1",
[junit4] 2> "state":"active"} current state version: 4
[junit4] 2> 9219 INFO (Thread-5) [ ] o.a.s.c.OverseerTest Killing
overseer.
[junit4] 2> 9220 INFO (Thread-5) [ ] o.a.s.c.c.SolrZkClient Using
default ZkCredentialsProvider
[junit4] 2> 9221 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ]
o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from
client sessionid 0x151bb583d4b0005, likely client has closed socket
[junit4] 2> at
org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at
org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 9222 INFO (Thread-5) [ ] o.a.s.c.c.ConnectionManager
Waiting for client to connect to ZooKeeper
[junit4] 2> 9224 INFO (zkCallback-12-thread-1) [ ]
o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@79ca3b13
name:ZooKeeperConnection Watcher:127.0.0.1:60416/solr got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 9224 INFO (Thread-5) [ ] o.a.s.c.c.ConnectionManager
Client is connected to ZooKeeper
[junit4] 2> 9224 INFO (Thread-5) [ ] o.a.s.c.c.SolrZkClient Using
default ZkACLProvider
[junit4] 2> 9224 INFO (Thread-5) [ ] o.a.s.c.Overseer Overseer
(id=95063054811332613-127.0.0.1:60416_solr-n_0000000001) closing
[junit4] 2> 9224 INFO
(OverseerStateUpdate-95063054811332613-127.0.0.1:60416_solr-n_0000000001) [
] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:60416_solr
[junit4] 2> 9225 INFO (Thread-5) [ ] o.a.s.u.UpdateShardHandler
Creating UpdateShardHandler HTTP client with params:
socketTimeout=600000&connTimeout=60000&retry=true
[junit4] 2> 9227 ERROR (OverseerExitThread) [ ] o.a.s.c.Overseer could
not read the data
[junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException:
KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at
org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:345)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:342)
[junit4] 2> at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:342)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:305)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:88)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater$2.run(Overseer.java:266)
[junit4] 2> 9230 INFO (Thread-5) [ ] o.a.s.c.Overseer Overseer
(id=null) closing
[junit4] 2> 9232 INFO (Thread-5) [ ] o.a.s.c.LeaderElector Joined
leadership election with path:
/overseer_elect/election/95063054811332616-127.0.0.1:60416_solr-n_0000000002
[junit4] 2> 9232 INFO (Thread-5) [ ] o.a.s.c.OverseerElectionContext I
am going to be the leader 127.0.0.1:60416_solr
[junit4] 2> 9233 INFO (Thread-5) [ ] o.a.s.c.c.SolrZkClient makePath:
/overseer_elect/leader
[junit4] 2> 9235 INFO (Thread-5) [ ] o.a.s.c.Overseer Overseer
(id=95063054811332616-127.0.0.1:60416_solr-n_0000000002) starting
[junit4] 2> 9253 INFO (Thread-5) [ ]
o.a.s.c.OverseerAutoReplicaFailoverThread Starting
OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000
autoReplicaFailoverWaitAfterExpiration=30000
autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 9254 INFO
(OverseerStateUpdate-95063054811332616-127.0.0.1:60416_solr-n_0000000002) [
] o.a.s.c.Overseer Starting to work on the main queue
[junit4] 2> 9254 INFO
(OverseerStateUpdate-95063054811332616-127.0.0.1:60416_solr-n_0000000002) [
] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 9260 INFO
(OverseerStateUpdate-95063054811332616-127.0.0.1:60416_solr-n_0000000002) [
] o.a.s.c.Overseer processMessage: workQueueSize: 2, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 9261 INFO
(OverseerStateUpdate-95063054811332616-127.0.0.1:60416_solr-n_0000000002) [
] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 9261 INFO
(OverseerStateUpdate-95063054811332616-127.0.0.1:60416_solr-n_0000000002) [
] o.a.s.c.o.ReplicaMutator shard=shard1 is already registered
[junit4] 2> 9263 INFO (zkCallback-2-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 1)
[junit4] 2> 9265 INFO (zkCallback-12-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 1)
[junit4] 2> 9265 INFO
(OverseerCollectionConfigSetProcessor-95063054811332616-127.0.0.1:60416_solr-n_0000000002)
[ ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer
operations
[junit4] 2> 9265 INFO (zkCallback-11-thread-2) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 1)
[junit4] 2> 9271 INFO
(OverseerStateUpdate-95063054811332616-127.0.0.1:60416_solr-n_0000000002) [
] o.a.s.c.Overseer processMessage: workQueueSize: 0, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "base_url":"http://node1/solr/";,
[junit4] 2> "core":"core1",
[junit4] 2> "state":"active"}
[junit4] 2> 9272 INFO (zkCallback-2-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 1)
[junit4] 2> 9273 INFO (zkCallback-11-thread-2) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 1)
[junit4] 2> 9273 INFO (zkCallback-12-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 1)
[junit4] 2> 10674 WARN (zkCallback-10-thread-2) [ ]
o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 10678 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.SolrZkClient makePath:
/collections/collection1/leader_elect/shard1/election
[junit4] 2> 10680 INFO
(OverseerStateUpdate-95063054811332616-127.0.0.1:60416_solr-n_0000000002) [
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"} current state version: 6
[junit4] 2> 10680 INFO
(OverseerStateUpdate-95063054811332616-127.0.0.1:60416_solr-n_0000000002) [
] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 10681 INFO
(OverseerStateUpdate-95063054811332616-127.0.0.1:60416_solr-n_0000000002) [
] o.a.s.c.o.ReplicaMutator shard=shard1 is already registered
[junit4] 2> 10685 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.LeaderElector Joined leadership election with path:
/collections/collection1/leader_elect/shard1/election/95063054811332615-node1_core1-n_0000000005
[junit4] 2> 10686 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.LeaderElector was going to be leader
/collections/collection1/leader_elect/shard1/election/95063054811332615-node1_core1-n_0000000005
, seq(0)
/collections/collection1/leader_elect/shard1/election/95063054811332615-node1_core1-n_0000000004
[junit4] 2> 10686 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 10690 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 10691 INFO (zkCallback-14-thread-1) [ ]
o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@7791e67 name:ZooKeeperConnection
Watcher:127.0.0.1:60416/solr got event WatchedEvent state:SyncConnected
type:None path:null path:null type:None
[junit4] 2> 10691 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 10692 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 10695 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 10700 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.SolrZkClient makePath: /live_nodes/node2
[junit4] 2> 10702 INFO (zkCallback-12-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 1)
[junit4] 2> 10702 INFO (zkCallback-2-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 1)
[junit4] 2> 10702 INFO (zkCallback-14-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 1)
[junit4] 2> 10702 INFO (zkCallback-11-thread-2) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 1)
[junit4] 2> 10707 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.SolrZkClient makePath:
/collections/collection1/leader_elect/shard1/election
[junit4] 2> 10709 INFO
(OverseerStateUpdate-95063054811332616-127.0.0.1:60416_solr-n_0000000002) [
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"} current state version: 6
[junit4] 2> 10709 INFO
(OverseerStateUpdate-95063054811332616-127.0.0.1:60416_solr-n_0000000002) [
] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 10709 INFO
(OverseerStateUpdate-95063054811332616-127.0.0.1:60416_solr-n_0000000002) [
] o.a.s.c.o.ReplicaMutator shard=shard1 is already registered
[junit4] 2> 10714 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.LeaderElector Joined leadership election with path:
/collections/collection1/leader_elect/shard1/election/95063054811332615-node1_core1-n_0000000006
[junit4] 2> 10715 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.LeaderElector was going to be leader
/collections/collection1/leader_elect/shard1/election/95063054811332615-node1_core1-n_0000000006
, seq(0)
/collections/collection1/leader_elect/shard1/election/95063054811332615-node1_core1-n_0000000004
[junit4] 2> 10721 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.SolrZkClient makePath:
/collections/collection1/leader_elect/shard1/election
[junit4] 2> 10722 INFO
(OverseerStateUpdate-95063054811332616-127.0.0.1:60416_solr-n_0000000002) [
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node2",
[junit4] 2> "core":"core4",
[junit4] 2> "core_node_name":"node2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node2/solr/"} current state version: 6
[junit4] 2> 10723 INFO
(OverseerStateUpdate-95063054811332616-127.0.0.1:60416_solr-n_0000000002) [
] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node2",
[junit4] 2> "core":"core4",
[junit4] 2> "core_node_name":"node2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node2/solr/"}
[junit4] 2> 10723 INFO
(OverseerStateUpdate-95063054811332616-127.0.0.1:60416_solr-n_0000000002) [
] o.a.s.c.o.ReplicaMutator shard=shard1 is already registered
[junit4] 2> 10728 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.LeaderElector Joined leadership election with path:
/collections/collection1/leader_elect/shard1/election/95063054811332617-node2_core4-n_0000000007
[junit4] 2> 10733 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.LeaderElector Watching path
/collections/collection1/leader_elect/shard1/election/95063054811332615-node1_core1-n_0000000006
to know if I could be the leader
[junit4] 2> 10735 INFO (zkCallback-11-thread-2) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 2)
[junit4] 2> 10735 INFO (zkCallback-2-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 2)
[junit4] 2> 10735 INFO (zkCallback-14-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 2)
[junit4] 2> 10735 INFO (zkCallback-12-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 2)
[junit4] 2> 10740 INFO (zkCallback-14-thread-2) [ ]
o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
[junit4] 2> 10746 INFO
(OverseerStateUpdate-95063054811332616-127.0.0.1:60416_solr-n_0000000002) [
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "base_url":"http://node2/solr/";,
[junit4] 2> "core":"core4",
[junit4] 2> "state":"active"} current state version: 6
[junit4] 2> 10855 INFO (zkCallback-12-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 1)
[junit4] 2> 10855 INFO (zkCallback-2-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 1)
[junit4] 2> 10856 INFO (zkCallback-14-thread-2) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 1)
[junit4] 2> 10943 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 10944 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 10946 INFO (zkCallback-15-thread-1) [ ]
o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@1bbe65a name:ZooKeeperConnection
Watcher:127.0.0.1:60416/solr got event WatchedEvent state:SyncConnected
type:None path:null path:null type:None
[junit4] 2> 10946 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 10947 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 10952 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 10960 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.SolrZkClient makePath: /live_nodes/node1
[junit4] 2> 10962 INFO (zkCallback-14-thread-2) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 1)
[junit4] 2> 10963 INFO (zkCallback-12-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 1)
[junit4] 2> 10963 INFO (zkCallback-2-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 1)
[junit4] 2> 10964 INFO (zkCallback-15-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 1)
[junit4] 2> 10968 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.SolrZkClient makePath:
/collections/collection1/leader_elect/shard1/election
[junit4] 2> 10969 INFO
(OverseerStateUpdate-95063054811332616-127.0.0.1:60416_solr-n_0000000002) [
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"} current state version: 7
[junit4] 2> 10969 INFO (Thread-5) [ ] o.a.s.c.OverseerTest Killing
overseer.
[junit4] 2> 10971 INFO
(OverseerStateUpdate-95063054811332616-127.0.0.1:60416_solr-n_0000000002) [
] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 10971 INFO
(OverseerStateUpdate-95063054811332616-127.0.0.1:60416_solr-n_0000000002) [
] o.a.s.c.o.ReplicaMutator shard=shard1 is already registered
[junit4] 2> 10974 INFO (Thread-5) [ ] o.a.s.c.c.SolrZkClient Using
default ZkCredentialsProvider
[junit4] 2> 10975 INFO (Thread-5) [ ] o.a.s.c.c.ConnectionManager
Waiting for client to connect to ZooKeeper
[junit4] 2> 10977 INFO (zkCallback-16-thread-1) [ ]
o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@5f3c9009
name:ZooKeeperConnection Watcher:127.0.0.1:60416/solr got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 10978 INFO (Thread-5) [ ] o.a.s.c.c.ConnectionManager
Client is connected to ZooKeeper
[junit4] 2> 10978 INFO (Thread-5) [ ] o.a.s.c.c.SolrZkClient Using
default ZkACLProvider
[junit4] 2> 10979 INFO (Thread-5) [ ] o.a.s.c.Overseer Overseer
(id=95063054811332616-127.0.0.1:60416_solr-n_0000000002) closing
[junit4] 2> 10979 INFO
(OverseerStateUpdate-95063054811332616-127.0.0.1:60416_solr-n_0000000002) [
] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:60416_solr
[junit4] 2> 10979 INFO (Thread-5) [ ] o.a.s.u.UpdateShardHandler
Creating UpdateShardHandler HTTP client with params:
socketTimeout=600000&connTimeout=60000&retry=true
[junit4] 2> 10983 ERROR (OverseerExitThread) [ ] o.a.s.c.Overseer could
not read the data
[junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException:
KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at
org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:345)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:342)
[junit4] 2> at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:342)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:305)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:88)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater$2.run(Overseer.java:266)
[junit4] 2> 10983 INFO (Thread-5) [ ] o.a.s.c.Overseer Overseer
(id=null) closing
[junit4] 2> 10985 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.LeaderElector Joined leadership election with path:
/collections/collection1/leader_elect/shard1/election/95063054811332618-node1_core1-n_0000000008
[junit4] 2> 10985 INFO (Thread-5) [ ] o.a.s.c.LeaderElector Joined
leadership election with path:
/overseer_elect/election/95063054811332619-127.0.0.1:60416_solr-n_0000000003
[junit4] 2> 10987 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.LeaderElector Watching path
/collections/collection1/leader_elect/shard1/election/95063054811332617-node2_core4-n_0000000007
to know if I could be the leader
[junit4] 2> 10988 INFO (zkCallback-14-thread-2) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 2)
[junit4] 2> 10988 INFO (zkCallback-15-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 2)
[junit4] 2> 10989 INFO (zkCallback-2-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 2)
[junit4] 2> 10989 INFO (Thread-5) [ ] o.a.s.c.OverseerElectionContext
I am going to be the leader 127.0.0.1:60416_solr
[junit4] 2> 10990 INFO (Thread-5) [ ] o.a.s.c.c.SolrZkClient makePath:
/overseer_elect/leader
[junit4] 2> 10990 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ]
o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from
client sessionid 0x151bb583d4b0009, likely client has closed socket
[junit4] 2> at
org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at
org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 10994 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.SolrZkClient makePath:
/collections/collection1/leader_elect/shard1/election
[junit4] 2> 10999 INFO (Thread-5) [ ] o.a.s.c.Overseer Overseer
(id=95063054811332619-127.0.0.1:60416_solr-n_0000000003) starting
[junit4] 2> 11000 INFO (zkCallback-15-thread-2) [ ]
o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
[junit4] 2> 11007 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.LeaderElector Joined leadership election with path:
/collections/collection1/leader_elect/shard1/election/95063054811332618-node1_core1-n_0000000009
[junit4] 2> 11008 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.LeaderElector was going to be leader
/collections/collection1/leader_elect/shard1/election/95063054811332618-node1_core1-n_0000000009
, seq(0)
/collections/collection1/leader_elect/shard1/election/95063054811332618-node1_core1-n_0000000008
[junit4] 2> 11008 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 11014 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 11018 INFO (zkCallback-18-thread-1) [ ]
o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@dd3c0f3 name:ZooKeeperConnection
Watcher:127.0.0.1:60416/solr got event WatchedEvent state:SyncConnected
type:None path:null path:null type:None
[junit4] 2> 11018 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 11019 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 11020 INFO (Thread-5) [ ]
o.a.s.c.OverseerAutoReplicaFailoverThread Starting
OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000
autoReplicaFailoverWaitAfterExpiration=30000
autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 11021 INFO
(OverseerCollectionConfigSetProcessor-95063054811332619-127.0.0.1:60416_solr-n_0000000003)
[ ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer
operations
[junit4] 2> 11027 INFO
(OverseerStateUpdate-95063054811332619-127.0.0.1:60416_solr-n_0000000003) [
] o.a.s.c.Overseer Starting to work on the main queue
[junit4] 2> 11031 INFO
(OverseerStateUpdate-95063054811332619-127.0.0.1:60416_solr-n_0000000003) [
] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 11032 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 11037 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.SolrZkClient makePath: /live_nodes/node2
[junit4] 2> 11039 INFO
(OverseerStateUpdate-95063054811332619-127.0.0.1:60416_solr-n_0000000003) [
] o.a.s.c.Overseer processMessage: queueSize: 3, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"} current state version: 7
[junit4] 2> 11040 INFO (zkCallback-18-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 1)
[junit4] 2> 11040 INFO (zkCallback-16-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 1)
[junit4] 2> 11040 INFO (zkCallback-2-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 1)
[junit4] 2> 11041 INFO (zkCallback-15-thread-2) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 1)
[junit4] 2> 11042 INFO
(OverseerStateUpdate-95063054811332619-127.0.0.1:60416_solr-n_0000000003) [
] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 11042 INFO
(OverseerStateUpdate-95063054811332619-127.0.0.1:60416_solr-n_0000000003) [
] o.a.s.c.o.ReplicaMutator shard=shard1 is already registered
[junit4] 2> 11046 INFO (zkCallback-16-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 2)
[junit4] 2> 11046 INFO (zkCallback-15-thread-2) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 2)
[junit4] 2> 11047 INFO (zkCallback-2-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 2)
[junit4] 2> 11047 INFO (zkCallback-18-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 2)
[junit4] 2> 11050 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.SolrZkClient makePath:
/collections/collection1/leader_elect/shard1/election
[junit4] 2> 11063 INFO
(OverseerStateUpdate-95063054811332619-127.0.0.1:60416_solr-n_0000000003) [
] o.a.s.c.Overseer processMessage: queueSize: 0, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"} current state version: 8
[junit4] 2> 11065 INFO
(OverseerStateUpdate-95063054811332619-127.0.0.1:60416_solr-n_0000000003) [
] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 11065 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.LeaderElector Joined leadership election with path:
/collections/collection1/leader_elect/shard1/election/95063054811332618-node1_core1-n_0000000010
[junit4] 2> 11066 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.LeaderElector was going to be leader
/collections/collection1/leader_elect/shard1/election/95063054811332618-node1_core1-n_0000000010
, seq(0)
/collections/collection1/leader_elect/shard1/election/95063054811332618-node1_core1-n_0000000008
[junit4] 2> 11065 INFO
(OverseerStateUpdate-95063054811332619-127.0.0.1:60416_solr-n_0000000003) [
] o.a.s.c.o.ReplicaMutator shard=shard1 is already registered
[junit4] 2> 11071 INFO
(OverseerStateUpdate-95063054811332619-127.0.0.1:60416_solr-n_0000000003) [
] o.a.s.c.Overseer processMessage: queueSize: 0, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "base_url":"http://node1/solr/";,
[junit4] 2> "core":"core1",
[junit4] 2> "state":"active"} current state version: 8
[junit4] 2> 11077 INFO
(OverseerStateUpdate-95063054811332619-127.0.0.1:60416_solr-n_0000000003) [
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"} current state version: 8
[junit4] 2> 11078 INFO
(OverseerStateUpdate-95063054811332619-127.0.0.1:60416_solr-n_0000000003) [
] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 11078 INFO
(OverseerStateUpdate-95063054811332619-127.0.0.1:60416_solr-n_0000000003) [
] o.a.s.c.o.ReplicaMutator shard=shard1 is already registered
[junit4] 2> 11183 INFO (zkCallback-18-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 2)
[junit4] 2> 11183 INFO (zkCallback-16-thread-2) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 2)
[junit4] 2> 11184 INFO (zkCallback-2-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 2)
[junit4] 2> 11184 INFO (zkCallback-15-thread-2) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 2)
[junit4] 2> 11283 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.SolrZkClient makePath:
/collections/collection1/leader_elect/shard1/election
[junit4] 2> 11286 INFO
(OverseerStateUpdate-95063054811332619-127.0.0.1:60416_solr-n_0000000003) [
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node2",
[junit4] 2> "core":"core4",
[junit4] 2> "core_node_name":"node2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node2/solr/"} current state version: 9
[junit4] 2> 11287 INFO
(OverseerStateUpdate-95063054811332619-127.0.0.1:60416_solr-n_0000000003) [
] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node2",
[junit4] 2> "core":"core4",
[junit4] 2> "core_node_name":"node2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node2/solr/"}
[junit4] 2> 11287 INFO
(OverseerStateUpdate-95063054811332619-127.0.0.1:60416_solr-n_0000000003) [
] o.a.s.c.o.ReplicaMutator shard=shard1 is already registered
[junit4] 2> 11292 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.LeaderElector Joined leadership election with path:
/collections/collection1/leader_elect/shard1/election/95063054811332620-node2_core4-n_0000000011
[junit4] 2> 11294 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.LeaderElector Watching path
/collections/collection1/leader_elect/shard1/election/95063054811332618-node1_core1-n_0000000010
to know if I could be the leader
[junit4] 2> 11295 INFO (zkCallback-16-thread-2) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 2)
[junit4] 2> 11295 INFO (zkCallback-18-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 2)
[junit4] 2> 11295 INFO (zkCallback-2-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 2)
[junit4] 2> 11295 INFO (zkCallback-15-thread-2) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 2)
[junit4] 2> 11300 INFO (zkCallback-18-thread-1) [ ]
o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
[junit4] 2> 11308 INFO
(OverseerStateUpdate-95063054811332619-127.0.0.1:60416_solr-n_0000000003) [
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "base_url":"http://node2/solr/";,
[junit4] 2> "core":"core4",
[junit4] 2> "state":"active"} current state version: 9
[junit4] 2> 11414 INFO (zkCallback-18-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 1)
[junit4] 2> 11414 INFO (zkCallback-2-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 1)
[junit4] 2> 11415 INFO (zkCallback-16-thread-2) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 1)
[junit4] 2> 12795 WARN (zkCallback-15-thread-2) [ ]
o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 12798 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 12799 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 12801 INFO (zkCallback-19-thread-1) [ ]
o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@5fd70f7e
name:ZooKeeperConnection Watcher:127.0.0.1:60416/solr got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 12801 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 12802 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 12805 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 12810 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.SolrZkClient makePath: /live_nodes/node1
[junit4] 2> 12812 INFO (zkCallback-18-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 1)
[junit4] 2> 12812 INFO (zkCallback-2-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 1)
[junit4] 2> 12812 INFO (zkCallback-16-thread-2) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 1)
[junit4] 2> 12813 INFO (zkCallback-19-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 1)
[junit4] 2> 12817 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.SolrZkClient makePath:
/collections/collection1/leader_elect/shard1/election
[junit4] 2> 12818 INFO
(OverseerStateUpdate-95063054811332619-127.0.0.1:60416_solr-n_0000000003) [
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"} current state version: 10
[junit4] 2> 12819 INFO
(OverseerStateUpdate-95063054811332619-127.0.0.1:60416_solr-n_0000000003) [
] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 12819 INFO
(OverseerStateUpdate-95063054811332619-127.0.0.1:60416_solr-n_0000000003) [
] o.a.s.c.o.ReplicaMutator shard=shard1 is already registered
[junit4] 2> 12826 INFO (Thread-5) [ ] o.a.s.c.OverseerTest Killing
overseer.
[junit4] 2> 12828 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ]
o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from
client sessionid 0x151bb583d4b000b, likely client has closed socket
[junit4] 2> at
org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at
org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 12828 INFO (Thread-5) [ ] o.a.s.c.c.SolrZkClient Using
default ZkCredentialsProvider
[junit4] 2> 12829 INFO (Thread-5) [ ] o.a.s.c.c.ConnectionManager
Waiting for client to connect to ZooKeeper
[junit4] 2> 12829 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.LeaderElector Joined leadership election with path:
/collections/collection1/leader_elect/shard1/election/95063054811332621-node1_core1-n_0000000012
[junit4] 2> 12831 INFO (zkCallback-20-thread-1) [ ]
o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@dfa0111 name:ZooKeeperConnection
Watcher:127.0.0.1:60416/solr got event WatchedEvent state:SyncConnected
type:None path:null path:null type:None
[junit4] 2> 12831 INFO (Thread-5) [ ] o.a.s.c.c.ConnectionManager
Client is connected to ZooKeeper
[junit4] 2> 12832 INFO (Thread-5) [ ] o.a.s.c.c.SolrZkClient Using
default ZkACLProvider
[junit4] 2> 12832 INFO (Thread-5) [ ] o.a.s.c.Overseer Overseer
(id=95063054811332619-127.0.0.1:60416_solr-n_0000000003) closing
[junit4] 2> 12832 INFO
(OverseerStateUpdate-95063054811332619-127.0.0.1:60416_solr-n_0000000003) [
] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:60416_solr
[junit4] 2> 12832 INFO (Thread-5) [ ] o.a.s.u.UpdateShardHandler
Creating UpdateShardHandler HTTP client with params:
socketTimeout=600000&connTimeout=60000&retry=true
[junit4] 2> 12833 INFO
(TEST-OverseerTest.testShardLeaderChange-seed#[1928F325F304463C]) [ ]
o.a.s.c.LeaderElector Watching path
/collections/collection1/leader_elect/shard1/election/95063054811332620-node2_core4-n_0000000011
to know if I could be t
[...truncated too long message...]
collection":"collection2",
[junit4] 2> "base_url":"http://127.0.0.1/solr/";,
[junit4] 2> "core":"core2",
[junit4] 2> "state":"active"} current state version: 6
[junit4] 2> 65397 INFO
(OverseerStateUpdate-95063058433441795-127.0.0.1:51416_solr-n_0000000000) [
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core3",
[junit4] 2> "core_node_name":"node3",
[junit4] 2> "collection":"collection2",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"} current state version: 6
[junit4] 2> 65398 INFO
(OverseerStateUpdate-95063058433441795-127.0.0.1:51416_solr-n_0000000000) [
] o.a.s.c.o.ReplicaMutator Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core3",
[junit4] 2> "core_node_name":"node3",
[junit4] 2> "collection":"collection2",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 65398 INFO
(OverseerStateUpdate-95063058433441795-127.0.0.1:51416_solr-n_0000000000) [
] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=3
[junit4] 2> 65398 INFO
(OverseerStateUpdate-95063058433441795-127.0.0.1:51416_solr-n_0000000000) [
] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 65501 INFO (zkCallback-112-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 1)
[junit4] 2> 65501 INFO (zkCallback-113-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 1)
[junit4] 2> 65501 INFO (zkCallback-115-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 1)
[junit4] 2> 65897 INFO
(TEST-OverseerTest.testBadQueueItem-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.SolrZkClient makePath:
/collections/collection2/leader_elect/shard1/election
[junit4] 2> 65901 INFO
(TEST-OverseerTest.testBadQueueItem-seed#[1928F325F304463C]) [ ]
o.a.s.c.LeaderElector Joined leadership election with path:
/collections/collection2/leader_elect/shard1/election/95063058433441796-127.0.0.1_core3-n_0000000000
[junit4] 2> 65902 INFO
(TEST-OverseerTest.testBadQueueItem-seed#[1928F325F304463C]) [ ]
o.a.s.c.c.SolrZkClient makePath: /collections/collection2/leaders/shard1
[junit4] 2> 65904 INFO
(TEST-OverseerTest.testBadQueueItem-seed#[1928F325F304463C]) [ ]
o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
[junit4] 2> 65907 INFO
(OverseerStateUpdate-95063058433441795-127.0.0.1:51416_solr-n_0000000000) [
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection2",
[junit4] 2> "base_url":"http://127.0.0.1/solr/";,
[junit4] 2> "core":"core3",
[junit4] 2> "state":"active"} current state version: 7
[junit4] 2> 66010 INFO (zkCallback-112-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 1)
[junit4] 2> 66011 INFO (zkCallback-113-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 1)
[junit4] 2> 66010 INFO (zkCallback-115-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 1)
[junit4] 2> 66062 INFO (zkCallback-113-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 1)
[junit4] 2> 66062 INFO (zkCallback-115-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 1)
[junit4] 2> 66066 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ]
o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from
client sessionid 0x151bb5915300004, likely client has closed socket
[junit4] 2> at
org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at
org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 67562 WARN (zkCallback-115-thread-1) [ ]
o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 67565 INFO
(TEST-OverseerTest.testBadQueueItem-seed#[1928F325F304463C]) [ ]
o.a.s.c.ZkTestServer connecting to 127.0.0.1:51416 51416
[junit4] 2> 67720 INFO (Thread-15) [ ] o.a.s.c.ZkTestServer connecting
to 127.0.0.1:51416 51416
[junit4] 2> 67721 WARN (Thread-15) [ ] o.a.s.c.ZkTestServer Watch
limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/aliases.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/clusterstate.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/collections
[junit4] 2> 3 /solr/live_nodes
[junit4] 2>
[junit4] 2> 67722 INFO
(TEST-OverseerTest.testBadQueueItem-seed#[1928F325F304463C]) [ ]
o.a.s.SolrTestCaseJ4 ###Ending testBadQueueItem
[junit4] 2> 67722 INFO
(TEST-OverseerTest.testBadQueueItem-seed#[1928F325F304463C]) [ ]
o.a.s.c.Overseer Overseer
(id=95063058433441795-127.0.0.1:51416_solr-n_0000000000) closing
[junit4] 2> 67724 INFO
(OverseerStateUpdate-95063058433441795-127.0.0.1:51416_solr-n_0000000000) [
] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:51416_solr
[junit4] 2> 67746 ERROR (OverseerExitThread) [ ] o.a.s.c.Overseer could
not read the data
[junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException:
KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at
org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:345)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:342)
[junit4] 2> at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:342)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:305)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:88)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater$2.run(Overseer.java:266)
[junit4] 2> 70724 INFO
(SUITE-OverseerTest-seed#[1928F325F304463C]-worker) [ ] o.a.s.SolrTestCaseJ4
###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.OverseerTest_1928F325F304463C-001
[junit4] 2> NOTE: test params are: codec=Lucene54,
sim=RandomSimilarityProvider(queryNorm=true,coord=no): {}, locale=ar_BH,
timezone=Pacific/Efate
[junit4] 2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation
1.7.0_80 (64-bit)/cpus=4,threads=1,free=307868432,total=338165760
[junit4] 2> NOTE: All tests run in this JVM:
[TestComplexPhraseQParserPlugin, OverseerTest]
[junit4] Completed [6/555 (1!)] on J2 in 63.98s, 12 tests, 1 failure, 1
skipped <<< FAILURES!
[...truncated 1735 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/build.xml:794:
The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/build.xml:738:
The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/build.xml:59:
The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build.xml:233:
The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/common-build.xml:526:
The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/lucene/common-build.xml:1456:
The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/lucene/common-build.xml:1010:
There were test failures: 555 suites (7 ignored), 2206 tests, 1 failure, 53
ignored (38 assumptions) [seed: 1928F325F304463C]
Total time: 61 minutes 49 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
