davidyuan1223 opened a new pull request, #6213:
URL: https://github.com/apache/kyuubi/pull/6213
# :mag: Description
## Issue References ๐
<!-- Append the issue number after #. If there is no issue for you to link
create one or -->
<!-- If there are no issues to link, please provide details here. -->
This pull request fixes #6034
## Describe Your Solution ๐ง
Currently, use beeline to connect kyuubiServer with HA mode, the strategy
only support random, this will lead to a high load on the machine. So i make
this pr to support choose strategy.
[description]
First, we need know, beeline connect kyuubiServer dependency on
kyuubi-hive-jdbc, it is isolated from the kyuubi cluster, so the code only
support random choose serverHost from zk node /${namespace}. Because
kyuubi-hive-jdbc is a stateless module, only run once, cannot store var about
get serverHost from zk node.
[Solution]
This pr, we could implement a interface named ChooseServerStrategy to choose
serverHost. I implement two strategy
1. poll: it will create a zk node named ${namespace}-counter, when a beeline
client want connect kyuubiServer, the node will increment 1, use this value to
take the remainder from serverHosts, like counter % serverHost.size, so we
could get a order serverHost
2. random: random get serverHost from serverHosts
3. User Definied Class: implemented the ChooseServerStrategy, then put the
jar to beeline-jars, it can use your strategy to choose serverHost
## Types of changes :bookmark:
<!--- What types of changes does your code introduce? Put an `x` in all the
boxes that apply: -->
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
## Test Plan ๐งช
Test the Strategy in my test Cluster
#### Behavior Without This Pull Request :coffin:
#### Behavior With This Pull Request :tada:
[Use Case]
1. poll: `bin/beeline -u
'jdbc:hive2://xxx:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=kyuubi;zooKeeperStrategy=poll?spark.yarn.queue=root.kylin;spark.app.name=testspark;spark.shuffle.useOldFetchProtocol=true'
-n mfw_hadoop --verbose=true --showNestedErrs=true`
2. random: `bin/beeline -u
'jdbc:hive2://xxx:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=kyuubi;zooKeeperStrategy=random?spark.yarn.queue=root.kylin;spark.app.name=testspark;spark.shuffle.useOldFetchProtocol=true'
-n mfw_hadoop --verbose=true --showNestedErrs=true` or `bin/beeline -u
'jdbc:hive2://xxx:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=kyuubi?spark.yarn.queue=root.kylin;spark.app.name=testspark;spark.shuffle.useOldFetchProtocol=true'
-n mfw_hadoop --verbose=true --showNestedErrs=true`
3. YourStrategy: `bin/beeline -u
'jdbc:hive2://xxx:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=kyuubi;zooKeeperStrategy=xxx.xxx.xxx.XxxChooseServerStrategy?spark.yarn.queue=root.kylin;spark.app.name=testspark;spark.shuffle.useOldFetchProtocol=true'
-n mfw_hadoop --verbose=true --showNestedErrs=true`
[Result: The Cluster have two Server (221,233)]
1. poll:
1.1. zkNode: counterValue
1.2. result:
2. random:
3. YourStrategy(the test case only get the first serverHost):
#### Related Unit Tests
There is no Unit Tests.
---
# Checklist ๐
<!--- Go over all the following points, and put an `x` in all the boxes that
apply. -->
<!--- If you're unsure about any of these, don't hesitate to ask. We're here
to help! -->
- [ ] This patch was not authored or co-authored using [Generative
Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
