[
https://issues.apache.org/jira/browse/PHOENIX-3654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15939837#comment-15939837
]
ASF GitHub Bot commented on PHOENIX-3654:
-----------------------------------------
GitHub user rahulsIOT opened a pull request:
https://github.com/apache/phoenix/pull/236
Loadbalancer
Hi All,
https://issues.apache.org/jira/browse/PHOENIX-3654
The following changes are made in the pull request
1. Added project phoenix-load-balancer. The project exposes Loadbalancer
mechanism for thin client.
2. The following additional phoenix parameters is exposed in
LoadBalancerConfiguration. These configuration is read from hbase-site.xml. The
configuration are as follows
clusterName = "phoenix.queryserver.base.path"
serviceName = "phoenix.queryserver.service.name"
defaultClusterName = "phoenix"
defaultServiceName = "queryserver"
zkLbUserName = "phoenix.queryserver.zookeeper.acl.username"
zkLbPassword = "phoenix.queryserver.zookeeper.acl.password"
defaultZkLbUserName = "phoenixuser"
defaultZkLbPassword = "Xsjdhxsd"
3. phoenix-load-balancer is not dependent on phoenix core, which was one of
requirement for the project. It has dependencies on apache-curator and
hbase-common for hbase configuration.
4. The project phoenix-queryserver is modified such that when server loads
up, it will find zookeeper configuration from hbase-site.xml and registers
itself to the configuration. The registered node will have patch
/clusterName/serviceName/hostname_port (e.g. /phoenix/queryserver/host1_1234).
This will be emphimaral node and gets removed when session has ended. With each
ephimeral node, there is a json attached. The Json is of the form
{"host":"host1","port":"1234"}. In future this can be expanded to store more
values for the node. The Node has ACL = digest with username:password as id.
There is default username/password shared between Loadbalancer project and
phoenix query server project. User can specify the username/password as
properties
"phoenix.queryserver.zookeeper.acl.username"/"phoenix.queryserver.zookeeper.acl.password"
in hbase-site.xml.
5. For phoenix-queryserver , there is a dependency of
phoenix-load-balancer. This is needed to load configuration related to only
LoadBalancer. To avoid code duplication, we have kept all the configuration for
LoadBalancer in the file LoadBalancerConfiguration.java within project
phoenix-load-balancer.
6. IT tests cases were written for load balancer.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/rahulsIOT/phoenix Loadbalancer
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/phoenix/pull/236.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #236
----
commit 8bff60831e55d2d2c1e4fffc6e662dc25fe5be0a
Author: Rahul <rshrivast...@salesforce.com>
Date: 2017-03-01T23:38:52Z
Initial set of files for Load Balancer
commit 4f2422562afc59106a02e82eb9672ac9d6429bbd
Author: Rahul <rshrivast...@salesforce.com>
Date: 2017-03-01T23:42:45Z
pom for load balancer project
commit 8951a7c24da1be331fc43ee4986ff5f33435616f
Author: Rahul <rshrivast...@salesforce.com>
Date: 2017-03-02T01:57:41Z
added code for service discovery
commit 2fd2e82667af5157f0e5a5bb12e7aec6319dc430
Author: Rahul <rshrivast...@salesforce.com>
Date: 2017-03-04T02:11:16Z
added registration code to query server
commit 55a59388fe523d7340440b1736fce400f4fe9429
Author: Rahul <rshrivast...@salesforce.com>
Date: 2017-03-08T21:44:05Z
added extra changes for including tests
commit 6420f7348e8a313b0c18d969654bbb49632c4d48
Author: Rahul <rshrivast...@salesforce.com>
Date: 2017-03-13T17:10:51Z
still making changes to initial commit
commit 8e712b4bbc52524554cd4ed3ca5ec923d99fdb46
Author: Rahul <rshrivast...@salesforce.com>
Date: 2017-03-14T18:03:03Z
Fixed naming of variables along with other cosmetic fixes
commit a7404435abaa2af5ad284972933d0ecb2cb94f45
Author: Rahul <rshrivast...@salesforce.com>
Date: 2017-03-21T07:07:18Z
fixed all the IT cases in LoadBalancer
commit 5a9b7021dc8d51884f546973db633b6ca0ee59e5
Author: Rahul <rshrivast...@salesforce.com>
Date: 2017-03-24T04:58:05Z
completed the initial patch for load balancer
commit d3d8bea7299dbb6830bbdf2925fc43512599df71
Author: Rahul <rshrivast...@salesforce.com>
Date: 2017-03-24T05:11:36Z
removed unneeded sections of pom.xml including curator version. This
version is in parent pom.xml
----
> Load Balancer for thin client
> -----------------------------
>
> Key: PHOENIX-3654
> URL: https://issues.apache.org/jira/browse/PHOENIX-3654
> Project: Phoenix
> Issue Type: New Feature
> Affects Versions: 4.8.0
> Environment: Linux 3.13.0-107-generic kernel, v4.9.0-HBase-0.98
> Reporter: Rahul Shrivastava
> Fix For: 4.9.0
>
> Attachments: LoadBalancerDesign.pdf
>
> Original Estimate: 240h
> Remaining Estimate: 240h
>
> We have been having internal discussion on load balancer for thin client for
> PQS. The general consensus we have is to have an embedded load balancer with
> the thin client instead of using external load balancer such as haproxy. The
> idea is to not to have another layer between client and PQS. This reduces
> operational cost for system, which currently leads to delay in executing
> projects.
> But this also comes with challenge of having an embedded load balancer which
> can maintain sticky sessions, do fair load balancing knowing the load
> downstream of PQS server. In addition, load balancer needs to know location
> of multiple PQS server. Now, the thin client needs to keep track of PQS
> servers via zookeeper ( or other means).
> In the new design, the client ( PQS client) , it is proposed to have an
> embedded load balancer.
> Where will the load Balancer sit ?
> The load load balancer will embedded within the app server client.
> How will the load balancer work ?
> Load balancer will contact zookeeper to get location of PQS. In this case,
> PQS needs to register to ZK itself once it comes online. Zookeeper location
> is in hbase-site.xml. It will maintain a small cache of connection to the
> PQS. When a request comes in, it will check for an open connection from the
> cache.
> How will load balancer know load on PQS ?
> To start with, it will pick a random open connection to PQS. This means that
> load balancer does not know PQS load. Later , we can augment the code so that
> thin client can receive load info from PQS and make intelligent decisions.
> How will load balancer maintain sticky sessions ?
> While we still need to investigate how to implement sticky sessions. We can
> look for some open source implementation for the same.
> How will PQS register itself to service locator ?
> PQS will have location of zookeeper in hbase-site.xml and it would register
> itself to the zookeeper. Thin client will find out PQS location using
> zookeeper.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
