[jira] [Commented] (HBASE-26149) Further improvements on ConnectionRegistry implementations
[ https://issues.apache.org/jira/browse/HBASE-26149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17725463#comment-17725463 ] Duo Zhang commented on HBASE-26149: --- Was thinking of introducing an URI as the connnection string for a HBase cluster. For example, zk://xxx:2181/hbase, we will use the zk based registry; hbase://xxx:16010,yyy:16010, we will use the rpc based regsitry; rest://xxx:8080, we will use a rest API to get the registry information(not implemented yet, just saying as I saw [~stack] mentioned that we could introduce a rest API in the design doc. And by using the service loader in java, it will be easier for our users to customized the registry implementation, for example, they can just implement their own registry, for example, k8s service based registry? And then put it in the service loader file, and use a special protocol string in the URI. Thanks. > Further improvements on ConnectionRegistry implementations > -- > > Key: HBASE-26149 > URL: https://issues.apache.org/jira/browse/HBASE-26149 > Project: HBase > Issue Type: Umbrella > Components: Client >Reporter: Duo Zhang >Priority: Major > > (Copied in-line from the attached 'Documentation' with some filler as > connecting script) > HBASE-23324 Deprecate clients that connect to Zookeeper > ^^^ This is always our goal, to remove the zookeeper dependency from the > client side. > > See the sub-task HBASE-25051 DIGEST based auth broken for MasterRegistry > When constructing RpcClient, we will pass the clusterid in, and it will be > used to select the authentication method. More specifically, it will be used > to select the tokens for digest based authentication, please see the code in > BuiltInProviderSelector. For ZKConnectionRegistry, we do not need to use > RpcClient to connect to zookeeper, so we could get the cluster id first, and > then create the RpcClient. But for MasterRegistry/RpcConnectionRegistry, we > need to use RpcClient to connect to the ClientMetaService endpoints and then > we can call the getClusterId method to get the cluster id. Because of this, > when creating RpcClient for MasterRegistry/RpcConnectionRegistry, we can only > pass null or the default cluster id, which means the digest based > authentication is broken. > This is a cyclic dependency problem. Maybe a possible way forward, is to make > getClusterId method available to all users, which means it does not require > any authentication, so we can always call getClusterId with simple > authentication, and then at client side, once we get the cluster id, we > create a new RpcClient to select the correct authentication way. > The work in the sub-task, HBASE-26150 Let region server also carry > ClientMetaService, is work to make it so the RegionServers can carry a > ConnectionRegistry (rather than have the Masters-only carry it as is the case > now). Adds a new method getBootstrapNodes to ClientMetaService, the > ConnectionRegistry proto Service, for refreshing the bootstrap nodes > periodically or on error. The new *RpcConnectionRegistry* [Created here but > defined in the next sub-task]will use this method to refresh the bootstrap > nodes, while the old MasterRegistry will use the getMasters method to refresh > the ‘bootstrap’ nodes. > The getBootstrapNodes method will return all the region servers, so after the > first refreshing, the client will go to region servers for later rpc calls. > But since masters and region servers both implement the ClientMetaService > interface, it is free for the client to configure master as the initial > bootstrap nodes. > The following sub-task then deprecates MasterRegistry, HBASE-26172 Deprecated > MasterRegistry > The implementation of MasterRegistry is almost the same with > RpcConnectionRegistry except that it uses getMasters instead of > getBootstrapNodes to refresh the ‘bootstrap’ nodes connected to. So we could > add configs in server side to control what nodes we want to return to client > in getBootstrapNodes, i.e, master or region server, then the > RpcConnectionRegistry can fully replace the old MasterRegistry. Deprecates > the MasterRegistry. > Sub-task HBASE-26173 Return only a sub set of region servers as bootstrap > nodes > For a large cluster which may have thousands of region servers, it is not a > good idea to return all the region servers as bootstrap nodes to clients. So > we should add a config at server side to control the max number of bootstrap > nodes we want to return to clients. I think the default value could be 5 or > 10, which is enough. > Sub-task HBASE-26174 Make rpc connection registry the default registry on > 3.0.0 > Just a follow up of HBASE-26172. MasterRegistry has been deprecated, we > should not make it default for 3.0.0 any
[jira] [Commented] (HBASE-26149) Further improvements on ConnectionRegistry implementations
[ https://issues.apache.org/jira/browse/HBASE-26149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17396887#comment-17396887 ] Michael Stack commented on HBASE-26149: --- The one-pager helped. Thanks. I put it here as the Jira description copying to sub-tasks description that was in this document bit missing from the sub-task JIRA's desciption. Hopefully makes it easier on others trying to follow-long whats going on here (Put some questions on the document for my own clarification). THanks. > Further improvements on ConnectionRegistry implementations > -- > > Key: HBASE-26149 > URL: https://issues.apache.org/jira/browse/HBASE-26149 > Project: HBase > Issue Type: Umbrella > Components: Client >Reporter: Duo Zhang >Priority: Major > > (Copied in-line from the attached 'Documentation' with some filler as > connecting script) > HBASE-23324 Deprecate clients that connect to Zookeeper > ^^^ This is always our goal, to remove the zookeeper dependency from the > client side. > > See the sub-task HBASE-25051 DIGEST based auth broken for MasterRegistry > When constructing RpcClient, we will pass the clusterid in, and it will be > used to select the authentication method. More specifically, it will be used > to select the tokens for digest based authentication, please see the code in > BuiltInProviderSelector. For ZKConnectionRegistry, we do not need to use > RpcClient to connect to zookeeper, so we could get the cluster id first, and > then create the RpcClient. But for MasterRegistry/RpcConnectionRegistry, we > need to use RpcClient to connect to the ClientMetaService endpoints and then > we can call the getClusterId method to get the cluster id. Because of this, > when creating RpcClient for MasterRegistry/RpcConnectionRegistry, we can only > pass null or the default cluster id, which means the digest based > authentication is broken. > This is a cyclic dependency problem. Maybe a possible way forward, is to make > getClusterId method available to all users, which means it does not require > any authentication, so we can always call getClusterId with simple > authentication, and then at client side, once we get the cluster id, we > create a new RpcClient to select the correct authentication way. > The work in the sub-task, HBASE-26150 Let region server also carry > ClientMetaService, is work to make it so the RegionServers can carry a > ConnectionRegistry (rather than have the Masters-only carry it as is the case > now). Adds a new method getBootstrapNodes to ClientMetaService, the > ConnectionRegistry proto Service, for refreshing the bootstrap nodes > periodically or on error. The new *RpcConnectionRegistry* [Created here but > defined in the next sub-task]will use this method to refresh the bootstrap > nodes, while the old MasterRegistry will use the getMasters method to refresh > the ‘bootstrap’ nodes. > The getBootstrapNodes method will return all the region servers, so after the > first refreshing, the client will go to region servers for later rpc calls. > But since masters and region servers both implement the ClientMetaService > interface, it is free for the client to configure master as the initial > bootstrap nodes. > The following sub-task then deprecates MasterRegistry, HBASE-26172 Deprecated > MasterRegistry > The implementation of MasterRegistry is almost the same with > RpcConnectionRegistry except that it uses getMasters instead of > getBootstrapNodes to refresh the ‘bootstrap’ nodes connected to. So we could > add configs in server side to control what nodes we want to return to client > in getBootstrapNodes, i.e, master or region server, then the > RpcConnectionRegistry can fully replace the old MasterRegistry. Deprecates > the MasterRegistry. > Sub-task HBASE-26173 Return only a sub set of region servers as bootstrap > nodes > For a large cluster which may have thousands of region servers, it is not a > good idea to return all the region servers as bootstrap nodes to clients. So > we should add a config at server side to control the max number of bootstrap > nodes we want to return to clients. I think the default value could be 5 or > 10, which is enough. > Sub-task HBASE-26174 Make rpc connection registry the default registry on > 3.0.0 > Just a follow up of HBASE-26172. MasterRegistry has been deprecated, we > should not make it default for 3.0.0 any more. > Sub-task HBASE-26180 Introduce a initial refresh interval for > RpcConnectionRegistry > As end users could configure any nodes in a cluster as the initial bootstrap > nodes, it is possible that different end users will configure the same > machine which makes the machine over load. So we should have a shorter delay > for the initial refresh, to let users quickly switch to the
[jira] [Commented] (HBASE-26149) Further improvements on ConnectionRegistry implementations
[ https://issues.apache.org/jira/browse/HBASE-26149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17396623#comment-17396623 ] Duo Zhang commented on HBASE-26149: --- Attach a simple doc to explain the goal and approach for all the sub tasks. > Further improvements on ConnectionRegistry implementations > -- > > Key: HBASE-26149 > URL: https://issues.apache.org/jira/browse/HBASE-26149 > Project: HBase > Issue Type: Umbrella > Components: Client >Reporter: Duo Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-26149) Further improvements on ConnectionRegistry implementations
[
https://issues.apache.org/jira/browse/HBASE-26149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17396455#comment-17396455
]
Duo Zhang commented on HBASE-26149:
---
Yes, as [~bharathv] said, there is no problem when we use ZKConnectionRegistry
in the past.
{quote}
RPC-based registry as opposed to which sort of Registry? One where we read from
configs?
{quote}
The design is like this. You need to provide a list of nodes in the config file
first, and once the RpcConnectionRegistry is up, it will call getBootstrapNodes
method to get a list of 'new' bootstrap nodes.
And now, both master and region server has implemented the ClientMetaService
interface, so you are free to provide either a master or a region server or
several of them as the initial nodes in config file, and then the
RpcConnectionRegistry will load new nodes automatically. The benefit here is
that, we can control the nodes we want to return to client, masters, or region
servers, or both, or a sub set of region servers, etc, which means we do not
need to let the end users consider what is the best way to config the initial
nodes. Just config the one you like and make sure you can connect to the
cluster.
Thanks.
> Further improvements on ConnectionRegistry implementations
> --
>
> Key: HBASE-26149
> URL: https://issues.apache.org/jira/browse/HBASE-26149
> Project: HBase
> Issue Type: Umbrella
> Components: Client
>Reporter: Duo Zhang
>Priority: Major
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-26149) Further improvements on ConnectionRegistry implementations
[ https://issues.apache.org/jira/browse/HBASE-26149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17396413#comment-17396413 ] Bharath Vissapragada commented on HBASE-26149: -- > This must have been in place always? [~stack] Please see HBASE-25051's description. Earlier we relied on an authenticated ZK client to get the cluster ID. Now its a chicken and egg problem if we are removing ZK from the picture. > Further improvements on ConnectionRegistry implementations > -- > > Key: HBASE-26149 > URL: https://issues.apache.org/jira/browse/HBASE-26149 > Project: HBase > Issue Type: Umbrella > Components: Client >Reporter: Duo Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-26149) Further improvements on ConnectionRegistry implementations
[ https://issues.apache.org/jira/browse/HBASE-26149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17396385#comment-17396385 ] Michael Stack commented on HBASE-26149: --- Thanks. This is interesting => " This is a cyclic dependency" (This must have been in place always?) Whats the refresh of? Registry content? Master hosts Resistry currently. Do these changes make it so RS can host Registry? Or is it just the mechanics that is being moved over? RPC-based registry as opposed to which sort of Registry? One where we read from configs? Thanks. > Further improvements on ConnectionRegistry implementations > -- > > Key: HBASE-26149 > URL: https://issues.apache.org/jira/browse/HBASE-26149 > Project: HBase > Issue Type: Umbrella > Components: Client >Reporter: Duo Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-26149) Further improvements on ConnectionRegistry implementations
[ https://issues.apache.org/jira/browse/HBASE-26149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17396349#comment-17396349 ] Duo Zhang commented on HBASE-26149: --- Basically there are two directions. One is how to deal with rpc, as now we rely on rpc to get cluster id, but we rely on cluster I’d to determine the authentication way of rpc. This is a cyclic dependency, in rpc based connection registry we just ignore cluster id and use the default authentication way. We need to find a way to deal with this. Another is improve the usability. Including region server based end point, allow disable refreshing, allow a short refresh interval for the initial refresh, etc. > Further improvements on ConnectionRegistry implementations > -- > > Key: HBASE-26149 > URL: https://issues.apache.org/jira/browse/HBASE-26149 > Project: HBase > Issue Type: Umbrella > Components: Client >Reporter: Duo Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-26149) Further improvements on ConnectionRegistry implementations
[ https://issues.apache.org/jira/browse/HBASE-26149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17396263#comment-17396263 ] Michael Stack commented on HBASE-26149: --- Is there an overarching intent here [~zhangduo] or just random improvements? Thanks. > Further improvements on ConnectionRegistry implementations > -- > > Key: HBASE-26149 > URL: https://issues.apache.org/jira/browse/HBASE-26149 > Project: HBase > Issue Type: Umbrella > Components: Client >Reporter: Duo Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
