[
https://issues.apache.org/jira/browse/HDFS-14118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16708032#comment-16708032
]
Fengnan Li edited comment on HDFS-14118 at 12/4/18 1:00 AM:
------------------------------------------------------------
[~elgoiri] Thanks for the pointer! Load balancer is a good idea and is more of
server side service discovery here. We don't have load balancer internally
right now since we don't use cloud so it is possible for us to either use other
load balancer tools like HAProxy or Nginx, or we will do client side service
discovery (using Zookeeper). We are evaluating possibilities and I will update
this ticket with some doc later.
[~surendrasingh] If we decide to use the DNS approach, we will need a
RouterResolver to 1) get the list of available routers from DNS, and 2) do
failover on those routers if necessary. We cannot just failover on the DNS
domain name since the order is undetermined.
was (Author: fengnanli):
??In some degree, this is related to HADOOP-15707.??
??There we allow the load balancers knowing the active Routers.??
[~elgoiri] Thanks for the pointer! Load balancer is a good idea and is more of
server side service discovery here. We don't have load balancer internally
right now since we don't use cloud so it is possible for us to either use other
load balancer tools like HAProxy or Nginx, or we will do client side service
discovery (using Zookeeper). We are evaluating possibilities and I will update
this ticket with some doc later.
??[~fengnanli], currenlty it is doing the same right ???
[~surendrasingh] If we decide to use the DNS approach, we will need a
RouterResolver to 1) get the list of available routers from DNS, and 2) do
failover on those routers if necessary. We cannot just failover on the DNS
domain name since the order is undetermined.
> RBF: Use DNS to help resolve routers
> ------------------------------------
>
> Key: HDFS-14118
> URL: https://issues.apache.org/jira/browse/HDFS-14118
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Fengnan Li
> Assignee: Fengnan Li
> Priority: Major
>
> Clients will need to know about routers to talk to the HDFS cluster
> (obviously), and having routers updating (adding/removing) will have to make
> every client change, which is a painful process.
> DNS can be used here to resolve the single domain name clients knows to a
> list of routers in the current config. However, DNS won't be able to consider
> only resolving to the working router based on certain health thresholds.
> There are some ways about how this can be solved. One way is to have a
> separate script to regularly check the status of the router and update the
> DNS records if a router fails the health thresholds. In this way, security
> might be carefully considered for this way. Another way is to have the client
> do the normal connecting/failover after they get the list of routers, which
> requires the change of current failover proxy provider.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
