[
https://issues.apache.org/jira/browse/HDFS-14118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yongjun Zhang updated HDFS-14118:
---------------------------------
Description:
In router based federation (RBF), clients will need to know about routers to
talk to the HDFS cluster (obviously), and having routers updating
(adding/removing) will have to make config change in every client, which is a
painful process.
DNS can be used here to resolve the single domain name clients knows to a list
of routers in the current config. However, DNS won't be able to consider only
resolving to the working router based on certain health thresholds.
There are some ways about how this can be solved. One way is to have a separate
script to regularly check the status of the router and update the DNS records
if a router fails the health thresholds. In this way, security might be
carefully considered for this way. Another way is to have the client do the
normal connecting/failover after they get the list of routers, which requires
the change of current failover proxy provider.
See the attached design document for details about the proposed solution.
was:
Clients will need to know about routers to talk to the HDFS cluster
(obviously), and having routers updating (adding/removing) will have to make
every client change, which is a painful process.
DNS can be used here to resolve the single domain name clients knows to a list
of routers in the current config. However, DNS won't be able to consider only
resolving to the working router based on certain health thresholds.
There are some ways about how this can be solved. One way is to have a separate
script to regularly check the status of the router and update the DNS records
if a router fails the health thresholds. In this way, security might be
carefully considered for this way. Another way is to have the client do the
normal connecting/failover after they get the list of routers, which requires
the change of current failover proxy provider.
> Support using DNS to resolve nameservices to IP addresses
> ---------------------------------------------------------
>
> Key: HDFS-14118
> URL: https://issues.apache.org/jira/browse/HDFS-14118
> Project: Hadoop HDFS
> Issue Type: New Feature
> Reporter: Fengnan Li
> Assignee: Fengnan Li
> Priority: Major
> Attachments: DNS testing log, HDFS design doc_ Single domain name for
> clients - Google Docs-1.pdf, HDFS design doc_ Single domain name for clients
> - Google Docs.pdf, HDFS-14118.001.patch, HDFS-14118.002.patch,
> HDFS-14118.003.patch, HDFS-14118.004.patch, HDFS-14118.005.patch,
> HDFS-14118.006.patch, HDFS-14118.007.patch, HDFS-14118.008.patch,
> HDFS-14118.009.patch, HDFS-14118.010.patch, HDFS-14118.011.patch,
> HDFS-14118.012.patch, HDFS-14118.013.patch, HDFS-14118.014.patch,
> HDFS-14118.015.patch, HDFS-14118.016.patch, HDFS-14118.017.patch,
> HDFS-14118.018.patch, HDFS-14118.019.patch, HDFS-14118.020.patch,
> HDFS-14118.021.patch, HDFS-14118.022.patch, HDFS-14118.023.patch,
> HDFS-14118.024.patch, HDFS-14118.patch
>
>
> In router based federation (RBF), clients will need to know about routers to
> talk to the HDFS cluster (obviously), and having routers updating
> (adding/removing) will have to make config change in every client, which is a
> painful process.
> DNS can be used here to resolve the single domain name clients knows to a
> list of routers in the current config. However, DNS won't be able to consider
> only resolving to the working router based on certain health thresholds.
> There are some ways about how this can be solved. One way is to have a
> separate script to regularly check the status of the router and update the
> DNS records if a router fails the health thresholds. In this way, security
> might be carefully considered for this way. Another way is to have the client
> do the normal connecting/failover after they get the list of routers, which
> requires the change of current failover proxy provider.
> See the attached design document for details about the proposed solution.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
