[
https://issues.apache.org/jira/browse/HDFS-14118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16775474#comment-16775474
]
Yongjun Zhang edited comment on HDFS-14118 at 2/22/19 6:33 PM:
---------------------------------------------------------------
Hi [~fengnanli],
Thanks for following-up. Would like to suggest some small changes in the config
description. Hope the suggested changes make sense to you:
{code:java}
<property>
<name>dfs.client.failover.random.order</name>
<value>false</value>
<description>
Determines if the failover proxies are picked in random order instead of the
configured order. Random order may be enabled for better load balancing
or to avoid always hitting failed ones first if the failed ones appear in the
beginning of the configured or resolved list.
For example, In the case of multiple RBF routers or ObserverNameNodes,
it is recommended to be turned on for load balancing.
The config name can be extended with an optional nameservice ID
(of form dfs.client.failover.random.order[.nameservice]) in case multiple
nameservices exist and random order should be enabled for specific
nameservices.
</description>
</property>
<property>
<name>dfs.client.failover.resolve-needed</name>
<value>false</value>
<description>
Determines if the given nameservice address is a domain name which needs to
be resolved (using the resolver configured by
dfs.client.failover.resolver-impl).
This adds a transparency layer in the client so physical server address
can change without changing the client. The config name can be extended with
an optional nameservice ID (of form
dfs.client.failover.resolve-needed[.nameservice])
to configure specific nameservices when multiple nameservices exist.
</description>
</property>
<property>
<name>dfs.client.failover.resolver.impl</name>
<value>org.apache.hadoop.net.DNSDomainNameResolver</value>
<description>
Determines what class to use to resolve name service domain name to specific
machine address. The config name can be extended with an optional
nameservice ID (of form dfs.client.failover.resolver.impl[.nameservice]) to
configure
specific nameservices when multiple nameservices exist.
</description>
</property>
{code}
was (Author: yzhangal):
Hi [~fengnanli],
Thanks for following-up. Would like to suggest some small changes in the config
description. Hope the suggested changes make sense to you:
{code:java}
<property>
<name>dfs.client.failover.random.order</name>
<value>false</value>
<description>
Determines if the failover proxies are picked in random order instead of the
configured order. Random order may be enabled for better load balancing
or to avoid always hitting failed ones first if the failed ones appear in the
beginning of the configured or resolved list.
For example, In the case of multiple RBF routers or ObserverNameNodes,
it is recommended to be turned on for load balancing.
The config name can be extended with an optional nameservice ID
(of form dfs.client.failover.random.order[.nameservice]) in case multiple
nameservices exist and random order should be enabled for specific
nameservices.
</description>
</property>
<property>
<name>dfs.client.failover.resolve-needed</name>
<value>false</value>
<description>
Determines if the given namenode address is a domain name which needs to
be resolved (using the resolver configured by
dfs.client.failover.resolver-impl).
This adds a transparency layer in the client so physical namenode address
can change without changing the client. The config name can be extended with
an optional nameservice ID (of form
dfs.client.failover.resolve-needed[.nameservice])
to configure specific nameservices when multiple nameservices exist.
</description>
</property>
<property>
<name>dfs.client.failover.resolver.impl</name>
<value>org.apache.hadoop.net.DNSDomainNameResolver</value>
<description>
Determines what class to use to resolve name service domain name to specific
machine address. The config name can be extended with an optional
nameservice ID (of form dfs.client.failover.resolver.impl[.nameservice]) to
configure
specific nameservices when multiple nameservices exist.
</description>
</property>
{code}
> Use DNS to resolve Namenodes and Routers
> ----------------------------------------
>
> Key: HDFS-14118
> URL: https://issues.apache.org/jira/browse/HDFS-14118
> Project: Hadoop HDFS
> Issue Type: New Feature
> Reporter: Fengnan Li
> Assignee: Fengnan Li
> Priority: Major
> Attachments: DNS testing log, HDFS design doc_ Single domain name for
> clients - Google Docs-1.pdf, HDFS design doc_ Single domain name for clients
> - Google Docs.pdf, HDFS-14118.001.patch, HDFS-14118.002.patch,
> HDFS-14118.003.patch, HDFS-14118.004.patch, HDFS-14118.005.patch,
> HDFS-14118.006.patch, HDFS-14118.007.patch, HDFS-14118.008.patch,
> HDFS-14118.009.patch, HDFS-14118.010.patch, HDFS-14118.011.patch,
> HDFS-14118.012.patch, HDFS-14118.013.patch, HDFS-14118.014.patch,
> HDFS-14118.015.patch, HDFS-14118.016.patch, HDFS-14118.017.patch,
> HDFS-14118.018.patch, HDFS-14118.019.patch, HDFS-14118.020.patch,
> HDFS-14118.021.patch, HDFS-14118.022.patch, HDFS-14118.patch
>
>
> Clients will need to know about routers to talk to the HDFS cluster
> (obviously), and having routers updating (adding/removing) will have to make
> every client change, which is a painful process.
> DNS can be used here to resolve the single domain name clients knows to a
> list of routers in the current config. However, DNS won't be able to consider
> only resolving to the working router based on certain health thresholds.
> There are some ways about how this can be solved. One way is to have a
> separate script to regularly check the status of the router and update the
> DNS records if a router fails the health thresholds. In this way, security
> might be carefully considered for this way. Another way is to have the client
> do the normal connecting/failover after they get the list of routers, which
> requires the change of current failover proxy provider.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
