[
https://issues.apache.org/jira/browse/HDFS-5122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13764788#comment-13764788
]
Jing Zhao commented on HDFS-5122:
---------------------------------
bq. 1* the standby NM should have a redirector HTTP servlet that bounces all
HTTP calls from the standby to the active.
Maybe we do not need to change the NN side here. A WebHdfsFileSystem call can
get StandbyException if it hits the SBN, and we can use it for
WebHdfsFileSystem side's failover. This is the same logic with current
DFSClient's failover and retry mechanism.
bq. 4* If the hostname is a logical name, resolve to any of the NN hosts, do a
cheap FS call using the chosen hostname.
bq. 4.1**If it works cache the chosen hostname and use it for all subsequent FS
operations while successful.
After the cheap call, we cannot guarantee that the subsequence calls can
succeed since the NN failover can happen in between. So I think we can skip the
cheap call here.
The current WebHDFSFileSystem retry logic does not consider the NN failover. So
I think we only need to add the failover (between the 2 NN URL) and retry
logic, and use FailoverOnNetworkExceptionRetry as the retry policy in
WebHDFSFileSystem for HA setup. Looks like this is what Haohui is doing in his
current patch.
> WebHDFS should support logical service names in URIs
> ----------------------------------------------------
>
> Key: HDFS-5122
> URL: https://issues.apache.org/jira/browse/HDFS-5122
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: ha, webhdfs
> Affects Versions: 2.1.0-beta
> Reporter: Arpit Gupta
> Assignee: Haohui Mai
> Attachments: HDFS-5122.patch
>
>
> For example if the dfs.nameservices is set to arpit
> {code}
> hdfs dfs -ls webhdfs://arpit:50070/tmp
> or
> hdfs dfs -ls webhdfs://arpit/tmp
> {code}
> does not work
> You have to provide the exact active namenode hostname. On an HA cluster
> using dfs client one should not need to provide the active nn hostname
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
