[
https://issues.apache.org/jira/browse/HDFS-5122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13764707#comment-13764707
]
Alejandro Abdelnur commented on HDFS-5122:
------------------------------------------
I would take a slightly different approach for this:
The WebHdfsFileSystem#getHttpUrlConnection() method is used for every FS call.
In this method, if the URI is for the NN (metadata operation, data transfers
will be to a DN) do the following:
1* the standby NM should have a redirector HTTP servlet that bounces all HTTP
calls from the standby to the active.
2* Use the same utility classes DistributedFileSystem class uses to determine
if hostname in the URI is a logical name or not.
3* If the hostname is no a logical name, do current logic.
4* If the hostname is a logical name, resolve to any of the NN hosts, do a
cheap FS call using the chosen hostname.
4.1**If it works cache the chosen hostname and use it for all subsequent FS
operations while successful.
4.2** If the call returns a redirect (automatic redirects are disabled) means
you hit the standby, select the other hostname and use it for all subsequent FS
operations while successful.
4.3** if the call returns a cannot connect or error means you hit a NN that is
'dead', fallback to the other NN hostname and use it for all subsequent FS
operations while successful.
5* When a subsequent URL call fails do #4.3
6* Make sure you have logic to avoid infinite loop of bouncing between NNs in
case both are dead.
7*The WebHDFS delegation token service should use similar logic like the DFS
delegation token to convert from logical name to hostname the service in the
token.
NOTE: I'm not familiar on WebHDFSFileSystem current retry logic, and some of
this could be already take can of it.
> WebHDFS should support logical service names in URIs
> ----------------------------------------------------
>
> Key: HDFS-5122
> URL: https://issues.apache.org/jira/browse/HDFS-5122
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: ha, webhdfs
> Affects Versions: 2.1.0-beta
> Reporter: Arpit Gupta
> Assignee: Haohui Mai
> Attachments: HDFS-5122.patch
>
>
> For example if the dfs.nameservices is set to arpit
> {code}
> hdfs dfs -ls webhdfs://arpit:50070/tmp
> or
> hdfs dfs -ls webhdfs://arpit/tmp
> {code}
> does not work
> You have to provide the exact active namenode hostname. On an HA cluster
> using dfs client one should not need to provide the active nn hostname
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
