[
https://issues.apache.org/jira/browse/HDFS-5122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13766128#comment-13766128
]
Jing Zhao commented on HDFS-5122:
---------------------------------
The new patch looks pretty good to me. Some further comments:
1.
{code}
+ } else {
+ Conf config = new Conf(conf);
+ this.retryPolicy = RetryPolicies
+ .failoverOnNetworkException(RetryPolicies.TRY_ONCE_THEN_FAIL,
+ config.maxFailoverAttempts, config.failoverSleepBaseMillis,
+ config.failoverSleepMaxMillis);
{code}
Here we use the same configuration properties for DFSClient. Not sure if we
need to create some new configuration properties for
WebHDFSFileSystem?
2.
{code}
+ private synchronized void resetStateToFailOver() {
+ currentNNAddrIndex = (currentNNAddrIndex + 1) % nnAddrs.length;
+ delegationToken = null;
+ hasInitedToken = false;
+ }
{code}
Can we cache the delegationToken for each NN here, and re-fetch it only when
necessary? (Possibly this can be addressed in a separate jira)
3. In the current patch the class HDFSUrlResolver acts as a helper class. So we
can move the static method to DFSUtil and remove the class.
4.
{code}
+ } else if (a.action ==
RetryPolicy.RetryAction.RetryDecision.FAILOVER_AND_RETRY) {
+ resetStateToFailOver();
+ Thread.sleep(a.delayMillis);
+ return;
{code}
We can add a log here. Besides, currently the isIdempotentOrAtMostOnce
parameter for RetryPolicy#shouldRetry is directly set to true. I went through
all the WebHDFSFileSystem methods and looks like it will be fine. If there is
other issue we can also address it later.
> WebHDFS should support logical service names in URIs
> ----------------------------------------------------
>
> Key: HDFS-5122
> URL: https://issues.apache.org/jira/browse/HDFS-5122
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: ha, webhdfs
> Affects Versions: 2.1.0-beta
> Reporter: Arpit Gupta
> Assignee: Haohui Mai
> Attachments: HDFS-5122.001.patch, HDFS-5122.patch
>
>
> For example if the dfs.nameservices is set to arpit
> {code}
> hdfs dfs -ls webhdfs://arpit:50070/tmp
> or
> hdfs dfs -ls webhdfs://arpit/tmp
> {code}
> does not work
> You have to provide the exact active namenode hostname. On an HA cluster
> using dfs client one should not need to provide the active nn hostname
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
