[
https://issues.apache.org/jira/browse/HDFS-7036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14146861#comment-14146861
]
Haohui Mai commented on HDFS-7036:
----------------------------------
My position has not changed since HDFS-6776. Just to recap:
# This use case is so specific to distcp that it should be fixed distcp instead
of the underlying filesystem implementation.
# This is a hack for compatibility. There are many more users of
{{WebHdfsFileSystem}} compared to distcp. It is more reasonable to contain the
changes at higher layers (i.e. distcp) to avoid surprises to other applications.
I understand that hacking {{WebHdfsFileSystem}} is an easy enough fix, I also
understand that a hack might be needed either here or there, but parsing the
exception message in {{WebHdfsFileSystem}} does not seem the right solution
here.
If it turns out that this type of hack is unavoidable, I suggest doing it in
distcp.
> HDFS-6776 fix requires to upgrade insecure cluster, which means quite some
> user pain
> ------------------------------------------------------------------------------------
>
> Key: HDFS-7036
> URL: https://issues.apache.org/jira/browse/HDFS-7036
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: webhdfs
> Affects Versions: 2.5.1
> Reporter: Yongjun Zhang
> Assignee: Yongjun Zhang
> Attachments: HDFS-7036.001.patch
>
>
> Issuing command
> {code}
> hadoop fs -lsr webhdfs://<insecureCluster>
> {code}
> at a secure cluster side fails with message "Failed to get the token ...",
> similar symptom as reported in HDFS-6776.
> If the fix of HDFS-6776 is applied to only the secure cluster, doing
> {code}
> distcp webhdfs://<insecureCluster> <secureCluster>
> {code}
> would fail same way.
> Basically running any application in secure cluster to access insecure
> cluster via webhdfs would fail the same way, if the HDFS-6776 fix is not
> applied to the insecure cluster.
> This could be quite some user pain. Filing this jira for a solution to make
> user's life easier.
> One proposed solution was to add a msg-parsing mechanism in webhdfs, which is
> a bit hacky. The other proposed solution is to do the same kind of hack at
> application side, which means the same hack need to be applied in each
> application.
> Thanks [~daryn], [~wheat9], [~jingzhao], [~tucu00] and [~atm] for the
> discussion in HDFS-6776.
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
