[
https://issues.apache.org/jira/browse/HDFS-7036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14146989#comment-14146989
]
Haohui Mai commented on HDFS-7036:
----------------------------------
bq. ...adding the hack in distcp and other applications...
Can you point out what the other applications are that need to access both
secure and insecure cluster at the same time? To my best knowledge, distcp is
the only use case. The title of this jira suggests it is focusing on distcp.
In contrast, YARN, MR and other applications that are built on top of them use
{{WebHdfsFileSystem}} extensively.
My message has been consistent since HDFS-6776 -- the hack should be contained
which result minimal damage in the codebase. I cannot +1 for the approach on
hacking {{WebHdfsFileSystem}} just for this issue.
If you still don't get it, it might be helpful to go through the distcp code
first.
> HDFS-6776 fix requires to upgrade insecure cluster, which means quite some
> user pain
> ------------------------------------------------------------------------------------
>
> Key: HDFS-7036
> URL: https://issues.apache.org/jira/browse/HDFS-7036
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: webhdfs
> Affects Versions: 2.5.1
> Reporter: Yongjun Zhang
> Assignee: Yongjun Zhang
> Attachments: HDFS-7036.001.patch
>
>
> Issuing command
> {code}
> hadoop fs -lsr webhdfs://<insecureCluster>
> {code}
> at a secure cluster side fails with message "Failed to get the token ...",
> similar symptom as reported in HDFS-6776.
> If the fix of HDFS-6776 is applied to only the secure cluster, doing
> {code}
> distcp webhdfs://<insecureCluster> <secureCluster>
> {code}
> would fail same way.
> Basically running any application in secure cluster to access insecure
> cluster via webhdfs would fail the same way, if the HDFS-6776 fix is not
> applied to the insecure cluster.
> This could be quite some user pain. Filing this jira for a solution to make
> user's life easier.
> One proposed solution was to add a msg-parsing mechanism in webhdfs, which is
> a bit hacky. The other proposed solution is to do the same kind of hack at
> application side, which means the same hack need to be applied in each
> application.
> Thanks [~daryn], [~wheat9], [~jingzhao], [~tucu00] and [~atm] for the
> discussion in HDFS-6776.
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
