[
https://issues.apache.org/jira/browse/MAPREDUCE-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016060#comment-13016060
]
Devaraj Das commented on MAPREDUCE-2420:
----------------------------------------
This patch handles the case of HDFS token renewals, when the JobTracker's HDFS
is running a different version of HDFS than the one the job is trying to use.
An example of such a job is distcp (where it uses hftp to talk to a different
source cluster to pull data to the cluster where distcp is running).
When the job is submitted, the client requests a delegation token over hftp and
stuffs it in the job. Today, the NameNode doesn't distinguish between hftp and
hdfs accesses, and issues HDFS tokens for both (and the token-kind field in the
token has the value as 'HDFS'). Ideally, that should be fixed to have the
token-kind as HFTP for hftp accesses. We should have the JobTracker handle all
sorts of token renewals, and have a way in which it can look at a token and
decide which protocol to use to talk to the server in question. This includes
HDFS, HFTP, and also HIVE (where the protocol is thrift).
I think this patch is okay for the short term - the JobTracker falls back to
hftp if it couldn't renew a token over hdfs. In the patch, there are a bunch of
white space changes that aren't required. The string comparisons for exception
messages and then instantiating a concrete exception could probably be replaced
with a forname() on the string exception.
When we fix this issue in trunk, please make it more generic on lines similar
to above.
> JobTracker should be able to renew delegation token over HTTP
> -------------------------------------------------------------
>
> Key: MAPREDUCE-2420
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2420
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.20.4
> Reporter: Boris Shkolnik
> Assignee: Boris Shkolnik
> Attachments: MR-2420.patch
>
>
> in case JobTracker has to talk to a NameNode running a different version (RPC
> version mismatch), Jobtracker should be able to fall back to HTTP renewal.
> Example of the case - running distcp between different versions using hfpt.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
