[
https://issues.apache.org/jira/browse/MAPREDUCE-2764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13094802#comment-13094802
]
Owen O'Malley commented on MAPREDUCE-2764:
------------------------------------------
The service field was a per-kind specific information that contained
information about which instance of the service generated the token. It is used
in two cases:
1. picking which token to use when opening an rpc connection to the service
(eg. NameNode, JobTracker, Oozie, HBase, ...)
2. connecting to the service to renew or cancel the token
For HDFS delegation tokens the service consists of the host and port of the
NameNode. For HFTP delegation tokens, it would have been ideal if it had always
contained the host and port of the HFTP server. Then all of the mapping would
have been done in the HFTP servlet code as it converted an HDFS token into a
HFTP one. Since the access mechanism is *totally* different for the two types
of tokens, they should be pulled apart into separate kinds so that they can
have different renewal/canceling mechanisms.
Given that 203 and 204 are in production, we can't make incompatible changes,
therefore the HFTP service needs to continue to send and receive HDFS
delegation tokens. Therefore, having the HFTP client manipulate the tokens into
the form that *should* have been sent seems like the best alternative.
> Fix renewal of dfs delegation tokens
> ------------------------------------
>
> Key: MAPREDUCE-2764
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2764
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Fix For: 0.20.205.0
>
> Attachments: MAPREDUCE-2764-2.patch, MAPREDUCE-2764.patch,
> delegation.patch
>
>
> The JT may have issues renewing hftp tokens which disrupt long distcp jobs.
> The problem is the JT's delegation token renewal code is built on brittle
> assumptions. The token's service field contains only the "ip:port" pair.
> The renewal process assumes that the scheme must be hdfs. If that fails due
> to a {{VersionMismatchException}}, it tries https based on another assumption
> that it must be hftp if it's not hdfs. A number of other exceptions, most
> commonly {{IOExceptions}}, can be generated which fouls up the renewal since
> it won't fallback to https.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
