[
https://issues.apache.org/jira/browse/MAPREDUCE-2764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13080014#comment-13080014
]
Daryn Sharp commented on MAPREDUCE-2764:
----------------------------------------
bq. The token is used for rpc connection and when token selector does a match
it needs the service to have rpc port. But if we store hftp uri in the token,
the port will be http port. How is that being handled?
The {{getServiceAuthority()}} is extracting the authority from the service if
it's a uri, else it uses the service itself if it's not a uri.
bq. One approach could be to store both issuer and service in the current
service field.
The the service is currently "authority", and I changed it to be either
"authority" or "scheme://authority". I'd like to eventually move everything to
"scheme://authority" hence the compatibility.
bq. I think the AbstractDelegationtionTokenSelector change and introduction of
TokenMatcher is unnecessary. BlockTokenSelector or JobTokenSelector shouldn't
need to change at all.
That means the token selectors go back to copy-n-paste... The base
{{selectTokens()}} method filters the tokens to find the expected token kind.
This eliminates the copy-n-paste code which limits the ability to alter the
standard behavior.
The {{TokenMatcher}} is used to allow tokens, if those so chose, to provide
custom selection criteria above and beyond the standard service match. This is
also used to match a RPC socket to its token, else it just adds even more
copy-n-paste.
{{JobTokenSelector}} only removed methods now in the base class.
{{BlockTokenSelector}} oddly doesn't care about service at all, so it's
{{selectToken()}} is overridden to provide a {{TokenMatcher}} that ignores
service.
bq. A change in hdfs's DelegationTokenSelector should suffice for this jira.
Iff we are going back to copy-n-paste...
> Fix renewal of dfs delegation tokens
> ------------------------------------
>
> Key: MAPREDUCE-2764
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2764
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Fix For: 0.20.205.0
>
> Attachments: MAPREDUCE-2764.patch
>
>
> The JT may have issues renewing hftp tokens which disrupt long distcp jobs.
> The problem is the JT's delegation token renewal code is built on brittle
> assumptions. The token's service field contains only the "ip:port" pair.
> The renewal process assumes that the scheme must be hdfs. If that fails due
> to a {{VersionMismatchException}}, it tries https based on another assumption
> that it must be hftp if it's not hdfs. A number of other exceptions, most
> commonly {{IOExceptions}}, can be generated which fouls up the renewal since
> it won't fallback to https.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
