[
https://issues.apache.org/jira/browse/HADOOP-13206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15386720#comment-15386720
]
Chris Nauroth commented on HADOOP-13206:
----------------------------------------
For case 1, this sounds like what I reported in MAPREDUCE-6565, where the job
submission client might have a different setting for
{{hadoop.security.token.service.use_ip}} compared to the cluster nodes that run
the map and reduce tasks. My thinking on this was that the right configuration
setting, as used by the job submission client, ought to propagate into the YARN
containers running the map and reduce tasks via job.xml. I never coded a
patch, but HADOOP-12954 provided a potential building block for that change, so
that applications would have a way to change that setting.
For case 2, I could be wrong, but I thought the decision on whether or not to
use IP address or host in the delegation token service was driven solely by
{{hadoop.security.token.service.use_ip}} and not driven by whether the
connection was established by hostname or by IP address. If so, then this
wouldn't be an issue. (Like I said though, I could be wrong, and I can't dig
into the code right now to confirm.)
The additional check you proposed might limit the DNS lookups a bit. However,
I'd really prefer to defer code review to [~daryn] or [~kihwal] if you decide
to proceed with that. I recall that avoiding these DNS lookups was crucial in
Yahoo's large-scale clusters.
> Delegation token cannot be fetched and used by different versions of client
> ---------------------------------------------------------------------------
>
> Key: HADOOP-13206
> URL: https://issues.apache.org/jira/browse/HADOOP-13206
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
> Affects Versions: 2.3.0, 2.6.1
> Reporter: Zhe Zhang
> Assignee: Zhe Zhang
> Attachments: HADOOP-13206.00.patch, HADOOP-13206.01.patch,
> HADOOP-13206.02.patch
>
>
> We have observed that an HDFS delegation token fetched by a 2.3.0 client
> cannot be used by a 2.6.1 client, and vice versa. Through some debugging I
> found that it's a mismatch between the token's {{service}} and the
> {{service}} of the filesystem (e.g. {{webhdfs://host.something.com:50070/}}).
> One would be in numerical IP address and one would be in non-numerical
> hostname format.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
