[
https://issues.apache.org/jira/browse/MAPREDUCE-1824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13072576#comment-13072576
]
Jitendra Nath Pandey commented on MAPREDUCE-1824:
-------------------------------------------------
The solution mentioned attempts to solve two issues
1) uri will probably contain hostname so no need for a dns lookup to get the
hostname to re-use dfs handle.
2) It is assumed that the file system is hdfs, the uri will contain a scheme
and will point to the correct file system.
For 1) my opinion is that we don't really need to solve that. Even if we don't
re-use existing handle, it will just cause one additional dfs object
corresponding to namenodes ip address. Any subsequent renewals will re-use the
same handle, therefore I think extra cost is just one dfs handle if we don't do
a dns lookup. Also dns caching will keep the dns lookup cost pretty low,
therefore it may also be fine to leave the current code as it is.
2) We also renew the token over https so we cannot assume to simply renew over
the uri. Token service is used in many contexts and a change there needs
extensive testing.
A less risky approach could be to use the "kind" of the token and map it to a
file system. This can be done in a utility class.
I also think hdfs assumption is not very wrong, because hdfs is the only
FileSystem implementation that issues tokens right now.
> JobTracker should reuse file system handle for delegation token renewal
> -----------------------------------------------------------------------
>
> Key: MAPREDUCE-1824
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1824
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Reporter: Jitendra Nath Pandey
> Assignee: Daryn Sharp
> Fix For: 0.20.205.0, 0.23.0
>
> Attachments: MR-1824.1.patch
>
>
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
