[
https://issues.apache.org/jira/browse/MAPREDUCE-3825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13203151#comment-13203151
]
Sanjay Radia commented on MAPREDUCE-3825:
-----------------------------------------
Problem statement:
# MR needs to be able to automatically get delegation tokens for a MR job so
that input, output and default file systems paths are automatically handled.
** This should work for when input and output are different from the default
file systems. It should work when any of above are viewfs.
# A Job can add additional file systems that might be needed in a job by adding
these to the job conf.
** e.g. a job is known to access data from some other cluster.
# An admin can add additional file systems that are commonly needed.
** e.g. admin knows that there are symlinks to other clusters.
The APIs should allow one to easily not *obtain* duplicate tokens (note I
didn't say *eliminate*; we don't want to get unnecessary tokens because they
cost a RPC calls during job submission.) Note we should be careful about *not
overstating the urgency of this jira* because job submission is quite slow due
to split calculations etc. However our APis should clearly allow this.
> Need generalized multi-token filesystem support
> -----------------------------------------------
>
> Key: MAPREDUCE-3825
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3825
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: security
> Affects Versions: 0.23.1, 0.24.0
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
>
> This is the counterpart to HADOOP-7967. The token cache currently tries to
> assume a filesystem's token service key. The assumption generally worked
> while there was a one to one mapping of filesystem to token. With the advent
> of multi-token filesystems like viewfs, the token cache will try to use a
> service key (ie. for viewfs) that will never exist (because it really gets
> the mounted fs tokens).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
