[
https://issues.apache.org/jira/browse/MAPREDUCE-5663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13870259#comment-13870259
]
Siddharth Seth commented on MAPREDUCE-5663:
-------------------------------------------
bq. On the AM client side you collect all the tokens you need and write them to
HDFS using the Credentials.writeTokenStorageFile() method to HDFS.
The question is on what basis are these tokens are collected. Once tokens are
available - it's fairly straightforward to make them available elsewhere on the
cluster.
Currently, tokens are automatically acquired when the MR JobClient calls
getSplits() and checkOutputSpecs. How tokens are obtained in these method calls
really depends upon the Input / OutputFormat being used.
As far as I know, Oozie doesn't actually call these methods on the client - I'm
not sure how it knows which tokens are required - guessing this is some
additional Oozie configuration.
[~vinodkv], yes MAPREDUCE-207 is a scenario for this, where we want to skip at
least the getSplits() operation on the client.
> Add an interface to Input/Ouput Formats to obtain delegation tokens
> -------------------------------------------------------------------
>
> Key: MAPREDUCE-5663
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5663
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Reporter: Siddharth Seth
> Assignee: Michael Weng
> Attachments: MAPREDUCE-5663.4.txt, MAPREDUCE-5663.5.txt,
> MAPREDUCE-5663.6.txt, MAPREDUCE-5663.patch.txt, MAPREDUCE-5663.patch.txt2,
> MAPREDUCE-5663.patch.txt3
>
>
> Currently, delegation tokens are obtained as part of the getSplits /
> checkOutputSpecs calls to the InputFormat / OutputFormat respectively.
> This works as long as the splits are generated on a node with kerberos
> credentials. For split generation elsewhere (AM for example), an explicit
> interface is required.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
