[
https://issues.apache.org/jira/browse/SPARK-13148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129136#comment-15129136
]
Steve Loughran commented on SPARK-13148:
----------------------------------------
Note that Hadoop's UGI class automatically loads the file referenced off
{{$HADOOP_TOKEN_FILE_LOCATION}} when it inits; this is the mechanism used to
get tokens in the YARN AM.
Client-side, they become the tokens of the current user. All that is needed is
for the Yarn client to recognise that the situation has occurred (i.e. the env
variable is set), add all those credentials to the AM's launch context —and
skip trying to acquire tokens for filesystems, HBase and Hive.
> support zero-keytab Oozie application launch on a secure cluster
> -----------------------------------------------------------------
>
> Key: SPARK-13148
> URL: https://issues.apache.org/jira/browse/SPARK-13148
> Project: Spark
> Issue Type: New Feature
> Components: YARN
> Affects Versions: 1.6.0
> Environment: YARN cluster with Kerberos enabled, launched from Oozie
> —where Oozie passes down the delegation tokens
> Reporter: Steve Loughran
>
> Oozie can launch Spark instances on insecure clusters, and on a secure
> cluster if Oozie is set up to provide a keytab.
> What it cannot currently do is launch a Spark application on a YARN cluster
> without a keytab. In this situation, Oozie collects the delegation tokens it
> is setup to collect (as a superuser in the cluster), saves them to a file,
> then points to the file in the `HADOOP_TOKEN_FILE_LOCATION` environment
> variable.
> These tokens need to be used to launch the application —rather than try to
> get some more
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
