[
https://issues.apache.org/jira/browse/OOZIE-3036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16127006#comment-16127006
]
Peter Cseh commented on OOZIE-3036:
-----------------------------------
+1
> Spark 2.2.0 support: tell Spark not to get any delegation tokens
> ----------------------------------------------------------------
>
> Key: OOZIE-3036
> URL: https://issues.apache.org/jira/browse/OOZIE-3036
> Project: Oozie
> Issue Type: Bug
> Affects Versions: 4.3.0
> Environment: * [*Spark
> 2.2.0*|https://github.com/apache/spark/tree/branch-2.2]
> * [*Hadoop 3 alpha
> 4*|https://github.com/apache/hadoop/tree/branch-3.0.0-alpha4]
> Reporter: Andras Piros
> Assignee: Andras Piros
> Fix For: 5.0.0
>
> Attachments: OOZIE-3036.001.patch
>
>
> For Spark 2.2.0 along w/ Hadoop 3 alpha 4, Spark uses another set of
> arguments that ensure Spark won't get any delegation tokens:
> * {{spark.yarn.security.credentials.hadoopfs.enabled}}
> * {{spark.yarn.security.credentials.hive.enabled}}
> * {{spark.yarn.security.credentials.hbase.enabled}}
> For Spark 1.6.0 along w/ Hadoop 3 alpha 4, we need following argument to set
> extra:
> * {{spark.yarn.security.tokens.hadoopfs.enabled}}
> As we can't tell runtime which Spark / Hadoop versions is the Oozie Spark
> action running on, it's best to use the union of those arguments.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
