[
https://issues.apache.org/jira/browse/HIVE-15767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16085612#comment-16085612
]
Yibing Shi commented on HIVE-15767:
-----------------------------------
bq. I think the Spark driver will get the tokens afterwards
I really doubt that Spark driver can do this. In Oozie environment, it is Oozie
server that obtains all the tokens *on behalf of the end user*. When the Hive
actions starts a Spark job, the Spark driver has no access to end user ticket
or keytab file. I don't think it can obtain necessary tokens.
I believe we should somehow extract all the tokens from existing toke file, and
pass it on to the Spark driver.
> Hive On Spark is not working on secure clusters from Oozie
> ----------------------------------------------------------
>
> Key: HIVE-15767
> URL: https://issues.apache.org/jira/browse/HIVE-15767
> Project: Hive
> Issue Type: Bug
> Components: Spark
> Affects Versions: 1.2.1, 2.1.1
> Reporter: Peter Cseh
> Assignee: Peter Cseh
> Attachments: HIVE-15767-001.patch, HIVE-15767-002.patch
>
>
> When a HiveAction is launched form Oozie with Hive On Spark enabled, we're
> getting errors:
> {noformat}
> Caused by: java.io.IOException: Exception reading
> file:/yarn/nm/usercache/yshi/appcache/application_1485271416004_0022/container_1485271416004_0022_01_000002/container_tokens
> at
> org.apache.hadoop.security.Credentials.readTokenStorageFile(Credentials.java:188)
> at
> org.apache.hadoop.mapreduce.security.TokenCache.mergeBinaryTokens(TokenCache.java:155)
> {noformat}
> This is caused by passing the {{mapreduce.job.credentials.binary}} property
> to the Spark configuration in RemoteHiveSparkClient.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
