[
https://issues.apache.org/jira/browse/HIVE-18436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16338501#comment-16338501
]
Sahil Takiar commented on HIVE-18436:
-------------------------------------
I'm not positive, but this looks like a bug in Spark introduced by SPARK-20434.
SPARK-20434 changed {{HiveDelegationTokenProvider}} so that it constructs
{{o.a.h.hive.conf.HiveConf}} inside {{HiveCredentialProvider#hiveConf}} rather
than trying to manually load the class via the class loader. Looks like with
the new code the JVM tries to load {{HiveConf}} as soon as
{{HiveDelegationTokenProvider}} is referenced. Since there is no try-catch
around the construction of {{HiveDelegationTokenProvider}} a
{{ClassNotFoundException}} is thrown, which causes spark-submit to crash.
Spark's {{docs/running-on-yarn.md}} says "a Hive token will be obtained if Hive
is on the classpath". This behavior would seem to contradict that.
[~vanzin] thoughts?
> Upgrade to Spark 2.3.0
> ----------------------
>
> Key: HIVE-18436
> URL: https://issues.apache.org/jira/browse/HIVE-18436
> Project: Hive
> Issue Type: Task
> Components: Spark
> Reporter: Sahil Takiar
> Assignee: Sahil Takiar
> Priority: Major
>
> Branching has been completed. Release candidates should be published soon.
> Might be a while before the actual release, but at least we get to identify
> any issues early.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
