[
https://issues.apache.org/jira/browse/SPARK-21377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcelo Vanzin resolved SPARK-21377.
------------------------------------
Resolution: Fixed
Assignee: Saisai Shao
Fix Version/s: 2.3.0
> Jars specified with --jars or --packages are not added into AM's system
> classpath
> ---------------------------------------------------------------------------------
>
> Key: SPARK-21377
> URL: https://issues.apache.org/jira/browse/SPARK-21377
> Project: Spark
> Issue Type: Bug
> Components: YARN
> Affects Versions: 2.2.0
> Reporter: Yesha Vora
> Assignee: Saisai Shao
> Priority: Minor
> Fix For: 2.3.0
>
>
> In this issue we have a long running Spark application with secure HBase,
> which requires {{HBaseCredentialProvider}} to get tokens periodically, we
> specify HBase related jars with {{\--packages}}, but these dependencies are
> not added into AM classpath, so when {{HBaseCredentialProvider}} tries to
> initialize HBase connections to get tokens, it will be failed.
> Currently because jars specified with {{\--jars}} or {{\--packages}} are not
> added into AM classpath, the only way to extend AM classpath is to use
> "spark.driver.extraClassPath" which supposed to be used in yarn cluster mode.
> So here we should figure out a solution either to put these dependencies to
> AM classpath or to extend AM classpath with correct configuration.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
