[ https://issues.apache.org/jira/browse/SPARK-21377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084929#comment-16084929 ]
Apache Spark commented on SPARK-21377: -------------------------------------- User 'jerryshao' has created a pull request for this issue: https://github.com/apache/spark/pull/18616 > Add a new configuration to extend AM classpath in yarn client mode > ------------------------------------------------------------------ > > Key: SPARK-21377 > URL: https://issues.apache.org/jira/browse/SPARK-21377 > Project: Spark > Issue Type: Bug > Components: YARN > Affects Versions: 2.2.0 > Reporter: Yesha Vora > Priority: Minor > > In this issue we have a long running Spark application with secure HBase, > which requires {{HBaseCredentialProvider}} to get tokens periodically, we > specify HBase related jars with {{\--packages}}, but these dependencies are > not added into AM classpath, so when {{HBaseCredentialProvider}} tries to > initialize HBase connections to get tokens, it will be failed. > Currently because jars specified with {{\--jars}} or {{\--packages}} are not > added into AM classpath, the only way to extend AM classpath is to use > "spark.driver.extraClassPath" which supposed to be used in yarn cluster mode. > So here we should figure out a solution either to put these dependencies to > AM classpath or to extend AM classpath with correct configuration. -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org