[
https://issues.apache.org/jira/browse/SPARK-21888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16149350#comment-16149350
]
Parth Gandhi edited comment on SPARK-21888 at 8/31/17 6:01 PM:
---------------------------------------------------------------
The spark job runs successfully only if hbase-site.xml is placed in
SPARK_CONF_DIR. If I add the xml file to --jars then it gets added to the
driver classpath which is required but hbase fails to get a valid Kerberos
token as the xml file is not found in the system classpath on the gateway where
I launch the application.
was (Author: pgandhi):
The spark job runs successfully only if hbase-site.xml is placed in
SPARK_CONF_DIR. If I add the xml file to --jars then it gets added to the
driver classmate which is required but hbase fails to get a valid Kerberos
token as the xml file is not found in the system classpath on the gateway where
I launch the application.
> Cannot add stuff to Client Classpath for Yarn Cluster Mode
> ----------------------------------------------------------
>
> Key: SPARK-21888
> URL: https://issues.apache.org/jira/browse/SPARK-21888
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 2.2.0
> Reporter: Parth Gandhi
> Priority: Minor
>
> While running Spark on Yarn in cluster mode, currently there is no way to add
> any config files, jars etc. to Client classpath. An example for this is that
> suppose you want to run an application that uses hbase. Then, unless and
> until we do not copy the necessary config files required by hbase to Spark
> Config folder, we cannot specify or set their exact locations in classpath on
> Client end which we could do so earlier by setting the environment variable
> "SPARK_CLASSPATH".
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
