GitHub user jerryshao opened a pull request:
https://github.com/apache/spark/pull/17388
[SPARK-20059][YARN] Use the correct classloader for HBaseCredentialProvider
## What changes were proposed in this pull request?
Currently we use system classloader to find HBase jars, if it is specified
by `--jars`, then it will be failed with ClassNotFound issue. So here changing
to use child classloader.
Also putting added jars and main jar into classpath of submitted
application in yarn cluster mode, otherwise HBase jars specified with `--jars`
will never be honored in cluster mode, and fetching tokens in client side will
always be failed.
## How was this patch tested?
Unit test and local verification.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jerryshao/apache-spark SPARK-20059
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/17388.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #17388
----
commit 9e153916356e5a7d321ff6fe42357c154c6250bf
Author: jerryshao <[email protected]>
Date: 2017-03-22T13:26:34Z
Use the right classloader and add jars to classpath in yarn-client mode
Change-Id: I04d6ffcdca58277e60d3dd0c456f4c6a2936b320
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
