GitHub user andrewor14 opened a pull request:
https://github.com/apache/spark/pull/710
[SPARK-1774] Respect SparkSubmit --jars on YARN (client)
SparkSubmit ignores `--jars` for YARN client. This is a bug.
This PR also automatically adds the application jar to `spark.jar`.
Previously, when running as yarn-client, you must specify the jar additionally
through `--files` (because `--jars` didn't work). Now you don't have to
explicitly specify it through either.
Tested on a YARN cluster.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/andrewor14/spark yarn-jars
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/710.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #710
----
commit 3bb75e8a46191ba0c8c18875426857d5ddef3445
Author: Andrew Or <[email protected]>
Date: 2014-05-09T04:25:55Z
Allow SparkSubmit --jars to take effect in yarn-client mode
commit 1407474d0d3acd349a561cc240d3463bdbacb480
Author: Andrew Or <[email protected]>
Date: 2014-05-09T04:35:28Z
Merge branch 'master' of github.com:apache/spark into yarn-jars
Conflicts:
core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala
commit 013d8405dfb6774e1969dfec0cf9441563a1d409
Author: Andrew Or <[email protected]>
Date: 2014-05-09T04:55:26Z
Fix tests
commit 269f9f397d821abd337982f0af7f90844695b5bf
Author: Andrew Or <[email protected]>
Date: 2014-05-09T05:06:32Z
Fix format
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
