GitHub user jerryshao opened a pull request:
https://github.com/apache/spark/pull/11603
[SPARK-12343] Simplify Yarn client and client argument
## What changes were proposed in this pull request?
Currently in Spark on YARN, configurations can be passed through SparkConf,
env and command arguments, some parts are duplicated, like client argument and
SparkConf. So here propose to simplify the command arguments.
## How was this patch tested?
This patch is tested manually with unit test.
CC @vanzin @tgravescs , please help to suggest this proposal. The original
purpose of this JIRA is to remove `ClientArguments`, through refactoring some
arguments like `--class`, `--arg` are not so easy to replace, so here I remove
the most part of command line arguments, only keep the minimal set.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jerryshao/apache-spark SPARK-12343
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/11603.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #11603
----
commit 868da72f7942e0352fcdaf20589c3ccf15418360
Author: jerryshao <[email protected]>
Date: 2016-03-08T09:42:06Z
Simply the functionality of Yarn ClientArguments
commit c5738ddc0d932fcb30940f70dc9481ba5d741a69
Author: jerryshao <[email protected]>
Date: 2016-03-09T07:01:33Z
Fix test failure
commit 0c0a95ca33c1d5d7629d528884d0a65aa9ff17db
Author: jerryshao <[email protected]>
Date: 2016-03-09T08:28:16Z
continue dealing with properties
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
