Repository: spark Updated Branches: refs/heads/master 4e1f12d99 -> 8782eb992
[SPARK-4990][Deploy]to find default properties file, search SPARK_CONF_DIR first https://issues.apache.org/jira/browse/SPARK-4990 Author: WangTaoTheTonic <[email protected]> Author: WangTao <[email protected]> Closes #3823 from WangTaoTheTonic/SPARK-4990 and squashes the following commits: 133c43e [WangTao] Update spark-submit2.cmd b1ab402 [WangTao] Update spark-submit 4cc7f34 [WangTaoTheTonic] rebase 55300bc [WangTaoTheTonic] use export to make it global d8d3cb7 [WangTaoTheTonic] remove blank line 07b9ebf [WangTaoTheTonic] check SPARK_CONF_DIR instead of checking properties file c5a85eb [WangTaoTheTonic] to find default properties file, search SPARK_CONF_DIR first Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/8782eb99 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/8782eb99 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/8782eb99 Branch: refs/heads/master Commit: 8782eb992f461502238c41ece3a3002efa67a792 Parents: 4e1f12d Author: WangTaoTheTonic <[email protected]> Authored: Fri Jan 9 17:10:02 2015 -0800 Committer: Andrew Or <[email protected]> Committed: Fri Jan 9 17:10:02 2015 -0800 ---------------------------------------------------------------------- bin/spark-submit | 5 ++++- bin/spark-submit2.cmd | 6 +++++- 2 files changed, 9 insertions(+), 2 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/8782eb99/bin/spark-submit ---------------------------------------------------------------------- diff --git a/bin/spark-submit b/bin/spark-submit index aefd38a..3e5cbdb 100755 --- a/bin/spark-submit +++ b/bin/spark-submit @@ -44,7 +44,10 @@ while (($#)); do shift done -DEFAULT_PROPERTIES_FILE="$SPARK_HOME/conf/spark-defaults.conf" +if [ -z "$SPARK_CONF_DIR" ]; then + export SPARK_CONF_DIR="$SPARK_HOME/conf" +fi +DEFAULT_PROPERTIES_FILE="$SPARK_CONF_DIR/spark-defaults.conf" if [ "$MASTER" == "yarn-cluster" ]; then SPARK_SUBMIT_DEPLOY_MODE=cluster fi http://git-wip-us.apache.org/repos/asf/spark/blob/8782eb99/bin/spark-submit2.cmd ---------------------------------------------------------------------- diff --git a/bin/spark-submit2.cmd b/bin/spark-submit2.cmd index daf0284..12244a9 100644 --- a/bin/spark-submit2.cmd +++ b/bin/spark-submit2.cmd @@ -24,7 +24,11 @@ set ORIG_ARGS=%* rem Reset the values of all variables used set SPARK_SUBMIT_DEPLOY_MODE=client -set SPARK_SUBMIT_PROPERTIES_FILE=%SPARK_HOME%\conf\spark-defaults.conf + +if not defined %SPARK_CONF_DIR% ( + set SPARK_CONF_DIR=%SPARK_HOME%\conf +) +set SPARK_SUBMIT_PROPERTIES_FILE=%SPARK_CONF_DIR%\spark-defaults.conf set SPARK_SUBMIT_DRIVER_MEMORY= set SPARK_SUBMIT_LIBRARY_PATH= set SPARK_SUBMIT_CLASSPATH= --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
