Updated Branches: refs/heads/master 0367981d4 -> 6b4eed779
fix "set MASTER automatically fails" bug. spark-shell intends to set MASTER automatically if we do not provide the option when we start the shell , but there's a problem. The condition is "if [[ "x" != "x$SPARK_MASTER_IP" && "y" != "y$SPARK_MASTER_PORT" ]];" we sure will set SPARK_MASTER_IP explicitly, the SPARK_MASTER_PORT option, however, we probably do not set just using spark default port 7077. So if we do not set SPARK_MASTER_PORT, the condition will never be true. We should just use default port if users do not set port explicitly I think. Project: http://git-wip-us.apache.org/repos/asf/incubator-spark/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-spark/commit/7a0c5b5a Tree: http://git-wip-us.apache.org/repos/asf/incubator-spark/tree/7a0c5b5a Diff: http://git-wip-us.apache.org/repos/asf/incubator-spark/diff/7a0c5b5a Branch: refs/heads/master Commit: 7a0c5b5a2362253e70f9c7d72dab1cad380e52c7 Parents: 263933d Author: CrazyJvm <crazy...@gmail.com> Authored: Thu Jan 16 11:45:02 2014 +0800 Committer: CrazyJvm <crazy...@gmail.com> Committed: Thu Jan 16 11:45:02 2014 +0800 ---------------------------------------------------------------------- bin/spark-shell | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-spark/blob/7a0c5b5a/bin/spark-shell ---------------------------------------------------------------------- diff --git a/bin/spark-shell b/bin/spark-shell index e6885b5..398f211 100755 --- a/bin/spark-shell +++ b/bin/spark-shell @@ -45,13 +45,18 @@ for o in "$@"; do done # Set MASTER from spark-env if possible +DEFAULT_SPARK_MASTER_PORT=7077 if [ -z "$MASTER" ]; then if [ -e "$FWDIR/conf/spark-env.sh" ]; then . "$FWDIR/conf/spark-env.sh" fi - if [[ "x" != "x$SPARK_MASTER_IP" && "y" != "y$SPARK_MASTER_PORT" ]]; then - MASTER="spark://${SPARK_MASTER_IP}:${SPARK_MASTER_PORT}" - export MASTER + if [ "x" != "x$SPARK_MASTER_IP" ];then + if [ "y" != "y$SPARK_MASTER_PORT" ];then + SPARK_MASTER_PORT="${SPARK_MASTER_PORT}" + else + SPARK_MASTER_PORT=$DEFAULT_SPARK_MASTER_PORT + fi + export MASTER="spark://${SPARK_MASTER_IP}:${SPARK_MASTER_PORT}" fi fi
