Github user aarondav commented on a diff in the pull request:
https://github.com/apache/spark/pull/730#discussion_r12511378
--- Diff: bin/spark-submit ---
@@ -35,8 +35,10 @@ while (($#)); do
shift
done
-if [ ! -z $DRIVER_MEMORY ] && [ ! -z $DEPLOY_MODE ] && [ $DEPLOY_MODE =
"client" ]; then
- export SPARK_MEM=$DRIVER_MEMORY
+DEPLOY_MODE=${DEPLOY_MODE:-"client"}
+
+if [ ! -z $DRIVER_MEMORY ] && [ ! $DEPLOY_MODE == "cluster" ]; then
--- End diff --
Sorry to not notice this earlier,but given the default value, I think the
equality check with "client" now may make more sense.
Also, along similar lines, perhaps we could use `-n` instead of `! -z` for
DRIVER_MEMORY, and also wrap the variables in quotes. For instance, right now
if $DRIVER_MEMORY is not set, this is actually evaluating `if [ ! -z ]`, which
happens to evaluate to false, but is confusing because `if [ ! -n ]` also
evaluates to false. By putting quotes around it, we'll make sure we actually
evaluate `if [ -n "" ]` if it's empty, which has the semantics we're actually
looking for.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
