Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/1845#discussion_r16280819
--- Diff: bin/spark-class ---
@@ -101,11 +106,16 @@ fi
# Set JAVA_OPTS to be able to load native libraries and to set heap size
JAVA_OPTS="-XX:MaxPermSize=128m $OUR_JAVA_OPTS"
JAVA_OPTS="$JAVA_OPTS -Xms$OUR_JAVA_MEM -Xmx$OUR_JAVA_MEM"
+
# Load extra JAVA_OPTS from conf/java-opts, if it exists
if [ -e "$FWDIR/conf/java-opts" ] ; then
JAVA_OPTS="$JAVA_OPTS `cat $FWDIR/conf/java-opts`"
fi
-export JAVA_OPTS
+
+# Split JAVA_OPTS properly to handle whitespace, double quotes and
backslashes
+# This exports the split java options into SPLIT_JAVA_OPTS
+split_java_options "$JAVA_OPTS"
--- End diff --
This actually solves a more general problem than those reported in
SPARK-2849 and SPARK-2914... the problem/feature is that in general we don't
support quotes in any of the java option strings we have. I tested this in
master and confirmed it doesn't work:
```
$ export SPARK_JAVA_OPTS="-Dfoo=\"bar baz\""
$ ./bin/spark-shell
Spark assembly has been built with Hive, including Datanucleus jars on
classpath
Error: Could not find or load main class baz"
```
So it might be good to create another JIRA as well for this PR. One that
just explains that none of the JAVA_OPTS variants we have correctly support
quoted strings.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
