Hey all,
Stuck trying to set a parameter in the spark-env.sh and I’m hoping someone here
knows how.
I want to set the JVM setting -XX:+ExitOnOutOfMemoryError for both Spark
executors and Spark workers in a standalone mode.
So far my best guess so far is:
Worker
SPARK_WORKER_OPTS=“${SPARK_WORKER_OPTS}
-Dspark.worker.extraJavaOptions=-XX:+ExitOnOutOfMemoryError”
Executor
SPARK_DAEMON_JAVA_OPTS=“${SPARK_DAEMON_JAVA_OPTS}
-Dspark.executor.extraJavaOptions=-XX:+ExitOnOutOfMemoryError”
Anyone know the actual way to set this or a good place to learn about how this
stuff works? I’ve already seen the Spark conf and standalone documentation and
it doesn’t really make this stuff clear.
Thanks a bunch,
Eevee.
