Ryan Blue created SPARK-23472:
---------------------------------
Summary: Add config properties for administrator JVM options
Key: SPARK-23472
URL: https://issues.apache.org/jira/browse/SPARK-23472
Project: Spark
Issue Type: Improvement
Components: Spark Core
Affects Versions: 2.3.0
Reporter: Ryan Blue
In our environment, users may need to add JVM options to their Spark
applications (e.g. to override log configuration). They typically use
{{--driver-java-options}} or {{spark.executor.extraJavaOptions}}. Both set
extraJavaOptions properties. We also have a set of administrator JVM options to
apply that set the garbage collector (G1GC) and kill the driver JVM on OOM.
These two use cases both need to set extraJavaOptions properties, but will
clobber one another. In the past we've maintained wrapper scripts, but this
causes our default properties to be maintained in scripts rather than our
spark-defaults.properties.
I think we should add defaultJavaOptions properties that are added along with
extraJavaOptions. Administrators could set defaultJavaOptions and these would
always get added to the JVM command line, along with any user options instead
of getting overwritten by user options.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
