skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag URL: https://github.com/apache/spark/pull/25229#issuecomment-518763517 @dongjoon-hyun a clarification here pls: > By default, DEFAULT_DRIVER_JVM_OPTIONS=-XX:OnOutOfMemoryError="kill -9 %p" and it will be appended before spark.driver.extraJavaOptions. Should user supplied `spark.driver.extraJavaOptions` (part of their spark submit) have any effect if DEFAULT_DRIVER_JVM_OPTIONS is not empty? Right now user supplied `spark.driver.extraJavaOptions` will be ignored because they are part of a properties files and when I pass `--conf spark.driver.extraJavaOptions=$DEFAULT_DRIVER_JVM_OPTIONS` to ``` CMD=( "$SPARK_HOME/bin/spark-submit" ...)```, this takes priority due to SparkConf rules. What do you mean by "appended before spark.driver.extraJavaOptions"? Do you refer to the CMD array or the final value of spark.driver.extraJavaOptions, so far I used sed to merge the options but probably you mean to keep the default ones only right? I have updated the PR locally but waiting for this clarification.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
