skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag URL: https://github.com/apache/spark/pull/25229#issuecomment-516594015 @dongjoon-hyun I updated the PR. Using the following submission cmd: ``` ./bin/spark-submit --master k8s://https://192.168.2.5:8443\ --deploy-mode cluster \ --name spark-pi \ --class org.apache.spark.examples.SparkPi \ --conf spark.executor.memory=1G \ --conf spark.kubernetes.namespace=spark \ --conf spark.kubernetes.driverEnv.DRIVER_VERBOSE=true \ --conf spark.kubernetes.authenticate.driver.serviceAccountName=spark-sa \ --conf spark.driver.memory=500m \ --conf spark.executor.instances=2 \ --conf spark.kubernetes.container.image.pullPolicy=Always \ --conf spark.kubernetes.container.image=skonto/spark:oom \ local:///opt/spark/examples/jars/spark-examples_2.12-3.0.0-SNAPSHOT.jar 1000000000``` ``` I get the flag added by default: ``` 19 org.apache.spark.deploy.SparkSubmit --deploy-mode client --conf spark.driver.bindAddress=172.17.0.4 --properties-file /tmp/spark.properties --class org.apache.spark.examples.SparkPi --verbose local:///opt/spark/examples/jars/spark-examples_2.12-3.0.0-SNAPSHOT.jar 1000000000 -Xmx500m -XX:OnOutOfMemoryError=kill -9 %p ``` If I add ` --conf "spark.driver.extraJavaOptions=-Ds=3" \` I get as expected: ``` 19 org.apache.spark.deploy.SparkSubmit --deploy-mode client --conf spark.driver.bindAddress=172.17.0.4 --properties-file /tmp/spark.properties --class org.apache.spark.examples.SparkPi --verbose local:///opt/spark/examples/jars/spark-examples_2.12-3.0.0-SNAPSHOT.jar 1000000000 -Xmx500m -XX:OnOutOfMemoryError=kill -9 %p -Ds=3 ```
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
