skonto commented on issue #25229: [SPARK-27900][K8s] Add jvm oom flag
URL: https://github.com/apache/spark/pull/25229#issuecomment-543645501
closing this in favor of https://github.com/apache/spark/pull/26161
This is an
skonto commented on issue #25229: [SPARK-27900][K8s] Add jvm oom flag
URL: https://github.com/apache/spark/pull/25229#issuecomment-542709426
back to this.
This is an automated message from the Apache Git Service.
To respond
skonto commented on issue #25229: [SPARK-27900][K8s] Add jvm oom flag
URL: https://github.com/apache/spark/pull/25229#issuecomment-533082486
thanks @holdenk @dongjoon-hyun I will target Spark 3.
This is an automated message
skonto commented on issue #25229: [SPARK-27900][K8s] Add jvm oom flag
URL: https://github.com/apache/spark/pull/25229#issuecomment-520250390
@dongjoon-hyun sorry for the name spelling thing :) I am on vacations but
will slowly have a look to what @squito suggests:)
skonto commented on issue #25229: [SPARK-27900][K8s] Add jvm oom flag
URL: https://github.com/apache/spark/pull/25229#issuecomment-518871183
@Dooyoung-Hwang that was one of my options. So this PR was intended to
protect the user from the spark issues when things dont go well and the driver
skonto commented on issue #25229: [SPARK-27900][K8s] Add jvm oom flag
URL: https://github.com/apache/spark/pull/25229#issuecomment-518763517
@dongjoon-hyun a clarification here:
> By default, DEFAULT_DRIVER_JVM_OPTIONS=-XX:OnOutOfMemoryError="kill -9 %p"
and it will be appended
skonto commented on issue #25229: [SPARK-27900][K8s] Add jvm oom flag
URL: https://github.com/apache/spark/pull/25229#issuecomment-518762919
a
This is an automated message from the Apache Git Service.
To respond to the
skonto commented on issue #25229: [SPARK-27900][K8s] Add jvm oom flag
URL: https://github.com/apache/spark/pull/25229#issuecomment-518759232
@dongjoon-hyun
By default the user will not be able to pass any
spark.driver.extraJavaOptions settings are his properties are put in a
skonto commented on issue #25229: [SPARK-27900][K8s] Add jvm oom flag
URL: https://github.com/apache/spark/pull/25229#issuecomment-518748224
@dongjoon-hyun one thing I noticed is that if I set:
```case "$1" in
driver)
shift 1
VERBOSE_FLAG=$(get_verbose_flag)
skonto commented on issue #25229: [SPARK-27900][K8s] Add jvm oom flag
URL: https://github.com/apache/spark/pull/25229#issuecomment-516594015
@dongjoon-hyun I updated the PR.
Using the following submission cmd:
```
./bin/spark-submit --master k8s://https://192.168.2.5:8443\
skonto commented on issue #25229: [SPARK-27900][K8s] Add jvm oom flag
URL: https://github.com/apache/spark/pull/25229#issuecomment-515909297
@dongjoon-hyun Ok I will improve the PR, I had in mind to test
`spark.driver.extraJavaOptions` with this anyway. I will not modify the tests,
test
skonto commented on issue #25229: [SPARK-27900][K8s] Add jvm oom flag
URL: https://github.com/apache/spark/pull/25229#issuecomment-515677267
we need to make sure its added by default and not rely to the user to pass
it. If you mean whether to pass this as a spark conf then in client mode
12 matches
Mail list logo
