GitHub user pmackles opened a pull request:
https://github.com/apache/spark/pull/21006
[SPARK-22256][MESOS] - Introduce spark.mesos.driver.memoryOverhead
When running spark driver in a container such as when using the Mesos
dispatcher service, we need to apply the same rules as for executors in order
to avoid the JVM going over the allotted limit and then killed.
Tested manually on spark 2.3 branch
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/pmackles/spark paul-SPARK-22256
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/21006.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #21006
----
commit 1197c0b2e4ae72c1353ab4cd132285da4cfed61e
Author: Paul Mackles <pmackles@...>
Date: 2018-04-06T17:44:38Z
[SPARK-22256] - Introduce spark.mesos.driver.memoryOverhead
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
