Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/19717#discussion_r155845314
--- Diff: core/src/main/scala/org/apache/spark/SparkConf.scala ---
@@ -668,7 +668,9 @@ private[spark] object SparkConf extends Logging {
MAX_REMOTE_BLOCK_SIZE_FETCH_TO_MEM.key -> Seq(
AlternateConfig("spark.reducer.maxReqSizeShuffleToMem", "2.3")),
LISTENER_BUS_EVENT_QUEUE_CAPACITY.key -> Seq(
- AlternateConfig("spark.scheduler.listenerbus.eventqueue.size",
"2.3"))
+ AlternateConfig("spark.scheduler.listenerbus.eventqueue.size",
"2.3")),
+ "spark.driver.memoryOverhead" -> Seq(
--- End diff --
`DRIVER_MEMORY_OVERHEAD.key`.
Don't you also need one for the executor in k8s?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
