Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/22298#discussion_r214491727
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/BasicDriverFeatureStep.scala
---
@@ -51,6 +51,7 @@ private[spark] class BasicDriverFeatureStep(
.get(DRIVER_MEMORY_OVERHEAD)
.getOrElse(math.max((conf.get(MEMORY_OVERHEAD_FACTOR) *
driverMemoryMiB).toInt,
MEMORY_OVERHEAD_MIN_MIB))
+ // TODO: Have memory limit checks on driverMemory
--- End diff --
Hm can you elaborate here? We already set the driver memory limit in this
step based on the overhead.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
