Github user ifilonenko commented on a diff in the pull request:
https://github.com/apache/spark/pull/22298#discussion_r214410560
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala
---
@@ -225,6 +225,13 @@ private[spark] object Config extends Logging {
"Ensure that major Python version is either Python2 or Python3")
.createWithDefault("2")
+ val APP_RESOURCE_TYPE =
--- End diff --
The reason for this is because we are already running binding steps that
configure the driver based on the app resource. I thought it might as well pass
the config down into the executors upon doing that binding bootstrap step.
Currently, we don't have any docker files that handle mixed pipelines so
such configurations should be addressed in a followup-PR, imo. But I am open to
suggestions (that are testable).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
