advancedxy commented on code in PR #41257:
URL: https://github.com/apache/spark/pull/41257#discussion_r1228944623
##########
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala:
##########
@@ -165,6 +165,16 @@ private[spark] object Config extends Logging {
.checkValue(_ <= 1048576, "Must have at most 1048576 bytes")
.createWithDefault(1048576) // 1.0 MiB
+ val SPARK_CONF_DIR_CONFIG_MAP_NAME =
+ ConfigBuilder("spark.kubernetes.sparkConfDir.configMapName")
Review Comment:
This configuration is marked as `internal()` and shall be used internally
only to bookkeep the config map name used for driver when the spark-submit
clients generates the `SPARK_CONF` config map for the driver pod.
> Can we simply this new feature by simply controlled by a boolean
configuration instead of a random string name?
For this case(k8s-cluster mode), I don't think executors should use a
different config map for driver and executor, thus I would prefer reusing the
config map is mandatory.
The reason why this configuration is defined is that we have to keep/record
the config map name, so the driver pod could refer the name. If there's other
ways to propagate such thing, I will change it accordingly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
