Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19954#discussion_r158651909
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala
---
@@ -132,30 +131,84 @@ private[spark] object Config extends Logging {
val JARS_DOWNLOAD_LOCATION =
ConfigBuilder("spark.kubernetes.mountDependencies.jarsDownloadDir")
- .doc("Location to download jars to in the driver and executors. When
using" +
- " spark-submit, this directory must be empty and will be mounted
as an empty directory" +
- " volume on the driver and executor pod.")
+ .doc("Location to download jars to in the driver and executors. When
using " +
+ "spark-submit, this directory must be empty and will be mounted as
an empty directory " +
+ "volume on the driver and executor pod.")
.stringConf
.createWithDefault("/var/spark-data/spark-jars")
val FILES_DOWNLOAD_LOCATION =
ConfigBuilder("spark.kubernetes.mountDependencies.filesDownloadDir")
- .doc("Location to download files to in the driver and executors.
When using" +
- " spark-submit, this directory must be empty and will be mounted
as an empty directory" +
- " volume on the driver and executor pods.")
+ .doc("Location to download files to in the driver and executors.
When using " +
+ "spark-submit, this directory must be empty and will be mounted as
an empty directory " +
+ "volume on the driver and executor pods.")
.stringConf
.createWithDefault("/var/spark-data/spark-files")
+ val INIT_CONTAINER_IMAGE =
+ ConfigBuilder("spark.kubernetes.initContainer.image")
+ .doc("Image for the driver and executor's init-container for
downloading dependencies.")
+ .stringConf
+ .createOptional
+
+ val INIT_CONTAINER_MOUNT_TIMEOUT =
+ ConfigBuilder("spark.kubernetes.mountDependencies.timeout")
+ .doc("Timeout before aborting the attempt to download and unpack
dependencies from remote " +
+ "locations into the driver and executor pods.")
+ .timeConf(TimeUnit.MINUTES)
+ .createWithDefault(5)
+
+ val INIT_CONTAINER_MAX_THREAD_POOL_SIZE =
+
ConfigBuilder("spark.kubernetes.mountDependencies.maxSimultaneousDownloads")
--- End diff --
I think the current name is already pretty long. Adding `initContainer`
makes it even longer without much added value.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
