tgravescs commented on a change in pull request #24703: [SPARK-27362][K8S]
Resource Scheduling support for k8s
URL: https://github.com/apache/spark/pull/24703#discussion_r288737831
##########
File path:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesConf.scala
##########
@@ -199,4 +199,13 @@ private[spark] object KubernetesConf {
.replaceAll("[^a-z0-9\\-]", "")
.replaceAll("-+", "-")
}
+
+ /**
+ * Build a resources name based on the vendor device plugin naming
+ * convention of: vendor-domain/resource. For example, an Nvidia GPU is
+ * advertised as nvidia.com/gpu.
+ */
+ def buildKubernetesResourceName(vendor: String, resourceName: String):
String = {
+ s"${vendor}/${resourceName}"
Review comment:
actually one question here. I was leaving the config name somewhat generic
with spark.executor.resource.{resourceName}.vendor because the intention was it
could be used by other cluster managers if it every applied. So do you think
its enough just to update the documentation to say its vendor-domain on k8s or
would it be more obvious to change the config name to something like
spark.executor.resource.{resourceName}.vendorDomain? Another option would be I
can change the name of the config to something even more generic like
spark.executor.resource.{resourceName}.resourceScheme
Any opinion on what would be most obvious to users?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
