This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 4d4c044 [SPARK-38392][K8S][TESTS] Add `spark-` prefix to namespaces
and `-driver` suffix to drivers during IT
4d4c044 is described below
commit 4d4c0444ac71f177711a5dab79b506f5048d9200
Author: Martin Tzvetanov Grigorov <[email protected]>
AuthorDate: Wed Mar 2 10:02:13 2022 -0800
[SPARK-38392][K8S][TESTS] Add `spark-` prefix to namespaces and `-driver`
suffix to drivers during IT
### What changes were proposed in this pull request?
There are two small proposals:
1) prefix the name of the temporary k8s namespaces with `"spark-"` so that
the output of `kubectl get ns" is more clear.
2) unify the name of the driver pod in non-test and IT tests to always use
`-driver` as a suffix.
### Why are the changes needed?
At the moment the name of the temporary namespace is just UUID without the
`-`s. When one reads the result of `kubectl get ns` it is a bit cryptic to see
UUIDs.
The names of the driver pods in ITs are not telling me that they are
Drivers.
In non-test (i.e. production) the driver pod names are suffixed with
`-driver`. I propose the same for IT tests.
Executor pods always use `-exec-` in their pod names, both in non-test and
ITs.
### Does this PR introduce _any_ user-facing change?
Yes! Developers who debug IT tests will see more clear names now.
### How was this patch tested?
Manually with `kubectl get ns --watch` and `kubectl get po --watch`.
Closes #35711 from martin-g/k8s-test-names-improvement.
Authored-by: Martin Tzvetanov Grigorov <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
.../org/apache/spark/deploy/k8s/integrationtest/KubernetesSuite.scala | 3 ++-
.../spark/deploy/k8s/integrationtest/KubernetesTestComponents.scala | 3 ++-
2 files changed, 4 insertions(+), 2 deletions(-)
diff --git
a/resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/KubernetesSuite.scala
b/resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/KubernetesSuite.scala
index 15ce487..9faf73f 100644
---
a/resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/KubernetesSuite.scala
+++
b/resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/KubernetesSuite.scala
@@ -184,7 +184,8 @@ class KubernetesSuite extends SparkFunSuite
protected def setUpTest(): Unit = {
appLocator = UUID.randomUUID().toString.replaceAll("-", "")
- driverPodName = "spark-test-app-" +
UUID.randomUUID().toString.replaceAll("-", "")
+ driverPodName = "spark-test-app-" +
+ UUID.randomUUID().toString.replaceAll("-", "") + "-driver"
sparkAppConf = kubernetesTestComponents.newSparkAppConf()
.set("spark.kubernetes.container.image", image)
.set("spark.kubernetes.driver.pod.name", driverPodName)
diff --git
a/resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/KubernetesTestComponents.scala
b/resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/KubernetesTestComponents.scala
index 411857f..4fdb89e 100644
---
a/resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/KubernetesTestComponents.scala
+++
b/resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/KubernetesTestComponents.scala
@@ -37,7 +37,8 @@ private[spark] class KubernetesTestComponents(defaultClient:
DefaultKubernetesCl
val namespaceOption = Option(System.getProperty(CONFIG_KEY_KUBE_NAMESPACE))
val hasUserSpecifiedNamespace = namespaceOption.isDefined
- val namespace =
namespaceOption.getOrElse(UUID.randomUUID().toString.replaceAll("-", ""))
+ val namespace = namespaceOption.getOrElse("spark-" +
+ UUID.randomUUID().toString.replaceAll("-", ""))
val serviceAccountName =
Option(System.getProperty(CONFIG_KEY_KUBE_SVC_ACCOUNT))
.getOrElse("default")
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
