This is an automated email from the ASF dual-hosted git repository.
yumwang pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 6becf4e [SPARK-38538][K8S][TESTS] Fix driver environment verification
in BasicDriverFeatureStepSuite
6becf4e is described below
commit 6becf4e93e68e36fbcdc82768de497d86072abeb
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Sun Mar 13 11:33:23 2022 +0800
[SPARK-38538][K8S][TESTS] Fix driver environment verification in
BasicDriverFeatureStepSuite
### What changes were proposed in this pull request?
This PR aims to fix the driver environment verification logic in
`BasicDriverFeatureStepSuite`.
### Why are the changes needed?
When SPARK-25876 added a test logic at Apache Spark 3.0.0, it used `envs(v)
=== v` instead of `envs(k) === v`.
https://github.com/apache/spark/blob/c032928515e74367137c668ce692d8fd53696485/resource-managers/kubernetes/core/src/test/scala/org/apache/spark/deploy/k8s/features/BasicDriverFeatureStepSuite.scala#L94-L96
This bug was hidden because the test key-value pairs have identical set. If
we have different strings for keys and values, the test case fails.
https://github.com/apache/spark/blob/c032928515e74367137c668ce692d8fd53696485/resource-managers/kubernetes/core/src/test/scala/org/apache/spark/deploy/k8s/features/BasicDriverFeatureStepSuite.scala#L42-L44
### Does this PR introduce _any_ user-facing change?
To have a correct test coverage.
### How was this patch tested?
Pass the CIs.
Closes #35828 from dongjoon-hyun/SPARK-38538.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Yuming Wang <[email protected]>
---
.../spark/deploy/k8s/features/BasicDriverFeatureStepSuite.scala | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git
a/resource-managers/kubernetes/core/src/test/scala/org/apache/spark/deploy/k8s/features/BasicDriverFeatureStepSuite.scala
b/resource-managers/kubernetes/core/src/test/scala/org/apache/spark/deploy/k8s/features/BasicDriverFeatureStepSuite.scala
index 0b54599..bf7fbcc 100644
---
a/resource-managers/kubernetes/core/src/test/scala/org/apache/spark/deploy/k8s/features/BasicDriverFeatureStepSuite.scala
+++
b/resource-managers/kubernetes/core/src/test/scala/org/apache/spark/deploy/k8s/features/BasicDriverFeatureStepSuite.scala
@@ -40,8 +40,8 @@ class BasicDriverFeatureStepSuite extends SparkFunSuite {
"customAnnotation" -> "customAnnotationValue",
"yunikorn.apache.org/app-id" -> "{{APPID}}")
private val DRIVER_ENVS = Map(
- "customDriverEnv1" -> "customDriverEnv2",
- "customDriverEnv2" -> "customDriverEnv2")
+ "customDriverEnv1" -> "customDriverEnv1Value",
+ "customDriverEnv2" -> "customDriverEnv2Value")
private val TEST_IMAGE_PULL_SECRETS = Seq("my-secret-1", "my-secret-2")
private val TEST_IMAGE_PULL_SECRET_OBJECTS =
TEST_IMAGE_PULL_SECRETS.map { secret =>
@@ -92,7 +92,7 @@ class BasicDriverFeatureStepSuite extends SparkFunSuite {
.map { env => (env.getName, env.getValue) }
.toMap
DRIVER_ENVS.foreach { case (k, v) =>
- assert(envs(v) === v)
+ assert(envs(k) === v)
}
assert(envs(ENV_SPARK_USER) === Utils.getCurrentUserName())
assert(envs(ENV_APPLICATION_ID) === kubernetesConf.appId)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
