This is an automated email from the ASF dual-hosted git repository.
dongjoon-hyun pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/spark-kubernetes-operator.git
The following commit(s) were added to refs/heads/main by this push:
new ad397f3f [SPARK-59958] Deduplicate E2E fixtures and state transition
assertions
ad397f3f is described below
commit ad397f3fe70448fc6a76f4205f2fd60ed4760d85
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Fri Oct 2 13:43:43 2026 -0700
[SPARK-59958] Deduplicate E2E fixtures and state transition assertions
### What changes were proposed in this pull request?
This PR aims to deduplicate E2E fixtures and state transition assertions.
- Add 5 shared fixtures to the new `tests/e2e/fixtures/` directory to
replace 10 duplicated fixtures. Like the files in `tests/e2e/assertions/`, they
take their names and namespaces from bindings.
- Use the shared `spark-state-transition.yaml` assertion in `pi-java25`,
`pi-with-comet`, and `pi-with-gluten` instead of their own copies.
### Why are the changes needed?
To reduce the maintenance cost. For example, a Spark version upgrade needs
to update `sparkVersion` in 16 E2E files instead of 21.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Pass the CIs.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Opus 5.5
Closes #928 from dongjoon-hyun/SPARK-59958.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
AGENTS.md | 4 +--
tests/e2e/events/chainsaw-test.yaml | 14 +++++++--
tests/e2e/events/spark-example-succeeded.yaml | 31 -------------------
.../spark-application-succeeded.yaml} | 4 +--
.../spark-application}/spark-example-suspend.yaml | 4 +--
.../spark-application}/spark-example.yaml | 2 +-
.../spark-cluster-example-suspend.yaml | 4 +--
.../spark-cluster}/spark-cluster-example.yaml | 6 ++--
tests/e2e/pi-java25/chainsaw-test.yaml | 4 ++-
tests/e2e/pi-java25/spark-state-transition.yaml | 32 -------------------
tests/e2e/pi-with-comet/chainsaw-test.yaml | 4 ++-
.../e2e/pi-with-comet/spark-state-transition.yaml | 32 -------------------
tests/e2e/pi-with-gluten/chainsaw-test.yaml | 4 ++-
.../e2e/pi-with-gluten/spark-state-transition.yaml | 32 -------------------
tests/e2e/state-transition/chainsaw-test.yaml | 10 ++++--
.../spark-cluster-example-succeeded.yaml | 32 -------------------
.../state-transition/spark-example-succeeded.yaml | 31 -------------------
tests/e2e/suspend-events/chainsaw-test.yaml | 14 +++++++--
.../spark-cluster-example-suspend.yaml | 36 ----------------------
.../e2e/suspend-events/spark-example-suspend.yaml | 32 -------------------
tests/e2e/suspend/chainsaw-test.yaml | 14 +++++++--
.../e2e/watched-namespaces-file/chainsaw-test.yaml | 8 +++--
tests/e2e/watched-namespaces/chainsaw-test.yaml | 12 ++++++--
23 files changed, 80 insertions(+), 286 deletions(-)
diff --git a/AGENTS.md b/AGENTS.md
index 71d4ed7e..4992f41a 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -61,8 +61,8 @@ Gradle multi-project build (see
[settings.gradle](settings.gradle)); root group
- `build-tools/docs-utils/` — Gradle subproject (`build-tools-docs-utils`)
that generates the
config and CRD documentation.
- `tests/e2e/` — [Chainsaw](https://kyverno.github.io/chainsaw/) end-to-end
tests, grouped by
- scenario (e.g. `state-transition`, `python`, `watched-namespaces`); shared
assertions live in
- `tests/e2e/assertions/`.
+ scenario (e.g. `state-transition`, `python`, `watched-namespaces`); shared
assertions and
+ fixtures live in `tests/e2e/assertions/` and `tests/e2e/fixtures/`.
- `build-tools/`, `config/`, `examples/`, `docs/`, `dev/` — packaging (Docker
/ Helm), linter
configs, example custom resources, documentation, and developer scripts.
diff --git a/tests/e2e/events/chainsaw-test.yaml
b/tests/e2e/events/chainsaw-test.yaml
index c708bd8b..9feb2ae8 100644
--- a/tests/e2e/events/chainsaw-test.yaml
+++ b/tests/e2e/events/chainsaw-test.yaml
@@ -42,7 +42,12 @@ spec:
- name: succeeded-application-publishes-normal-events
try:
- apply:
- file: spark-example-succeeded.yaml
+ bindings:
+ - name: SPARK_APP_NAMESPACE
+ value: default
+ - name: SPARK_APPLICATION_NAME
+ value: spark-job-succeeded-events-test
+ file: "../fixtures/spark-application/spark-application-succeeded.yaml"
- assert:
bindings:
- name: SPARK_APP_NAMESPACE
@@ -191,7 +196,12 @@ spec:
- name: cluster-publishes-running-healthy-event
try:
- apply:
- file: spark-cluster-example.yaml
+ bindings:
+ - name: SPARK_APP_NAMESPACE
+ value: default
+ - name: SPARK_CLUSTER_NAME
+ value: spark-cluster-events-test
+ file: "../fixtures/spark-cluster/spark-cluster-example.yaml"
- assert:
bindings:
- name: SPARK_APP_NAMESPACE
diff --git a/tests/e2e/events/spark-example-succeeded.yaml
b/tests/e2e/events/spark-example-succeeded.yaml
deleted file mode 100644
index 2b8b1514..00000000
--- a/tests/e2e/events/spark-example-succeeded.yaml
+++ /dev/null
@@ -1,31 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-
-apiVersion: spark.apache.org/v1
-kind: SparkApplication
-metadata:
- name: spark-job-succeeded-events-test
- namespace: default
-spec:
- mainClass: "org.apache.spark.examples.SparkPi"
- jars: "local:///opt/spark/examples/jars/spark-examples.jar"
- sparkConf:
- spark.executor.instances: "1"
- spark.kubernetes.container.image: "apache/spark:{{SPARK_VERSION}}"
- spark.kubernetes.authenticate.driver.serviceAccountName: "spark"
- runtimeVersions:
- sparkVersion: "4.2.0"
diff --git a/tests/e2e/watched-namespaces/spark-example.yaml
b/tests/e2e/fixtures/spark-application/spark-application-succeeded.yaml
similarity index 96%
rename from tests/e2e/watched-namespaces/spark-example.yaml
rename to tests/e2e/fixtures/spark-application/spark-application-succeeded.yaml
index 9cc2d939..be9b31d3 100644
--- a/tests/e2e/watched-namespaces/spark-example.yaml
+++ b/tests/e2e/fixtures/spark-application/spark-application-succeeded.yaml
@@ -18,14 +18,14 @@
apiVersion: spark.apache.org/v1
kind: SparkApplication
metadata:
- name: spark-job-succeeded-test
+ name: ($SPARK_APPLICATION_NAME)
namespace: ($SPARK_APP_NAMESPACE)
spec:
mainClass: "org.apache.spark.examples.SparkPi"
jars: "local:///opt/spark/examples/jars/spark-examples.jar"
sparkConf:
spark.executor.instances: "1"
- spark.kubernetes.container.image: "apache/spark:{{SPARK_VERSION}}-scala"
+ spark.kubernetes.container.image: "apache/spark:{{SPARK_VERSION}}"
spark.kubernetes.authenticate.driver.serviceAccountName: "spark"
runtimeVersions:
sparkVersion: "4.2.0"
diff --git a/tests/e2e/suspend/spark-example-suspend.yaml
b/tests/e2e/fixtures/spark-application/spark-example-suspend.yaml
similarity index 94%
rename from tests/e2e/suspend/spark-example-suspend.yaml
rename to tests/e2e/fixtures/spark-application/spark-example-suspend.yaml
index 4272d077..b12eef15 100644
--- a/tests/e2e/suspend/spark-example-suspend.yaml
+++ b/tests/e2e/fixtures/spark-application/spark-example-suspend.yaml
@@ -18,8 +18,8 @@
apiVersion: spark.apache.org/v1
kind: SparkApplication
metadata:
- name: spark-job-suspend-test
- namespace: default
+ name: ($SPARK_APPLICATION_NAME)
+ namespace: ($SPARK_APP_NAMESPACE)
spec:
suspend: true
mainClass: "org.apache.spark.examples.SparkPi"
diff --git a/tests/e2e/watched-namespaces-file/spark-example.yaml
b/tests/e2e/fixtures/spark-application/spark-example.yaml
similarity index 97%
rename from tests/e2e/watched-namespaces-file/spark-example.yaml
rename to tests/e2e/fixtures/spark-application/spark-example.yaml
index 9cc2d939..7576eea0 100644
--- a/tests/e2e/watched-namespaces-file/spark-example.yaml
+++ b/tests/e2e/fixtures/spark-application/spark-example.yaml
@@ -18,7 +18,7 @@
apiVersion: spark.apache.org/v1
kind: SparkApplication
metadata:
- name: spark-job-succeeded-test
+ name: ($SPARK_APPLICATION_NAME)
namespace: ($SPARK_APP_NAMESPACE)
spec:
mainClass: "org.apache.spark.examples.SparkPi"
diff --git a/tests/e2e/suspend/spark-cluster-example-suspend.yaml
b/tests/e2e/fixtures/spark-cluster/spark-cluster-example-suspend.yaml
similarity index 94%
rename from tests/e2e/suspend/spark-cluster-example-suspend.yaml
rename to tests/e2e/fixtures/spark-cluster/spark-cluster-example-suspend.yaml
index 7cee6519..c43dfdf6 100644
--- a/tests/e2e/suspend/spark-cluster-example-suspend.yaml
+++ b/tests/e2e/fixtures/spark-cluster/spark-cluster-example-suspend.yaml
@@ -18,8 +18,8 @@
apiVersion: spark.apache.org/v1
kind: SparkCluster
metadata:
- name: spark-cluster-suspend-test
- namespace: default
+ name: ($SPARK_CLUSTER_NAME)
+ namespace: ($SPARK_APP_NAMESPACE)
spec:
suspend: true
runtimeVersions:
diff --git a/tests/e2e/events/spark-cluster-example.yaml
b/tests/e2e/fixtures/spark-cluster/spark-cluster-example.yaml
similarity index 90%
rename from tests/e2e/events/spark-cluster-example.yaml
rename to tests/e2e/fixtures/spark-cluster/spark-cluster-example.yaml
index bb091a4e..2a5f3fcc 100644
--- a/tests/e2e/events/spark-cluster-example.yaml
+++ b/tests/e2e/fixtures/spark-cluster/spark-cluster-example.yaml
@@ -18,8 +18,8 @@
apiVersion: spark.apache.org/v1
kind: SparkCluster
metadata:
- name: spark-cluster-events-test
- namespace: default
+ name: ($SPARK_CLUSTER_NAME)
+ namespace: ($SPARK_APP_NAMESPACE)
spec:
runtimeVersions:
sparkVersion: "4.2.0"
@@ -30,6 +30,6 @@ spec:
maxWorkers: 1
sparkConf:
spark.kubernetes.container.image: "apache/spark:{{SPARK_VERSION}}"
- spark.master.ui.title: "Spark Cluster Events E2E Test"
+ spark.master.ui.title: "Spark Cluster E2E Test"
spark.master.rest.host: "0.0.0.0"
spark.ui.reverseProxy: "true"
diff --git a/tests/e2e/pi-java25/chainsaw-test.yaml
b/tests/e2e/pi-java25/chainsaw-test.yaml
index 93dcc436..9f8a933c 100644
--- a/tests/e2e/pi-java25/chainsaw-test.yaml
+++ b/tests/e2e/pi-java25/chainsaw-test.yaml
@@ -29,8 +29,10 @@ spec:
bindings:
- name: SPARK_APP_NAMESPACE
value: default
+ - name: SPARK_APPLICATION_NAME
+ value: pi-java25
timeout: 10m
- file: spark-state-transition.yaml
+ file: "../assertions/spark-application/spark-state-transition.yaml"
catch:
- describe:
apiVersion: spark.apache.org/v1
diff --git a/tests/e2e/pi-java25/spark-state-transition.yaml
b/tests/e2e/pi-java25/spark-state-transition.yaml
deleted file mode 100644
index 2953a1ef..00000000
--- a/tests/e2e/pi-java25/spark-state-transition.yaml
+++ /dev/null
@@ -1,32 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-
-apiVersion: spark.apache.org/v1
-kind: SparkApplication
-metadata:
- name: pi-java25
- namespace: ($SPARK_APP_NAMESPACE)
-status:
- stateTransitionHistory:
- (*.currentStateSummary):
- - "Submitted"
- - "DriverRequested"
- - "DriverStarted"
- - "DriverReady"
- - "RunningHealthy"
- - "Succeeded"
- - "ResourceReleased"
diff --git a/tests/e2e/pi-with-comet/chainsaw-test.yaml
b/tests/e2e/pi-with-comet/chainsaw-test.yaml
index 6a9228ac..d22ee910 100644
--- a/tests/e2e/pi-with-comet/chainsaw-test.yaml
+++ b/tests/e2e/pi-with-comet/chainsaw-test.yaml
@@ -29,8 +29,10 @@ spec:
bindings:
- name: SPARK_APP_NAMESPACE
value: default
+ - name: SPARK_APPLICATION_NAME
+ value: pi-with-comet
timeout: 10m
- file: spark-state-transition.yaml
+ file: "../assertions/spark-application/spark-state-transition.yaml"
catch:
- describe:
apiVersion: spark.apache.org/v1
diff --git a/tests/e2e/pi-with-comet/spark-state-transition.yaml
b/tests/e2e/pi-with-comet/spark-state-transition.yaml
deleted file mode 100644
index 310136d8..00000000
--- a/tests/e2e/pi-with-comet/spark-state-transition.yaml
+++ /dev/null
@@ -1,32 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-
-apiVersion: spark.apache.org/v1
-kind: SparkApplication
-metadata:
- name: pi-with-comet
- namespace: ($SPARK_APP_NAMESPACE)
-status:
- stateTransitionHistory:
- (*.currentStateSummary):
- - "Submitted"
- - "DriverRequested"
- - "DriverStarted"
- - "DriverReady"
- - "RunningHealthy"
- - "Succeeded"
- - "ResourceReleased"
diff --git a/tests/e2e/pi-with-gluten/chainsaw-test.yaml
b/tests/e2e/pi-with-gluten/chainsaw-test.yaml
index c3dd935f..f96abc02 100644
--- a/tests/e2e/pi-with-gluten/chainsaw-test.yaml
+++ b/tests/e2e/pi-with-gluten/chainsaw-test.yaml
@@ -29,8 +29,10 @@ spec:
bindings:
- name: SPARK_APP_NAMESPACE
value: default
+ - name: SPARK_APPLICATION_NAME
+ value: pi-with-gluten
timeout: 10m
- file: spark-state-transition.yaml
+ file: "../assertions/spark-application/spark-state-transition.yaml"
catch:
- describe:
apiVersion: spark.apache.org/v1
diff --git a/tests/e2e/pi-with-gluten/spark-state-transition.yaml
b/tests/e2e/pi-with-gluten/spark-state-transition.yaml
deleted file mode 100644
index d426d813..00000000
--- a/tests/e2e/pi-with-gluten/spark-state-transition.yaml
+++ /dev/null
@@ -1,32 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-
-apiVersion: spark.apache.org/v1
-kind: SparkApplication
-metadata:
- name: pi-with-gluten
- namespace: ($SPARK_APP_NAMESPACE)
-status:
- stateTransitionHistory:
- (*.currentStateSummary):
- - "Submitted"
- - "DriverRequested"
- - "DriverStarted"
- - "DriverReady"
- - "RunningHealthy"
- - "Succeeded"
- - "ResourceReleased"
diff --git a/tests/e2e/state-transition/chainsaw-test.yaml
b/tests/e2e/state-transition/chainsaw-test.yaml
index dd6c0796..23e1b184 100644
--- a/tests/e2e/state-transition/chainsaw-test.yaml
+++ b/tests/e2e/state-transition/chainsaw-test.yaml
@@ -25,9 +25,9 @@ spec:
- name: TEST_NAME
value: succeeded
- name: APPLICATION_FILE_NAME
- value: spark-example-succeeded.yaml
+ value: ../fixtures/spark-application/spark-application-succeeded.yaml
- name: CLUSTER_FILE_NAME
- value: spark-cluster-example-succeeded.yaml
+ value: ../fixtures/spark-cluster/spark-cluster-example.yaml
- name: SPARK_APPLICATION_NAME
value: spark-job-succeeded-test
- name: SPARK_CLUSTER_NAME
@@ -35,6 +35,9 @@ spec:
steps:
- try:
- apply:
+ bindings:
+ - name: SPARK_APP_NAMESPACE
+ value: default
file: ($APPLICATION_FILE_NAME)
- assert:
bindings:
@@ -57,6 +60,9 @@ spec:
kubectl delete sparkapplication $SPARK_APPLICATION_NAME
- try:
- apply:
+ bindings:
+ - name: SPARK_APP_NAMESPACE
+ value: default
file: ($CLUSTER_FILE_NAME)
- assert:
bindings:
diff --git a/tests/e2e/state-transition/spark-cluster-example-succeeded.yaml
b/tests/e2e/state-transition/spark-cluster-example-succeeded.yaml
deleted file mode 100644
index 3f1b07ee..00000000
--- a/tests/e2e/state-transition/spark-cluster-example-succeeded.yaml
+++ /dev/null
@@ -1,32 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-apiVersion: spark.apache.org/v1
-kind: SparkCluster
-metadata:
- name: spark-cluster-succeeded-test
- namespace: default
-spec:
- runtimeVersions:
- sparkVersion: "4.2.0"
- clusterTolerations:
- instanceConfig:
- initWorkers: 1
- minWorkers: 1
- maxWorkers: 1
- sparkConf:
- spark.kubernetes.container.image: "apache/spark:{{SPARK_VERSION}}"
- spark.master.ui.title: "Spark Cluster E2E Test"
- spark.master.rest.host: "0.0.0.0"
- spark.ui.reverseProxy: "true"
diff --git a/tests/e2e/state-transition/spark-example-succeeded.yaml
b/tests/e2e/state-transition/spark-example-succeeded.yaml
deleted file mode 100644
index cf45b805..00000000
--- a/tests/e2e/state-transition/spark-example-succeeded.yaml
+++ /dev/null
@@ -1,31 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-
-apiVersion: spark.apache.org/v1
-kind: SparkApplication
-metadata:
- name: spark-job-succeeded-test
- namespace: default
-spec:
- mainClass: "org.apache.spark.examples.SparkPi"
- jars: "local:///opt/spark/examples/jars/spark-examples.jar"
- sparkConf:
- spark.executor.instances: "1"
- spark.kubernetes.container.image: "apache/spark:{{SPARK_VERSION}}"
- spark.kubernetes.authenticate.driver.serviceAccountName: "spark"
- runtimeVersions:
- sparkVersion: "4.2.0"
diff --git a/tests/e2e/suspend-events/chainsaw-test.yaml
b/tests/e2e/suspend-events/chainsaw-test.yaml
index bdbdb7f6..b39c50b2 100644
--- a/tests/e2e/suspend-events/chainsaw-test.yaml
+++ b/tests/e2e/suspend-events/chainsaw-test.yaml
@@ -38,7 +38,12 @@ spec:
- name: suspended-application-publishes-suspend-held-event
try:
- apply:
- file: spark-example-suspend.yaml
+ bindings:
+ - name: SPARK_APP_NAMESPACE
+ value: default
+ - name: SPARK_APPLICATION_NAME
+ value: spark-job-suspend-events-test
+ file: "../fixtures/spark-application/spark-example-suspend.yaml"
- assert:
timeout: 60s
resource:
@@ -94,7 +99,12 @@ spec:
- name: suspended-cluster-publishes-suspend-held-event
try:
- apply:
- file: spark-cluster-example-suspend.yaml
+ bindings:
+ - name: SPARK_APP_NAMESPACE
+ value: default
+ - name: SPARK_CLUSTER_NAME
+ value: spark-cluster-suspend-events-test
+ file: "../fixtures/spark-cluster/spark-cluster-example-suspend.yaml"
- assert:
timeout: 60s
resource:
diff --git a/tests/e2e/suspend-events/spark-cluster-example-suspend.yaml
b/tests/e2e/suspend-events/spark-cluster-example-suspend.yaml
deleted file mode 100644
index e4619891..00000000
--- a/tests/e2e/suspend-events/spark-cluster-example-suspend.yaml
+++ /dev/null
@@ -1,36 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-
-apiVersion: spark.apache.org/v1
-kind: SparkCluster
-metadata:
- name: spark-cluster-suspend-events-test
- namespace: default
-spec:
- suspend: true
- runtimeVersions:
- sparkVersion: "4.2.0"
- clusterTolerations:
- instanceConfig:
- initWorkers: 1
- minWorkers: 1
- maxWorkers: 1
- sparkConf:
- spark.kubernetes.container.image: "apache/spark:{{SPARK_VERSION}}"
- spark.master.ui.title: "Spark Cluster Suspend Events E2E Test"
- spark.master.rest.host: "0.0.0.0"
- spark.ui.reverseProxy: "true"
diff --git a/tests/e2e/suspend-events/spark-example-suspend.yaml
b/tests/e2e/suspend-events/spark-example-suspend.yaml
deleted file mode 100644
index 0c344820..00000000
--- a/tests/e2e/suspend-events/spark-example-suspend.yaml
+++ /dev/null
@@ -1,32 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-
-apiVersion: spark.apache.org/v1
-kind: SparkApplication
-metadata:
- name: spark-job-suspend-events-test
- namespace: default
-spec:
- suspend: true
- mainClass: "org.apache.spark.examples.SparkPi"
- jars: "local:///opt/spark/examples/jars/spark-examples.jar"
- sparkConf:
- spark.executor.instances: "1"
- spark.kubernetes.container.image: "apache/spark:{{SPARK_VERSION}}"
- spark.kubernetes.authenticate.driver.serviceAccountName: "spark"
- runtimeVersions:
- sparkVersion: "4.2.0"
diff --git a/tests/e2e/suspend/chainsaw-test.yaml
b/tests/e2e/suspend/chainsaw-test.yaml
index a70842c5..bff80456 100644
--- a/tests/e2e/suspend/chainsaw-test.yaml
+++ b/tests/e2e/suspend/chainsaw-test.yaml
@@ -23,7 +23,12 @@ spec:
steps:
- try:
- apply:
- file: spark-example-suspend.yaml
+ bindings:
+ - name: SPARK_APP_NAMESPACE
+ value: default
+ - name: SPARK_APPLICATION_NAME
+ value: spark-job-suspend-test
+ file: "../fixtures/spark-application/spark-example-suspend.yaml"
- sleep:
duration: 30s
- script:
@@ -75,7 +80,12 @@ spec:
kubectl delete sparkapplication spark-job-suspend-test -n default
--ignore-not-found=true
- try:
- apply:
- file: spark-cluster-example-suspend.yaml
+ bindings:
+ - name: SPARK_APP_NAMESPACE
+ value: default
+ - name: SPARK_CLUSTER_NAME
+ value: spark-cluster-suspend-test
+ file: "../fixtures/spark-cluster/spark-cluster-example-suspend.yaml"
- sleep:
duration: 30s
- script:
diff --git a/tests/e2e/watched-namespaces-file/chainsaw-test.yaml
b/tests/e2e/watched-namespaces-file/chainsaw-test.yaml
index 94c8f644..8220b963 100644
--- a/tests/e2e/watched-namespaces-file/chainsaw-test.yaml
+++ b/tests/e2e/watched-namespaces-file/chainsaw-test.yaml
@@ -39,7 +39,9 @@ spec:
bindings:
- name: SPARK_APP_NAMESPACE
value: spark-1
- file: spark-example.yaml
+ - name: SPARK_APPLICATION_NAME
+ value: spark-job-succeeded-test
+ file: "../fixtures/spark-application/spark-example.yaml"
- assert:
bindings:
- name: SPARK_APP_NAMESPACE
@@ -52,7 +54,9 @@ spec:
bindings:
- name: SPARK_APP_NAMESPACE
value: spark-2
- file: spark-example.yaml
+ - name: SPARK_APPLICATION_NAME
+ value: spark-job-succeeded-test
+ file: "../fixtures/spark-application/spark-example.yaml"
- sleep:
duration: 150s
- script:
diff --git a/tests/e2e/watched-namespaces/chainsaw-test.yaml
b/tests/e2e/watched-namespaces/chainsaw-test.yaml
index e72ff98e..167a138d 100644
--- a/tests/e2e/watched-namespaces/chainsaw-test.yaml
+++ b/tests/e2e/watched-namespaces/chainsaw-test.yaml
@@ -35,7 +35,9 @@ spec:
bindings:
- name: SPARK_APP_NAMESPACE
value: spark-1
- file: spark-example.yaml
+ - name: SPARK_APPLICATION_NAME
+ value: spark-job-succeeded-test
+ file: "../fixtures/spark-application/spark-example.yaml"
- assert:
bindings:
- name: SPARK_APP_NAMESPACE
@@ -48,7 +50,9 @@ spec:
bindings:
- name: SPARK_APP_NAMESPACE
value: spark-2
- file: spark-example.yaml
+ - name: SPARK_APPLICATION_NAME
+ value: spark-job-succeeded-test
+ file: "../fixtures/spark-application/spark-example.yaml"
- sleep:
duration: 60s
- script:
@@ -86,7 +90,9 @@ spec:
bindings:
- name: SPARK_APP_NAMESPACE
value: spark-3
- file: spark-example.yaml
+ - name: SPARK_APPLICATION_NAME
+ value: spark-job-succeeded-test
+ file: "../fixtures/spark-application/spark-example.yaml"
- sleep:
duration: 60s
- script:
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
