This is an automated email from the ASF dual-hosted git repository.
dongjoon-hyun pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/spark-kubernetes-operator.git
The following commit(s) were added to refs/heads/main by this push:
new 2368a6af [SPARK-61105] Upgrade docker images to Apache Spark 4.3.0,
4.1.4, and 4.0.5
2368a6af is described below
commit 2368a6af81d36ef71ce383f887f31853c0be3889
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Sat Nov 7 09:25:45 2026 -0800
[SPARK-61105] Upgrade docker images to Apache Spark 4.3.0, 4.1.4, and 4.0.5
### What changes were proposed in this pull request?
This PR aims to upgrade docker images to Apache Spark 4.3.0, 4.1.4, and
4.0.5.
- [v4.3.0](https://github.com/apache/spark/releases/tag/v4.3.0) (2026-10-27)
- [v4.1.4](https://github.com/apache/spark/releases/tag/v4.1.4) (2026-10-21)
- [v4.0.5](https://github.com/apache/spark/releases/tag/v4.0.5) (2026-10-21)
Specifically,
- Update all examples, E2E tests, benchmark script, and docs from `4.2.0`
to `4.3.0`.
- Update the default Spark cluster image of `SparkClusterResourceSpec` from
`apache/spark:4.2.0` to `apache/spark:4.3.0`.
- Update `pi-with-comet.yaml` and `spark-connect-server-iceberg.yaml` from
`4.1.3` to `4.1.4`, and `pi-with-gluten.yaml` from `4.0.4` to `4.0.5`. They
stay on the Spark 4.1 and 4.0 lines because their `Apache DataFusion Comet`,
`Apache Iceberg`, and `Apache Gluten` artifacts are built for them.
### Why are the changes needed?
Since the `apache/spark` docker images of these releases are published, the
examples and tests should use them. The build dependency is already on Spark
4.3.0 since SPARK-60793, which did not change the images because they were not
published at that time.
### Does this PR introduce _any_ user-facing change?
Yes, the default Spark cluster image becomes `apache/spark:4.3.0` when
`spark.kubernetes.container.image` is not set. `docs/migration_guide.md`
describes it and how to restore the previous behavior. The other changes are
limited to docs, examples, and tests.
### How was this patch tested?
Pass the CIs.
I also verified that all referenced image tags exist in Docker Hub,
including the ones resolved from `apache/spark:{{SPARK_VERSION}}*` with
`spec.runtimeVersions.sparkVersion`:
- `4.3.0`, `4.3.0-scala`, `4.3.0-scala-java17`, `4.3.0-scala-java25`,
`4.3.0-python3`
- `4.1.4`, `4.1.4-scala`
- `4.0.5-scala`
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Opus 5.5
Closes #993 from dongjoon-hyun/SPARK-61105.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
README.md | 6 +++---
docs/migration_guide.md | 3 +++
docs/spark_custom_resources.md | 8 ++++----
examples/cluster-on-kueue.yaml | 2 +-
examples/cluster-on-yunikorn.yaml | 2 +-
examples/cluster-with-hpa-template.yaml | 2 +-
examples/cluster-with-hpa.yaml | 2 +-
examples/cluster-with-jws-filter.yaml | 2 +-
examples/cluster-with-template.yaml | 2 +-
examples/cluster.yaml | 2 +-
examples/dfs-read-write.yaml | 2 +-
examples/group-by-with-recovery-mode.yaml | 2 +-
examples/pi-java17.yaml | 2 +-
examples/pi-java25-deployment.yaml | 4 ++--
examples/pi-java25-with-executor-pvc-resize-plugin.yaml | 4 ++--
examples/pi-java25-with-executor-resize-plugin.yaml | 4 ++--
examples/pi-java25.yaml | 4 ++--
examples/pi-on-kueue.yaml | 2 +-
examples/pi-on-volcano.yaml | 2 +-
examples/pi-on-yunikorn.yaml | 2 +-
examples/pi-python.yaml | 2 +-
examples/pi-statefulset.yaml | 2 +-
examples/pi-with-comet.yaml | 4 ++--
examples/pi-with-driver-timeout.yaml | 2 +-
examples/pi-with-eventlog.yaml | 2 +-
examples/pi-with-gluten.yaml | 4 ++--
examples/pi-with-instanceconfig.yaml | 2 +-
examples/pi-with-on-demand-pvc.yaml | 2 +-
examples/pi-with-one-pod.yaml | 2 +-
examples/pi-with-spark-connect-plugin.yaml | 2 +-
examples/pi-with-template.yaml | 2 +-
examples/pi.yaml | 2 +-
examples/prod-cluster-with-three-workers.yaml | 2 +-
examples/qa-cluster-with-one-worker.yaml | 2 +-
examples/spark-connect-server-iceberg.yaml | 4 ++--
examples/spark-connect-server-usw2-az2.yaml | 2 +-
examples/spark-connect-server-with-spark-cluster.yaml | 2 +-
examples/spark-connect-server.yaml | 2 +-
examples/spark-history-server-with-jws-filter.yaml | 2 +-
examples/spark-history-server.yaml | 2 +-
examples/spark-thrift-server.yaml | 2 +-
examples/sql.yaml | 2 +-
examples/stream-word-count.yaml | 2 +-
examples/word-count-celeborn.yaml | 2 +-
examples/word-count.yaml | 2 +-
.../org/apache/spark/k8s/operator/SparkClusterResourceSpec.java | 2 +-
tests/benchmark/sparkapps.sh | 2 +-
.../driver-start-timeout/spark-example-driver-start-timeout.yaml | 2 +-
tests/e2e/events/spark-example-failed.yaml | 2 +-
tests/e2e/fixtures/spark-application/spark-example-suspend.yaml | 2 +-
tests/e2e/fixtures/spark-application/spark-example.yaml | 2 +-
.../e2e/fixtures/spark-cluster/spark-cluster-example-suspend.yaml | 2 +-
tests/e2e/fixtures/spark-cluster/spark-cluster-example.yaml | 2 +-
.../kueue-pods-ready/spark-cluster-example-pods-not-ready.yaml | 2 +-
tests/e2e/kueue-pods-ready/spark-cluster-example-pods-ready.yaml | 2 +-
.../spark-connect-example-dynamic-allocation.yaml | 2 +-
tests/e2e/kueue-pods-ready/spark-example-executors-not-ready.yaml | 2 +-
tests/e2e/kueue/spark-cluster-example-admission-check.yaml | 2 +-
tests/e2e/kueue/spark-cluster-example-preempted.yaml | 2 +-
tests/e2e/kueue/spark-example-admission-check.yaml | 2 +-
tests/e2e/kueue/spark-example-preempted.yaml | 2 +-
tests/e2e/kueue/spark-example-prioritized.yaml | 2 +-
tests/e2e/kueue/spark-example-retained-queued.yaml | 2 +-
tests/e2e/python/chainsaw-test.yaml | 4 ++--
.../resource-retain-duration/spark-example-retain-duration.yaml | 2 +-
.../spark-example-retain-on-failure.yaml | 2 +-
tests/e2e/resource-selector/chainsaw-test.yaml | 4 ++--
tests/e2e/spark-versions/chainsaw-test.yaml | 8 ++++----
tests/e2e/suspend/spark-example-running.yaml | 2 +-
69 files changed, 88 insertions(+), 85 deletions(-)
diff --git a/README.md b/README.md
index f3dc8c67..495a5c82 100644
--- a/README.md
+++ b/README.md
@@ -96,7 +96,7 @@ $ ./examples/submit-pi-to-prod.sh
{
"action" : "CreateSubmissionResponse",
"message" : "Driver successfully submitted as driver-20260110030233-0000",
- "serverSparkVersion" : "4.2.0",
+ "serverSparkVersion" : "4.3.0",
"submissionId" : "driver-20260110030233-0000",
"success" : true
}
@@ -105,7 +105,7 @@ $ curl
http://localhost:6066/v1/submissions/status/driver-20260110030233-0000/
{
"action" : "SubmissionStatusResponse",
"driverState" : "FINISHED",
- "serverSparkVersion" : "4.2.0",
+ "serverSparkVersion" : "4.3.0",
"submissionId" : "driver-20260110030233-0000",
"success" : true,
"workerHostPort" : "10.1.1.172:44233",
@@ -142,7 +142,7 @@ Events:
Normal Scheduling 1s yunikorn default/pi-on-yunikorn-0-driver
is queued and waiting for allocation
Normal Scheduled 1s yunikorn Successfully assigned
default/pi-on-yunikorn-0-driver to node docker-desktop
Normal PodBindSuccessful 1s yunikorn Pod
default/pi-on-yunikorn-0-driver is successfully bound to node docker-desktop
- Normal Pulled 0s kubelet Container image
"apache/spark:4.2.0-scala" already present on machine
+ Normal Pulled 0s kubelet Container image
"apache/spark:4.3.0-scala" already present on machine
Normal Created 0s kubelet Created container:
spark-kubernetes-driver
Normal Started 0s kubelet Started container
spark-kubernetes-driver
diff --git a/docs/migration_guide.md b/docs/migration_guide.md
index df474610..9184380f 100644
--- a/docs/migration_guide.md
+++ b/docs/migration_guide.md
@@ -225,6 +225,9 @@ The changes of the master and worker resources below apply
when the operator cre
a `SparkCluster` created after the upgrade. A running `SparkCluster` keeps its
resources until it
is recreated, e.g. by suspending and resuming it with
[`spec.suspend`](spark_custom_resources.md#suspend).
+- Since 1.1.0, the master and worker containers use the `apache/spark:4.3.0`
image instead of
+ `apache/spark:4.2.0` when `spark.kubernetes.container.image` is not set in
`spec.sparkConf`. To
+ restore the behavior before 1.1.0, set it to `apache/spark:4.2.0`.
- Since 1.1.0, the workers start the external shuffle service by default, so
that dynamic
allocation works out of the box. To restore the behavior before 1.1.0, set
`spark.shuffle.service.enabled` to `false` in `spec.sparkConf`.
diff --git a/docs/spark_custom_resources.md b/docs/spark_custom_resources.md
index 769d62f0..6501ba5c 100644
--- a/docs/spark_custom_resources.md
+++ b/docs/spark_custom_resources.md
@@ -51,12 +51,12 @@ spec:
spark.dynamicAllocation.enabled: "true"
spark.dynamicAllocation.maxExecutors: "3"
spark.kubernetes.authenticate.driver.serviceAccountName: "spark"
- spark.kubernetes.container.image: "apache/spark:4.2.0-scala"
+ spark.kubernetes.container.image: "apache/spark:4.3.0-scala"
applicationTolerations:
resourceRetainPolicy: OnFailure
ttlAfterStopMillis: 10000
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
```
After application is submitted, Operator will add status information to your
application based on
@@ -557,7 +557,7 @@ spec:
mainClass: "org.apache.spark.examples.SparkPi"
jars: "local:///opt/spark/examples/jars/spark-examples.jar"
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
```
* Setting it to `true` on a `SparkApplication` whose driver is requested, i.e.
from
@@ -650,7 +650,7 @@ spec:
sparkConf:
spark.executor.instances: "1"
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
```
The following table summarizes which Kueue features are supported. The items
after it describe the
diff --git a/examples/cluster-on-kueue.yaml b/examples/cluster-on-kueue.yaml
index 0585ba0c..fc2175f4 100644
--- a/examples/cluster-on-kueue.yaml
+++ b/examples/cluster-on-kueue.yaml
@@ -23,7 +23,7 @@ metadata:
kueue.x-k8s.io/queue-name: spark-queue
spec:
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
clusterTolerations:
instanceConfig:
initWorkers: 1
diff --git a/examples/cluster-on-yunikorn.yaml
b/examples/cluster-on-yunikorn.yaml
index 3ddd22e8..02529977 100644
--- a/examples/cluster-on-yunikorn.yaml
+++ b/examples/cluster-on-yunikorn.yaml
@@ -18,7 +18,7 @@ metadata:
name: cluster-on-yunikorn
spec:
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
clusterTolerations:
instanceConfig:
initWorkers: 1
diff --git a/examples/cluster-with-hpa-template.yaml
b/examples/cluster-with-hpa-template.yaml
index 4a4ce64e..bcec3e9a 100644
--- a/examples/cluster-with-hpa-template.yaml
+++ b/examples/cluster-with-hpa-template.yaml
@@ -18,7 +18,7 @@ metadata:
name: cluster-with-hpa-template
spec:
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
clusterTolerations:
instanceConfig:
initWorkers: 1
diff --git a/examples/cluster-with-hpa.yaml b/examples/cluster-with-hpa.yaml
index 7170a4e0..8e3f891b 100644
--- a/examples/cluster-with-hpa.yaml
+++ b/examples/cluster-with-hpa.yaml
@@ -18,7 +18,7 @@ metadata:
name: cluster-with-hpa
spec:
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
clusterTolerations:
instanceConfig:
initWorkers: 3
diff --git a/examples/cluster-with-jws-filter.yaml
b/examples/cluster-with-jws-filter.yaml
index 685ceb67..9938d7b8 100644
--- a/examples/cluster-with-jws-filter.yaml
+++ b/examples/cluster-with-jws-filter.yaml
@@ -18,7 +18,7 @@ metadata:
name: cluster-with-jws-filter
spec:
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
clusterTolerations:
instanceConfig:
initWorkers: 3
diff --git a/examples/cluster-with-template.yaml
b/examples/cluster-with-template.yaml
index cb454fea..c8ec86ea 100644
--- a/examples/cluster-with-template.yaml
+++ b/examples/cluster-with-template.yaml
@@ -18,7 +18,7 @@ metadata:
name: cluster-with-template
spec:
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
clusterTolerations:
instanceConfig:
initWorkers: 1
diff --git a/examples/cluster.yaml b/examples/cluster.yaml
index f887057b..c9259212 100644
--- a/examples/cluster.yaml
+++ b/examples/cluster.yaml
@@ -18,7 +18,7 @@ metadata:
name: cluster
spec:
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
clusterTolerations:
instanceConfig:
initWorkers: 3
diff --git a/examples/dfs-read-write.yaml b/examples/dfs-read-write.yaml
index 4c90c371..83531af1 100644
--- a/examples/dfs-read-write.yaml
+++ b/examples/dfs-read-write.yaml
@@ -35,4 +35,4 @@ spec:
spark.hadoop.fs.s3a.access.key: "test"
spark.hadoop.fs.s3a.secret.key: "test"
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
diff --git a/examples/group-by-with-recovery-mode.yaml
b/examples/group-by-with-recovery-mode.yaml
index cdde7c1f..69655970 100644
--- a/examples/group-by-with-recovery-mode.yaml
+++ b/examples/group-by-with-recovery-mode.yaml
@@ -36,4 +36,4 @@ spec:
applicationTolerations:
resourceRetainPolicy: OnFailure
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
diff --git a/examples/pi-java17.yaml b/examples/pi-java17.yaml
index a8be1342..4c1f7d54 100644
--- a/examples/pi-java17.yaml
+++ b/examples/pi-java17.yaml
@@ -23,4 +23,4 @@ spec:
spark.kubernetes.authenticate.driver.serviceAccountName: "spark"
spark.kubernetes.container.image:
"apache/spark:{{SPARK_VERSION}}-scala-java17"
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
diff --git a/examples/pi-java25-deployment.yaml
b/examples/pi-java25-deployment.yaml
index 5c32509c..c9dbd64a 100644
--- a/examples/pi-java25-deployment.yaml
+++ b/examples/pi-java25-deployment.yaml
@@ -24,6 +24,6 @@ spec:
spark.executor.instances: "3"
spark.kubernetes.allocation.pods.allocator: "deployment"
spark.kubernetes.authenticate.driver.serviceAccountName: "spark"
- spark.kubernetes.container.image: "apache/spark:4.2.0-scala-java25"
+ spark.kubernetes.container.image: "apache/spark:4.3.0-scala-java25"
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
diff --git a/examples/pi-java25-with-executor-pvc-resize-plugin.yaml
b/examples/pi-java25-with-executor-pvc-resize-plugin.yaml
index 9c392458..94fc4626 100644
--- a/examples/pi-java25-with-executor-pvc-resize-plugin.yaml
+++ b/examples/pi-java25-with-executor-pvc-resize-plugin.yaml
@@ -22,7 +22,7 @@ spec:
jars: "local:///opt/spark/examples/jars/spark-examples.jar"
sparkConf:
spark.kubernetes.authenticate.driver.serviceAccountName: "spark"
- spark.kubernetes.container.image: "apache/spark:4.2.0-scala-java25"
+ spark.kubernetes.container.image: "apache/spark:4.3.0-scala-java25"
spark.kubernetes.executor.pvc.resizeInterval: "5min"
spark.kubernetes.executor.pvc.resizeThreshold: "0.5"
spark.kubernetes.executor.pvc.resizeFactor: "1.0"
@@ -35,4 +35,4 @@ spec:
resourceRetainPolicy: OnFailure
ttlAfterStopMillis: 10000
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
diff --git a/examples/pi-java25-with-executor-resize-plugin.yaml
b/examples/pi-java25-with-executor-resize-plugin.yaml
index 067d701d..d0ff785d 100644
--- a/examples/pi-java25-with-executor-resize-plugin.yaml
+++ b/examples/pi-java25-with-executor-resize-plugin.yaml
@@ -24,7 +24,7 @@ spec:
spark.dynamicAllocation.enabled: "true"
spark.dynamicAllocation.maxExecutors: "1"
spark.kubernetes.authenticate.driver.serviceAccountName: "spark"
- spark.kubernetes.container.image: "apache/spark:4.2.0-scala-java25"
+ spark.kubernetes.container.image: "apache/spark:4.3.0-scala-java25"
spark.kubernetes.executor.resizeInterval: "1min"
spark.kubernetes.executor.resizeThreshold: "0.25"
spark.plugins:
"org.apache.spark.scheduler.cluster.k8s.ExecutorResizePlugin"
@@ -32,4 +32,4 @@ spec:
resourceRetainPolicy: OnFailure
ttlAfterStopMillis: 10000
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
diff --git a/examples/pi-java25.yaml b/examples/pi-java25.yaml
index 072f7a13..ca0307c2 100644
--- a/examples/pi-java25.yaml
+++ b/examples/pi-java25.yaml
@@ -21,9 +21,9 @@ spec:
jars: "local:///opt/spark/examples/jars/spark-examples.jar"
sparkConf:
spark.kubernetes.authenticate.driver.serviceAccountName: "spark"
- spark.kubernetes.container.image: "apache/spark:4.2.0-scala-java25"
+ spark.kubernetes.container.image: "apache/spark:4.3.0-scala-java25"
applicationTolerations:
resourceRetainPolicy: OnFailure
ttlAfterStopMillis: 10000
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
diff --git a/examples/pi-on-kueue.yaml b/examples/pi-on-kueue.yaml
index 35e85b2b..7f329d1e 100644
--- a/examples/pi-on-kueue.yaml
+++ b/examples/pi-on-kueue.yaml
@@ -33,4 +33,4 @@ spec:
spark.kubernetes.authenticate.driver.serviceAccountName: "spark"
spark.kubernetes.container.image: "apache/spark:{{SPARK_VERSION}}-scala"
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
diff --git a/examples/pi-on-volcano.yaml b/examples/pi-on-volcano.yaml
index c6b86733..9c81b57d 100644
--- a/examples/pi-on-volcano.yaml
+++ b/examples/pi-on-volcano.yaml
@@ -28,4 +28,4 @@ spec:
# spark.kubernetes.executor.pod.featureSteps:
"org.apache.spark.deploy.k8s.features.VolcanoFeatureStep"
spark.kubernetes.scheduler.name: "volcano"
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
diff --git a/examples/pi-on-yunikorn.yaml b/examples/pi-on-yunikorn.yaml
index 494d8707..0b9ada2a 100644
--- a/examples/pi-on-yunikorn.yaml
+++ b/examples/pi-on-yunikorn.yaml
@@ -29,4 +29,4 @@ spec:
spark.kubernetes.driver.annotation.yunikorn.apache.org/app-id: "{{APP_ID}}"
spark.kubernetes.executor.annotation.yunikorn.apache.org/app-id:
"{{APP_ID}}"
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
diff --git a/examples/pi-python.yaml b/examples/pi-python.yaml
index dd933946..b3468abf 100644
--- a/examples/pi-python.yaml
+++ b/examples/pi-python.yaml
@@ -22,4 +22,4 @@ spec:
spark.kubernetes.authenticate.driver.serviceAccountName: "spark"
spark.kubernetes.container.image: "apache/spark:{{SPARK_VERSION}}-python3"
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
diff --git a/examples/pi-statefulset.yaml b/examples/pi-statefulset.yaml
index c065257b..1a4a7d23 100644
--- a/examples/pi-statefulset.yaml
+++ b/examples/pi-statefulset.yaml
@@ -26,4 +26,4 @@ spec:
spark.kubernetes.authenticate.driver.serviceAccountName: "spark"
spark.kubernetes.container.image: "apache/spark:{{SPARK_VERSION}}-scala"
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
diff --git a/examples/pi-with-comet.yaml b/examples/pi-with-comet.yaml
index 490d860b..f8aa3f46 100644
--- a/examples/pi-with-comet.yaml
+++ b/examples/pi-with-comet.yaml
@@ -27,7 +27,7 @@ spec:
spark.driver.extraClassPath: "local:///comet/comet.jar"
spark.executor.extraClassPath: "local:///comet/comet.jar"
spark.kubernetes.authenticate.driver.serviceAccountName: "spark"
- spark.kubernetes.container.image: "apache/spark:4.1.3-scala"
+ spark.kubernetes.container.image: "apache/spark:4.1.4-scala"
spark.memory.offHeap.enabled: "true"
spark.memory.offHeap.size: "1g"
spark.plugins: "org.apache.spark.CometPlugin"
@@ -79,4 +79,4 @@ spec:
emptyDir:
sizeLimit: 200Mi
runtimeVersions:
- sparkVersion: "4.1.3"
+ sparkVersion: "4.1.4"
diff --git a/examples/pi-with-driver-timeout.yaml
b/examples/pi-with-driver-timeout.yaml
index 8db06464..7c391299 100644
--- a/examples/pi-with-driver-timeout.yaml
+++ b/examples/pi-with-driver-timeout.yaml
@@ -28,4 +28,4 @@ spec:
applicationTolerations:
resourceRetainPolicy: OnFailure
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
diff --git a/examples/pi-with-eventlog.yaml b/examples/pi-with-eventlog.yaml
index a10df9f3..956f2af2 100644
--- a/examples/pi-with-eventlog.yaml
+++ b/examples/pi-with-eventlog.yaml
@@ -34,4 +34,4 @@ spec:
spark.hadoop.fs.s3a.secret.key: "test"
spark.driver.memory: "2g"
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
diff --git a/examples/pi-with-gluten.yaml b/examples/pi-with-gluten.yaml
index 771a0974..6945a6a6 100644
--- a/examples/pi-with-gluten.yaml
+++ b/examples/pi-with-gluten.yaml
@@ -23,7 +23,7 @@ spec:
spark.driver.extraClassPath: "local:///gluten/gluten.jar"
spark.executor.extraClassPath: "local:///gluten/gluten.jar"
spark.kubernetes.authenticate.driver.serviceAccountName: "spark"
- spark.kubernetes.container.image: "apache/spark:4.0.4-scala"
+ spark.kubernetes.container.image: "apache/spark:4.0.5-scala"
spark.kubernetes.node.selector.kubernetes.io/arch: "amd64"
spark.memory.offHeap.enabled: "true"
spark.memory.offHeap.size: "1g"
@@ -81,4 +81,4 @@ spec:
emptyDir:
sizeLimit: 500Mi
runtimeVersions:
- sparkVersion: "4.0.4"
+ sparkVersion: "4.0.5"
diff --git a/examples/pi-with-instanceconfig.yaml
b/examples/pi-with-instanceconfig.yaml
index dba81497..5014004a 100644
--- a/examples/pi-with-instanceconfig.yaml
+++ b/examples/pi-with-instanceconfig.yaml
@@ -29,4 +29,4 @@ spec:
maxExecutors: 3
minExecutors: 3
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
diff --git a/examples/pi-with-on-demand-pvc.yaml
b/examples/pi-with-on-demand-pvc.yaml
index 8080002a..dba30539 100644
--- a/examples/pi-with-on-demand-pvc.yaml
+++ b/examples/pi-with-on-demand-pvc.yaml
@@ -29,4 +29,4 @@ spec:
spark.kubernetes.executor.volumes.persistentVolumeClaim.spark-local-dir-1.options.sizeLimit:
"1Gi"
spark.kubernetes.executor.volumes.persistentVolumeClaim.spark-local-dir-1.options.storageClass:
"local-path"
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
diff --git a/examples/pi-with-one-pod.yaml b/examples/pi-with-one-pod.yaml
index e1afb482..405d24bb 100644
--- a/examples/pi-with-one-pod.yaml
+++ b/examples/pi-with-one-pod.yaml
@@ -26,4 +26,4 @@ spec:
spark.kubernetes.authenticate.driver.serviceAccountName: "spark"
spark.kubernetes.container.image: "apache/spark:{{SPARK_VERSION}}-scala"
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
diff --git a/examples/pi-with-spark-connect-plugin.yaml
b/examples/pi-with-spark-connect-plugin.yaml
index cb7abfa9..1f014ea1 100644
--- a/examples/pi-with-spark-connect-plugin.yaml
+++ b/examples/pi-with-spark-connect-plugin.yaml
@@ -25,4 +25,4 @@ spec:
spark.kubernetes.authenticate.driver.serviceAccountName: "spark"
spark.kubernetes.container.image: "apache/spark:{{SPARK_VERSION}}-scala"
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
diff --git a/examples/pi-with-template.yaml b/examples/pi-with-template.yaml
index 79f83931..b70f85f6 100644
--- a/examples/pi-with-template.yaml
+++ b/examples/pi-with-template.yaml
@@ -33,4 +33,4 @@ spec:
priorityClassName: system-cluster-critical
terminationGracePeriodSeconds: 0
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
diff --git a/examples/pi.yaml b/examples/pi.yaml
index 896a4548..2b1e7b04 100644
--- a/examples/pi.yaml
+++ b/examples/pi.yaml
@@ -28,4 +28,4 @@ spec:
resourceRetainPolicy: OnFailure
ttlAfterStopMillis: 10000
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
diff --git a/examples/prod-cluster-with-three-workers.yaml
b/examples/prod-cluster-with-three-workers.yaml
index 4b2b74c3..81367728 100644
--- a/examples/prod-cluster-with-three-workers.yaml
+++ b/examples/prod-cluster-with-three-workers.yaml
@@ -18,7 +18,7 @@ metadata:
name: prod
spec:
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
clusterTolerations:
instanceConfig:
initWorkers: 3
diff --git a/examples/qa-cluster-with-one-worker.yaml
b/examples/qa-cluster-with-one-worker.yaml
index 28826530..d4de18a6 100644
--- a/examples/qa-cluster-with-one-worker.yaml
+++ b/examples/qa-cluster-with-one-worker.yaml
@@ -18,7 +18,7 @@ metadata:
name: qa
spec:
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
clusterTolerations:
instanceConfig:
initWorkers: 1
diff --git a/examples/spark-connect-server-iceberg.yaml
b/examples/spark-connect-server-iceberg.yaml
index 6c3b326b..075bb79e 100644
--- a/examples/spark-connect-server-iceberg.yaml
+++ b/examples/spark-connect-server-iceberg.yaml
@@ -26,7 +26,7 @@ spec:
spark.jars.ivy: "/tmp/.ivy2.5.2"
spark.jars.packages:
"org.apache.hadoop:hadoop-aws:3.4.2,org.apache.iceberg:iceberg-spark-runtime-4.1_2.13:1.12.0"
spark.kubernetes.authenticate.driver.serviceAccountName: "spark"
- spark.kubernetes.container.image: "apache/spark:4.1.3"
+ spark.kubernetes.container.image: "apache/spark:4.1.4"
spark.kubernetes.executor.podNamePrefix: "spark-connect-server-iceberg"
spark.scheduler.mode: "FAIR"
spark.sql.catalog.s3.type: "hadoop"
@@ -35,4 +35,4 @@ spec:
spark.sql.defaultCatalog: "s3"
spark.sql.extensions:
"org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions"
runtimeVersions:
- sparkVersion: "4.1.3"
+ sparkVersion: "4.1.4"
diff --git a/examples/spark-connect-server-usw2-az2.yaml
b/examples/spark-connect-server-usw2-az2.yaml
index 21c9997d..a287447a 100644
--- a/examples/spark-connect-server-usw2-az2.yaml
+++ b/examples/spark-connect-server-usw2-az2.yaml
@@ -34,4 +34,4 @@ spec:
nodeSelector:
topology.k8s.aws/zone-id: usw2-az2
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
diff --git a/examples/spark-connect-server-with-spark-cluster.yaml
b/examples/spark-connect-server-with-spark-cluster.yaml
index 5f19b702..b8a4f6b2 100644
--- a/examples/spark-connect-server-with-spark-cluster.yaml
+++ b/examples/spark-connect-server-with-spark-cluster.yaml
@@ -26,4 +26,4 @@ spec:
spark.kubernetes.container.image: "apache/spark:{{SPARK_VERSION}}-scala"
spark.ui.reverseProxy: "true"
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
diff --git a/examples/spark-connect-server.yaml
b/examples/spark-connect-server.yaml
index f6312f3d..47001648 100644
--- a/examples/spark-connect-server.yaml
+++ b/examples/spark-connect-server.yaml
@@ -24,4 +24,4 @@ spec:
spark.kubernetes.executor.podNamePrefix: "spark-connect-server"
spark.scheduler.mode: "FAIR"
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
diff --git a/examples/spark-history-server-with-jws-filter.yaml
b/examples/spark-history-server-with-jws-filter.yaml
index 8c612c7b..6a1153af 100644
--- a/examples/spark-history-server-with-jws-filter.yaml
+++ b/examples/spark-history-server-with-jws-filter.yaml
@@ -39,7 +39,7 @@ spec:
spark.ui.filters: "org.apache.spark.ui.JWSFilter"
spark.org.apache.spark.ui.JWSFilter.param.secretKey:
"VmlzaXQgaHR0cHM6Ly9zcGFyay5hcGFjaGUub3JnIHRvIGRvd25sb2FkIEFwYWNoZSBTcGFyay4="
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
applicationTolerations:
restartConfig:
restartPolicy: Always
diff --git a/examples/spark-history-server.yaml
b/examples/spark-history-server.yaml
index 0ad14fc3..af44cafc 100644
--- a/examples/spark-history-server.yaml
+++ b/examples/spark-history-server.yaml
@@ -36,7 +36,7 @@ spec:
spark.hadoop.fs.s3a.access.key: "test"
spark.hadoop.fs.s3a.secret.key: "test"
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
applicationTolerations:
restartConfig:
restartPolicy: Always
diff --git a/examples/spark-thrift-server.yaml
b/examples/spark-thrift-server.yaml
index 2d494d89..79792f36 100644
--- a/examples/spark-thrift-server.yaml
+++ b/examples/spark-thrift-server.yaml
@@ -24,7 +24,7 @@ spec:
spark.kubernetes.executor.podNamePrefix: "spark-thrift-server"
spark.scheduler.mode: "FAIR"
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
applicationTolerations:
restartConfig:
restartPolicy: Always
diff --git a/examples/sql.yaml b/examples/sql.yaml
index 68761ae4..ddd8566b 100644
--- a/examples/sql.yaml
+++ b/examples/sql.yaml
@@ -24,4 +24,4 @@ spec:
spark.kubernetes.authenticate.driver.serviceAccountName: "spark"
spark.kubernetes.container.image: "apache/spark:{{SPARK_VERSION}}-scala"
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
diff --git a/examples/stream-word-count.yaml b/examples/stream-word-count.yaml
index 6b9799ed..ea8f4a0c 100644
--- a/examples/stream-word-count.yaml
+++ b/examples/stream-word-count.yaml
@@ -31,4 +31,4 @@ spec:
spark.hadoop.fs.s3a.access.key: "test"
spark.hadoop.fs.s3a.secret.key: "test"
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
diff --git a/examples/word-count-celeborn.yaml
b/examples/word-count-celeborn.yaml
index 3d1c8678..48a63b4a 100644
--- a/examples/word-count-celeborn.yaml
+++ b/examples/word-count-celeborn.yaml
@@ -34,4 +34,4 @@ spec:
spark.sql.adaptive.localShuffleReader.enabled: "false"
spark.serializer: "org.apache.spark.serializer.KryoSerializer"
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
diff --git a/examples/word-count.yaml b/examples/word-count.yaml
index cbbb5801..3d74a228 100644
--- a/examples/word-count.yaml
+++ b/examples/word-count.yaml
@@ -24,4 +24,4 @@ spec:
spark.kubernetes.authenticate.driver.serviceAccountName: "spark"
spark.kubernetes.container.image: "apache/spark:{{SPARK_VERSION}}-scala"
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
diff --git
a/spark-submission-worker/src/main/java/org/apache/spark/k8s/operator/SparkClusterResourceSpec.java
b/spark-submission-worker/src/main/java/org/apache/spark/k8s/operator/SparkClusterResourceSpec.java
index b0a5509d..ecaba4c0 100644
---
a/spark-submission-worker/src/main/java/org/apache/spark/k8s/operator/SparkClusterResourceSpec.java
+++
b/spark-submission-worker/src/main/java/org/apache/spark/k8s/operator/SparkClusterResourceSpec.java
@@ -69,7 +69,7 @@ public class SparkClusterResourceSpec {
String clusterName = cluster.getMetadata().getName();
String scheduler = conf.get(Config.KUBERNETES_SCHEDULER_NAME().key(),
"default-scheduler");
String namespace = conf.get(Config.KUBERNETES_NAMESPACE().key(),
clusterNamespace);
- String image = conf.get(Config.CONTAINER_IMAGE().key(),
"apache/spark:4.2.0");
+ String image = conf.get(Config.CONTAINER_IMAGE().key(),
"apache/spark:4.3.0");
ClusterSpec spec = cluster.getSpec();
String version = spec.getRuntimeVersions().getSparkVersion();
StringBuilder options = new StringBuilder();
diff --git a/tests/benchmark/sparkapps.sh b/tests/benchmark/sparkapps.sh
index 53b2273e..4527e0e0 100755
--- a/tests/benchmark/sparkapps.sh
+++ b/tests/benchmark/sparkapps.sh
@@ -46,7 +46,7 @@ spec:
spark.kubernetes.driver.pod.excludedFeatureSteps:
"org.apache.spark.deploy.k8s.features.KerberosConfDriverFeatureStep"
spark.kubernetes.driver.request.cores: "100m"
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
---
EOF
done
diff --git
a/tests/e2e/driver-start-timeout/spark-example-driver-start-timeout.yaml
b/tests/e2e/driver-start-timeout/spark-example-driver-start-timeout.yaml
index 9eabf4b9..735288e4 100644
--- a/tests/e2e/driver-start-timeout/spark-example-driver-start-timeout.yaml
+++ b/tests/e2e/driver-start-timeout/spark-example-driver-start-timeout.yaml
@@ -31,4 +31,4 @@ spec:
spark.kubernetes.container.image: "apache/spark:non-existent-image"
spark.kubernetes.authenticate.driver.serviceAccountName: "spark"
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
diff --git a/tests/e2e/events/spark-example-failed.yaml
b/tests/e2e/events/spark-example-failed.yaml
index 47e0fe95..75d5d4b9 100644
--- a/tests/e2e/events/spark-example-failed.yaml
+++ b/tests/e2e/events/spark-example-failed.yaml
@@ -28,4 +28,4 @@ spec:
spark.kubernetes.container.image: "apache/spark:{{SPARK_VERSION}}"
spark.kubernetes.authenticate.driver.serviceAccountName: "spark"
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
diff --git a/tests/e2e/fixtures/spark-application/spark-example-suspend.yaml
b/tests/e2e/fixtures/spark-application/spark-example-suspend.yaml
index b12eef15..63290f4a 100644
--- a/tests/e2e/fixtures/spark-application/spark-example-suspend.yaml
+++ b/tests/e2e/fixtures/spark-application/spark-example-suspend.yaml
@@ -29,4 +29,4 @@ spec:
spark.kubernetes.container.image: "apache/spark:{{SPARK_VERSION}}"
spark.kubernetes.authenticate.driver.serviceAccountName: "spark"
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
diff --git a/tests/e2e/fixtures/spark-application/spark-example.yaml
b/tests/e2e/fixtures/spark-application/spark-example.yaml
index 7576eea0..6da6faa4 100644
--- a/tests/e2e/fixtures/spark-application/spark-example.yaml
+++ b/tests/e2e/fixtures/spark-application/spark-example.yaml
@@ -28,4 +28,4 @@ spec:
spark.kubernetes.container.image: "apache/spark:{{SPARK_VERSION}}-scala"
spark.kubernetes.authenticate.driver.serviceAccountName: "spark"
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
diff --git
a/tests/e2e/fixtures/spark-cluster/spark-cluster-example-suspend.yaml
b/tests/e2e/fixtures/spark-cluster/spark-cluster-example-suspend.yaml
index c43dfdf6..b42f7f6b 100644
--- a/tests/e2e/fixtures/spark-cluster/spark-cluster-example-suspend.yaml
+++ b/tests/e2e/fixtures/spark-cluster/spark-cluster-example-suspend.yaml
@@ -23,7 +23,7 @@ metadata:
spec:
suspend: true
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
clusterTolerations:
instanceConfig:
initWorkers: 1
diff --git a/tests/e2e/fixtures/spark-cluster/spark-cluster-example.yaml
b/tests/e2e/fixtures/spark-cluster/spark-cluster-example.yaml
index 2a5f3fcc..b8b59d1f 100644
--- a/tests/e2e/fixtures/spark-cluster/spark-cluster-example.yaml
+++ b/tests/e2e/fixtures/spark-cluster/spark-cluster-example.yaml
@@ -22,7 +22,7 @@ metadata:
namespace: ($SPARK_APP_NAMESPACE)
spec:
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
clusterTolerations:
instanceConfig:
initWorkers: 1
diff --git
a/tests/e2e/kueue-pods-ready/spark-cluster-example-pods-not-ready.yaml
b/tests/e2e/kueue-pods-ready/spark-cluster-example-pods-not-ready.yaml
index c6cf9d58..959d1304 100644
--- a/tests/e2e/kueue-pods-ready/spark-cluster-example-pods-not-ready.yaml
+++ b/tests/e2e/kueue-pods-ready/spark-cluster-example-pods-not-ready.yaml
@@ -24,7 +24,7 @@ metadata:
kueue.x-k8s.io/queue-name: ($LOCAL_QUEUE)
spec:
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
clusterTolerations:
instanceConfig:
initWorkers: 1
diff --git a/tests/e2e/kueue-pods-ready/spark-cluster-example-pods-ready.yaml
b/tests/e2e/kueue-pods-ready/spark-cluster-example-pods-ready.yaml
index 3f119575..2533be53 100644
--- a/tests/e2e/kueue-pods-ready/spark-cluster-example-pods-ready.yaml
+++ b/tests/e2e/kueue-pods-ready/spark-cluster-example-pods-ready.yaml
@@ -24,7 +24,7 @@ metadata:
kueue.x-k8s.io/queue-name: ($LOCAL_QUEUE)
spec:
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
clusterTolerations:
instanceConfig:
initWorkers: 1
diff --git
a/tests/e2e/kueue-pods-ready/spark-connect-example-dynamic-allocation.yaml
b/tests/e2e/kueue-pods-ready/spark-connect-example-dynamic-allocation.yaml
index 904b9622..25415826 100644
--- a/tests/e2e/kueue-pods-ready/spark-connect-example-dynamic-allocation.yaml
+++ b/tests/e2e/kueue-pods-ready/spark-connect-example-dynamic-allocation.yaml
@@ -40,4 +40,4 @@ spec:
spark.kubernetes.container.image: "apache/spark:{{SPARK_VERSION}}-scala"
spark.kubernetes.authenticate.driver.serviceAccountName: "spark"
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
diff --git a/tests/e2e/kueue-pods-ready/spark-example-executors-not-ready.yaml
b/tests/e2e/kueue-pods-ready/spark-example-executors-not-ready.yaml
index 2aa215b1..c8c47b2f 100644
--- a/tests/e2e/kueue-pods-ready/spark-example-executors-not-ready.yaml
+++ b/tests/e2e/kueue-pods-ready/spark-example-executors-not-ready.yaml
@@ -34,4 +34,4 @@ spec:
spark.kubernetes.container.image: "apache/spark:{{SPARK_VERSION}}-scala"
spark.kubernetes.authenticate.driver.serviceAccountName: "spark"
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
diff --git a/tests/e2e/kueue/spark-cluster-example-admission-check.yaml
b/tests/e2e/kueue/spark-cluster-example-admission-check.yaml
index e3591192..357b854c 100644
--- a/tests/e2e/kueue/spark-cluster-example-admission-check.yaml
+++ b/tests/e2e/kueue/spark-cluster-example-admission-check.yaml
@@ -24,7 +24,7 @@ metadata:
kueue.x-k8s.io/queue-name: ($LOCAL_QUEUE)
spec:
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
clusterTolerations:
instanceConfig:
initWorkers: 1
diff --git a/tests/e2e/kueue/spark-cluster-example-preempted.yaml
b/tests/e2e/kueue/spark-cluster-example-preempted.yaml
index 4ba10f00..a44ef9e5 100644
--- a/tests/e2e/kueue/spark-cluster-example-preempted.yaml
+++ b/tests/e2e/kueue/spark-cluster-example-preempted.yaml
@@ -30,7 +30,7 @@ metadata:
kueue.x-k8s.io/priority-class: ($LOW_PRIORITY_CLASS)
spec:
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
clusterTolerations:
instanceConfig:
initWorkers: 1
diff --git a/tests/e2e/kueue/spark-example-admission-check.yaml
b/tests/e2e/kueue/spark-example-admission-check.yaml
index b9241ac2..97992457 100644
--- a/tests/e2e/kueue/spark-example-admission-check.yaml
+++ b/tests/e2e/kueue/spark-example-admission-check.yaml
@@ -32,4 +32,4 @@ spec:
spark.kubernetes.container.image: "apache/spark:{{SPARK_VERSION}}-scala"
spark.kubernetes.authenticate.driver.serviceAccountName: "spark"
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
diff --git a/tests/e2e/kueue/spark-example-preempted.yaml
b/tests/e2e/kueue/spark-example-preempted.yaml
index 2f2eec03..8166f34f 100644
--- a/tests/e2e/kueue/spark-example-preempted.yaml
+++ b/tests/e2e/kueue/spark-example-preempted.yaml
@@ -39,4 +39,4 @@ spec:
spark.kubernetes.container.image: "apache/spark:{{SPARK_VERSION}}-scala"
spark.kubernetes.authenticate.driver.serviceAccountName: "spark"
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
diff --git a/tests/e2e/kueue/spark-example-prioritized.yaml
b/tests/e2e/kueue/spark-example-prioritized.yaml
index cc45a83f..7e232174 100644
--- a/tests/e2e/kueue/spark-example-prioritized.yaml
+++ b/tests/e2e/kueue/spark-example-prioritized.yaml
@@ -37,4 +37,4 @@ spec:
spark.kubernetes.container.image: "apache/spark:{{SPARK_VERSION}}-scala"
spark.kubernetes.authenticate.driver.serviceAccountName: "spark"
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
diff --git a/tests/e2e/kueue/spark-example-retained-queued.yaml
b/tests/e2e/kueue/spark-example-retained-queued.yaml
index 80998717..8db690ab 100644
--- a/tests/e2e/kueue/spark-example-retained-queued.yaml
+++ b/tests/e2e/kueue/spark-example-retained-queued.yaml
@@ -36,4 +36,4 @@ spec:
spark.kubernetes.container.image: "apache/spark:{{SPARK_VERSION}}-scala"
spark.kubernetes.authenticate.driver.serviceAccountName: "spark"
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
diff --git a/tests/e2e/python/chainsaw-test.yaml
b/tests/e2e/python/chainsaw-test.yaml
index 3d912bc4..75d7f7af 100644
--- a/tests/e2e/python/chainsaw-test.yaml
+++ b/tests/e2e/python/chainsaw-test.yaml
@@ -23,11 +23,11 @@ spec:
scenarios:
- bindings:
- name: "SPARK_VERSION"
- value: "4.2.0"
+ value: "4.3.0"
- name: "SCALA_VERSION"
value: "2.13"
- name: "IMAGE"
- value: "apache/spark:4.2.0-python3"
+ value: "apache/spark:4.3.0-python3"
steps:
- name: install-spark-application
try:
diff --git
a/tests/e2e/resource-retain-duration/spark-example-retain-duration.yaml
b/tests/e2e/resource-retain-duration/spark-example-retain-duration.yaml
index 87fbcf04..d52e0c5c 100644
--- a/tests/e2e/resource-retain-duration/spark-example-retain-duration.yaml
+++ b/tests/e2e/resource-retain-duration/spark-example-retain-duration.yaml
@@ -32,4 +32,4 @@ spec:
spark.kubernetes.container.image: "apache/spark:{{SPARK_VERSION}}-scala"
spark.kubernetes.authenticate.driver.serviceAccountName: "spark"
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
diff --git
a/tests/e2e/resource-retain-on-failure/spark-example-retain-on-failure.yaml
b/tests/e2e/resource-retain-on-failure/spark-example-retain-on-failure.yaml
index 23aeaa19..93c3c6b0 100644
--- a/tests/e2e/resource-retain-on-failure/spark-example-retain-on-failure.yaml
+++ b/tests/e2e/resource-retain-on-failure/spark-example-retain-on-failure.yaml
@@ -30,4 +30,4 @@ spec:
spark.kubernetes.container.image: "apache/spark:{{SPARK_VERSION}}-scala"
spark.kubernetes.authenticate.driver.serviceAccountName: "spark"
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
diff --git a/tests/e2e/resource-selector/chainsaw-test.yaml
b/tests/e2e/resource-selector/chainsaw-test.yaml
index 39dca419..f3622974 100644
--- a/tests/e2e/resource-selector/chainsaw-test.yaml
+++ b/tests/e2e/resource-selector/chainsaw-test.yaml
@@ -27,9 +27,9 @@ spec:
- name: SPARK_APPLICATION_NAME
value: "spark-example-resource-selector"
- name: SPARK_VERSION
- value: "4.2.0"
+ value: "4.3.0"
- name: IMAGE
- value: "apache/spark:4.2.0-scala"
+ value: "apache/spark:4.3.0-scala"
steps:
- name: install-spark-application-and-apply-label
try:
diff --git a/tests/e2e/spark-versions/chainsaw-test.yaml
b/tests/e2e/spark-versions/chainsaw-test.yaml
index c6c29087..ca6ef0b8 100644
--- a/tests/e2e/spark-versions/chainsaw-test.yaml
+++ b/tests/e2e/spark-versions/chainsaw-test.yaml
@@ -23,22 +23,22 @@ spec:
scenarios:
- bindings:
- name: "SPARK_VERSION"
- value: "4.2.0"
+ value: "4.3.0"
- name: "SCALA_VERSION"
value: "2.13"
- name: "JAVA_VERSION"
value: "17"
- name: "IMAGE"
- value: "apache/spark:4.2.0-scala-java17"
+ value: "apache/spark:4.3.0-scala-java17"
- bindings:
- name: "SPARK_VERSION"
- value: "4.2.0"
+ value: "4.3.0"
- name: "SCALA_VERSION"
value: "2.13"
- name: "JAVA_VERSION"
value: "21"
- name: "IMAGE"
- value: 'apache/spark:4.2.0-scala'
+ value: 'apache/spark:4.3.0-scala'
steps:
- name: install-spark-application
try:
diff --git a/tests/e2e/suspend/spark-example-running.yaml
b/tests/e2e/suspend/spark-example-running.yaml
index 5bbfaab2..3cb28d6f 100644
--- a/tests/e2e/suspend/spark-example-running.yaml
+++ b/tests/e2e/suspend/spark-example-running.yaml
@@ -32,4 +32,4 @@ spec:
spark.kubernetes.container.image: "apache/spark:{{SPARK_VERSION}}-scala"
spark.kubernetes.authenticate.driver.serviceAccountName: "spark"
runtimeVersions:
- sparkVersion: "4.2.0"
+ sparkVersion: "4.3.0"
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]