This is an automated email from the ASF dual-hosted git repository.
dongjoon-hyun pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark-docker.git
The following commit(s) were added to refs/heads/master by this push:
new 7c6afb0 [SPARK-57290] Remove Spark 3.3/3.4 logic from
`add-dockerfiles.sh` (#119)
7c6afb0 is described below
commit 7c6afb0874b41c0768f3391b01cad1422d268ffe
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Fri Jun 5 19:36:09 2026 -0700
[SPARK-57290] Remove Spark 3.3/3.4 logic from `add-dockerfiles.sh` (#119)
### What changes were proposed in this pull request?
Cleans up `add-dockerfiles.sh`:
- Removes the dead `^3.3|^3.4` guard that skipped `java17` tags (3.3/3.4
are gone;
every remaining `3.x` image ships `java17`).
- Bumps the default `VERSION` from `3.5.0` to `4.1.0`.
- Updates the usage example comments.
### Why are the changes needed?
Spark 3.3/3.4 are EOL and already removed from this repo, so the
`^3.3|^3.4` branch
is dead code. Dropping it simplifies the `3.x` path, and the default bump
keeps the
no-argument run on the current release line.
### Does this PR introduce _any_ user-facing change?
No. It only touches the Dockerfile-scaffolding helper. (No-argument runs
now default
to `4.1.0` instead of `3.5.0`.)
### How was this patch tested?
`bash -n add-dockerfiles.sh`, plus running it for a `3.5.x` and a `4.x`
version and
confirming the generated tags match the committed layout.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Claude Opus 4.8)
---
add-dockerfiles.sh | 21 ++++++++-------------
1 file changed, 8 insertions(+), 13 deletions(-)
diff --git a/add-dockerfiles.sh b/add-dockerfiles.sh
index c418c02..f5924bc 100755
--- a/add-dockerfiles.sh
+++ b/add-dockerfiles.sh
@@ -21,12 +21,12 @@
# Generate dockerfiles for specified spark version.
#
# Examples:
-# - Add 3.3.0 dockerfiles:
+# - Add 4.1.0 dockerfiles:
# $ ./add-dockerfiles.sh
-# - Add 3.3.1 dockerfiles:
-# $ ./add-dockerfiles.sh 3.3.1
+# - Add 4.1.1 dockerfiles:
+# $ ./add-dockerfiles.sh 4.1.1
-VERSION=${1:-"3.5.0"}
+VERSION=${1:-"4.1.0"}
if echo $VERSION | grep -Eq "^4."; then
# 4.x default
@@ -47,16 +47,11 @@ elif echo $VERSION | grep -Eq "^3."; then
scala2.12-java11-python3-ubuntu
scala2.12-java11-r-ubuntu
scala2.12-java11-ubuntu
+ scala2.12-java17-python3-r-ubuntu
+ scala2.12-java17-python3-ubuntu
+ scala2.12-java17-r-ubuntu
+ scala2.12-java17-ubuntu
"
- # java17 images were added in 3.5.0. We need to skip java17 for 3.3.x and
3.4.x
- if ! echo $VERSION | grep -Eq "^3.3|^3.4"; then
- TAGS+="
- scala2.12-java17-python3-r-ubuntu
- scala2.12-java17-python3-ubuntu
- scala2.12-java17-r-ubuntu
- scala2.12-java17-ubuntu
- "
- fi
fi
for TAG in $TAGS; do
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
