This is an automated email from the ASF dual-hosted git repository.
HyukjinKwon pushed a commit to branch branch-4.2
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.2 by this push:
new eb88c288870f [SPARK-60568][INFRA] Use master branch's
`free_disk_space` script for release builds
eb88c288870f is described below
commit eb88c288870fc534958e2f8748e512bf541a0c8e
Author: Hyukjin Kwon <[email protected]>
AuthorDate: Mon Oct 26 16:07:54 2026 -0700
[SPARK-60568][INFRA] Use master branch's `free_disk_space` script for
release builds
### What changes were proposed in this pull request?
This PR makes the release workflow (`.github/workflows/release.yml`) always
run master's `dev/free_disk_space` instead of the copy in the branch being
released, as SPARK-55115 did for `Dockerfile.base`. The previous step already
fetches `origin/master`, so the step now checks out master's script and runs it.
### Why are the changes needed?
The `ubuntu-latest` label started migrating to Ubuntu 26.04 on October 19
(https://github.com/actions/runner-images/issues/14748), and `apt-get` there
can no longer locate the `powershell` package. Master's script tolerates this
since SPARK-58934, which landed only in `master` and `branch-4.x`. The copies
in `branch-4.1`, `branch-4.2` and `branch-4.3` still run the following under
`set -e`, and the release workflow has no `continue-on-error` on that step:
```
sudo apt-get remove --purge -y azure-cli google-cloud-sdk mono-devel
powershell libgl1-mesa-dri
```
So releasing these branches fails before the release starts, e.g., a
`branch-4.3` run in
https://github.com/HeartSaVioR/spark/actions/runs/40288893806/job/121589366145:
```
+ sudo apt-get remove --purge -y azure-cli google-cloud-sdk mono-devel
powershell libgl1-mesa-dri
E: Unable to locate package powershell
Package 'google-cloud-sdk' is not installed, so not removed
##[error]Process completed with exit code 100.
```
The script only prepares the GitHub runner, which is the same
`ubuntu-latest` for every branch released, so it should come from master, which
is maintained against the current image. Backporting SPARK-58934 would also fix
this one case, but the release branches' copies would break again on the next
image change. Master's script already works on Ubuntu 26.04: the scheduled dry
run in `apache/spark` hits the same `Unable to locate package powershell`,
tolerates it, and passes, e.g., htt [...]
This does not affect release artifacts: the release builds from a fresh
clone inside Docker, and only `dev/free_disk_space` is taken from master.
### Does this PR introduce _any_ user-facing change?
No, dev-only.
### How was this patch tested?
Dispatched this workflow from this branch in my fork as a `branch-4.3` dry
run: https://github.com/HyukjinKwon/spark/actions/runs/40416194036. The "Free
up disk space" step, which fails in the run above, passed. The rest of the dry
run was still running when this PR was opened.
Also ran the workflow's git steps on a fresh shallow `branch-4.3` checkout
to confirm the checked-out `dev/free_disk_space` is executable and identical to
master's.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Opus 5.5)
This pull request and its description were written by Isaac.
Closes #59807 from HyukjinKwon/SPARK-60568-release-master-free-disk-space.
Authored-by: Hyukjin Kwon <[email protected]>
Signed-off-by: Hyukjin Kwon <[email protected]>
(cherry picked from commit a2f5edbe297e704169d267c92f289a8cb805f353)
Signed-off-by: Hyukjin Kwon <[email protected]>
---
.github/workflows/release.yml | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml
index ef9a19e3e018..1e38a76b6aed 100644
--- a/.github/workflows/release.yml
+++ b/.github/workflows/release.yml
@@ -127,10 +127,11 @@ jobs:
git checkout origin/master --
dev/create-release/spark-rm/Dockerfile.base
echo "Using master branch's Dockerfile.base for building release
image"
- name: Free up disk space
+ # Always use master's script, which is maintained against the current
runner image.
+ # Old branch scripts may fail on newer images (e.g., packages missing
from apt sources).
run: |
- if [ -f ./dev/free_disk_space ]; then
- ./dev/free_disk_space
- fi
+ git checkout origin/master -- dev/free_disk_space
+ ./dev/free_disk_space
- name: Release Apache Spark
env:
GIT_BRANCH: "${{ inputs.branch }}"
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
