This is an automated email from the ASF dual-hosted git repository.
zhengruifeng pushed a commit to branch branch-4.2
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.2 by this push:
new 1c261641cf3c [SPARK-56934][4.2][INFRA] Make build_infra_images_cache
workflow error tolerant
1c261641cf3c is described below
commit 1c261641cf3caac259a43f1a9b09c026a31bfc75
Author: Ruifeng Zheng <[email protected]>
AuthorDate: Wed May 20 10:02:06 2026 +0800
[SPARK-56934][4.2][INFRA] Make build_infra_images_cache workflow error
tolerant
Backport of #55972 to `branch-4.2`. Cherry-picked from master commit
83b2d0723543984b074591e1c04fcd95c92c1f3e and adapted for `branch-4.2`:
- Kept the `docker/build-push-action` SHA pin already in use on
`branch-4.2` (the master change was on a newer SHA).
- Also applied `continue-on-error: true` to the `Build and push (PySpark
with Python 3.10)` step, which is 4.2-only.
- Added a corresponding `pyspark-python-310` entry to the final "Fail if
any image build failed" summary step.
### What changes were proposed in this pull request?
Make the `build_infra_images_cache.yml` workflow tolerant of individual
image build failures:
- Add `continue-on-error: true` to each `Build and push` step (13 on
`branch-4.2`) so a failure in one does not abort the remaining builds. In
particular, a failure of the base `./dev/infra/` image build should no longer
prevent the other image builds from running.
- Add a final "Fail if any image build failed" step that runs with `if:
always()`, prints each build step's `outcome`, and exits non-zero if any was
`failure`.
### Why are the changes needed?
Today, a single image build failure aborts the workflow immediately,
leaving the remaining cache layers stale until someone re-triggers the job.
This is especially impactful when the first step (the `./dev/infra/` base
image) fails, because every subsequent image build is then skipped on that run.
With this change every image still gets a chance to build and refresh its cache
on each run, while the overall workflow still fails if any image build did not
succeed.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
YAML parses cleanly (`python3 -c "import yaml; yaml.safe_load(...)"`).
Verified all 13 build steps received `continue-on-error: true` and that the
final aggregator step references every build step's `outcome`.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Claude Opus 4.7)
Closes #56005 from zhengruifeng/build-infra-images-continue-on-error-4.2.
Authored-by: Ruifeng Zheng <[email protected]>
Signed-off-by: Ruifeng Zheng <[email protected]>
---
.github/workflows/build_infra_images_cache.yml | 42 ++++++++++++++++++++++++++
1 file changed, 42 insertions(+)
diff --git a/.github/workflows/build_infra_images_cache.yml
b/.github/workflows/build_infra_images_cache.yml
index 891ecad3f12d..ae0c5b0b9292 100644
--- a/.github/workflows/build_infra_images_cache.yml
+++ b/.github/workflows/build_infra_images_cache.yml
@@ -64,6 +64,7 @@ jobs:
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build and push
id: docker_build
+ continue-on-error: true
uses: docker/build-push-action@10e90e3645eae34f1e60eeb005ba3a3d33f178e8
with:
context: ./dev/infra/
@@ -76,6 +77,7 @@ jobs:
- name: Build and push (Documentation)
if: hashFiles('dev/spark-test-image/docs/Dockerfile') != ''
id: docker_build_docs
+ continue-on-error: true
uses: docker/build-push-action@10e90e3645eae34f1e60eeb005ba3a3d33f178e8
with:
context: ./dev/spark-test-image/docs/
@@ -89,6 +91,7 @@ jobs:
- name: Build and push (Linter)
if: hashFiles('dev/spark-test-image/lint/Dockerfile') != ''
id: docker_build_lint
+ continue-on-error: true
uses: docker/build-push-action@10e90e3645eae34f1e60eeb005ba3a3d33f178e8
with:
context: ./dev/spark-test-image/lint/
@@ -102,6 +105,7 @@ jobs:
- name: Build and push (SparkR)
if: hashFiles('dev/spark-test-image/sparkr/Dockerfile') != ''
id: docker_build_sparkr
+ continue-on-error: true
uses: docker/build-push-action@10e90e3645eae34f1e60eeb005ba3a3d33f178e8
with:
context: ./dev/spark-test-image/sparkr/
@@ -115,6 +119,7 @@ jobs:
- name: Build and push (PySpark with old dependencies)
if: hashFiles('dev/spark-test-image/python-minimum/Dockerfile') != ''
id: docker_build_pyspark_python_minimum
+ continue-on-error: true
uses: docker/build-push-action@10e90e3645eae34f1e60eeb005ba3a3d33f178e8
with:
context: ./dev/spark-test-image/python-minimum/
@@ -128,6 +133,7 @@ jobs:
- name: Build and push (PySpark with Python 3.10)
if: hashFiles('dev/spark-test-image/python-310/Dockerfile') != ''
id: docker_build_pyspark_python_310
+ continue-on-error: true
uses: docker/build-push-action@10e90e3645eae34f1e60eeb005ba3a3d33f178e8
with:
context: ./dev/spark-test-image/python-310/
@@ -141,6 +147,7 @@ jobs:
- name: Build and push (PySpark with Python 3.11)
if: hashFiles('dev/spark-test-image/python-311/Dockerfile') != ''
id: docker_build_pyspark_python_311
+ continue-on-error: true
uses: docker/build-push-action@10e90e3645eae34f1e60eeb005ba3a3d33f178e8
with:
context: ./dev/spark-test-image/python-311/
@@ -154,6 +161,7 @@ jobs:
- name: Build and push (PySpark Classic Only with Python 3.12)
if:
hashFiles('dev/spark-test-image/python-312-classic-only/Dockerfile') != ''
id: docker_build_pyspark_python_312_classic_only
+ continue-on-error: true
uses: docker/build-push-action@10e90e3645eae34f1e60eeb005ba3a3d33f178e8
with:
context: ./dev/spark-test-image/python-312-classic-only/
@@ -167,6 +175,7 @@ jobs:
- name: Build and push (PySpark with Python 3.12)
if: hashFiles('dev/spark-test-image/python-312/Dockerfile') != ''
id: docker_build_pyspark_python_312
+ continue-on-error: true
uses: docker/build-push-action@10e90e3645eae34f1e60eeb005ba3a3d33f178e8
with:
context: ./dev/spark-test-image/python-312/
@@ -180,6 +189,7 @@ jobs:
- name: Build and push (PySpark with Python 3.12 Pandas 3)
if: hashFiles('dev/spark-test-image/python-312-pandas-3/Dockerfile')
!= ''
id: docker_build_pyspark_python_312_pandas_3
+ continue-on-error: true
uses: docker/build-push-action@10e90e3645eae34f1e60eeb005ba3a3d33f178e8
with:
context: ./dev/spark-test-image/python-312-pandas-3/
@@ -193,6 +203,7 @@ jobs:
- name: Build and push (PySpark with Python 3.13)
if: hashFiles('dev/spark-test-image/python-313/Dockerfile') != ''
id: docker_build_pyspark_python_313
+ continue-on-error: true
uses: docker/build-push-action@10e90e3645eae34f1e60eeb005ba3a3d33f178e8
with:
context: ./dev/spark-test-image/python-313/
@@ -206,6 +217,7 @@ jobs:
- name: Build and push (PySpark with Python 3.14)
if: hashFiles('dev/spark-test-image/python-314/Dockerfile') != ''
id: docker_build_pyspark_python_314
+ continue-on-error: true
uses: docker/build-push-action@10e90e3645eae34f1e60eeb005ba3a3d33f178e8
with:
context: ./dev/spark-test-image/python-314/
@@ -219,6 +231,7 @@ jobs:
- name: Build and push (PySpark with Python 3.14 no GIL)
if: hashFiles('dev/spark-test-image/python-314-nogil/Dockerfile') != ''
id: docker_build_pyspark_python_314_nogil
+ continue-on-error: true
uses: docker/build-push-action@10e90e3645eae34f1e60eeb005ba3a3d33f178e8
with:
context: ./dev/spark-test-image/python-314-nogil/
@@ -229,3 +242,32 @@ jobs:
- name: Image digest (PySpark with Python 3.14 no GIL)
if: hashFiles('dev/spark-test-image/python-314-nogil/Dockerfile') != ''
run: echo ${{
steps.docker_build_pyspark_python_314_nogil.outputs.digest }}
+ - name: Fail if any image build failed
+ if: always()
+ run: |
+ status=0
+ check() {
+ local name="$1"
+ local outcome="$2"
+ echo "$name: $outcome"
+ if [ "$outcome" = "failure" ]; then
+ status=1
+ fi
+ }
+ check "base" "${{
steps.docker_build.outcome }}"
+ check "docs" "${{
steps.docker_build_docs.outcome }}"
+ check "lint" "${{
steps.docker_build_lint.outcome }}"
+ check "sparkr" "${{
steps.docker_build_sparkr.outcome }}"
+ check "pyspark-python-minimum" "${{
steps.docker_build_pyspark_python_minimum.outcome }}"
+ check "pyspark-python-310" "${{
steps.docker_build_pyspark_python_310.outcome }}"
+ check "pyspark-python-311" "${{
steps.docker_build_pyspark_python_311.outcome }}"
+ check "pyspark-python-312-classic-only" "${{
steps.docker_build_pyspark_python_312_classic_only.outcome }}"
+ check "pyspark-python-312" "${{
steps.docker_build_pyspark_python_312.outcome }}"
+ check "pyspark-python-312-pandas-3" "${{
steps.docker_build_pyspark_python_312_pandas_3.outcome }}"
+ check "pyspark-python-313" "${{
steps.docker_build_pyspark_python_313.outcome }}"
+ check "pyspark-python-314" "${{
steps.docker_build_pyspark_python_314.outcome }}"
+ check "pyspark-python-314-nogil" "${{
steps.docker_build_pyspark_python_314_nogil.outcome }}"
+ if [ "$status" -ne 0 ]; then
+ echo "::error::One or more image builds failed; see entries above
marked 'failure'."
+ fi
+ exit "$status"
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
