This is an automated email from the ASF dual-hosted git repository.
zhengruifeng pushed a commit to branch branch-4.x
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.x by this push:
new 964f8da18a61 [SPARK-57388][INFRA] Pin downstream actions/checkout to a
single resolved SHA in maven_test.yml and python_hosted_runner_test.yml
964f8da18a61 is described below
commit 964f8da18a6102db1d07c6ccb4c81557f8dda7eb
Author: Ruifeng Zheng <[email protected]>
AuthorDate: Fri Jun 12 17:01:35 2026 +0800
[SPARK-57388][INFRA] Pin downstream actions/checkout to a single resolved
SHA in maven_test.yml and python_hosted_runner_test.yml
### What changes were proposed in this pull request?
In `.github/workflows/maven_test.yml` and
`.github/workflows/python_hosted_runner_test.yml`, add a step to the precompile
job (`precompile-maven` / `precompile`) that captures `git rev-parse HEAD`
right after the apache/spark checkout, expose it as a `head_sha` job output,
and switch the downstream `build` job's `actions/checkout` from `ref: ${{
inputs.branch }}` to `ref: ${{ needs.<precompile-job>.outputs.head_sha ||
inputs.branch }}`.
This is the same pinning that SPARK-56866 applied to `build_and_test.yml`,
with one adaptation. Unlike the mandatory `precondition` job there (downstream
jobs are skipped when it fails), the precompile jobs here are best-effort: they
run with `continue-on-error: true` and the `build` job proceeds with `if:
(!cancelled())` even when precompile fails. The `|| inputs.branch` fallback
keeps that degraded path intact: if the precompile job dies before resolving
the SHA, the `build` job res [...]
### Why are the changes needed?
These two reusable workflows have the same cross-job checkout race that
SPARK-56866 fixed in `build_and_test.yml`: the precompile job builds Spark and
uploads the compiled output as an artifact, then each `build` matrix entry
independently re-resolves `ref: ${{ inputs.branch }}` at the moment its runner
picks it up. The matrix only starts after the full precompile build finishes
(tens of minutes), so the drift window is structurally long. If the branch
advances in between, a `build` e [...]
Workflows covered by this change:
- `maven_test.yml` is called by 11 workflows: `build_maven.yml`,
`build_maven_java21.yml`, `build_maven_java21_arm.yml`,
`build_maven_java21_macos26.yml`, `build_maven_java25.yml`, and
`build_branch4{0,1,2}_maven*.yml`.
- `python_hosted_runner_test.yml` is called by `build_python_3.12_arm.yml`
and `build_python_3.12_macos26.yml`.
### Does this PR introduce _any_ user-facing change?
No. CI infrastructure only.
### How was this patch tested?
YAML syntax validated locally. These workflows are not exercised by PR CI
(their callers are schedule-triggered daily jobs), so the change takes effect
on their next scheduled runs; it mirrors the `build_and_test.yml` change that
has been running since 2026-05-19.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code (claude-fable-5)
Closes #56450 from zhengruifeng/ci-pin-checkout-sha-other-workflows-dev4.
Authored-by: Ruifeng Zheng <[email protected]>
Signed-off-by: Ruifeng Zheng <[email protected]>
(cherry picked from commit 0e1101aa9349ec5c4dd3abb8fd4d16fb05a25ae7)
Signed-off-by: Ruifeng Zheng <[email protected]>
---
.github/workflows/maven_test.yml | 9 ++++++++-
.github/workflows/python_hosted_runner_test.yml | 9 ++++++++-
2 files changed, 16 insertions(+), 2 deletions(-)
diff --git a/.github/workflows/maven_test.yml b/.github/workflows/maven_test.yml
index 967946c2c3b5..5b0cc81a6510 100644
--- a/.github/workflows/maven_test.yml
+++ b/.github/workflows/maven_test.yml
@@ -61,6 +61,9 @@ jobs:
# If this job fails or is cancelled, the matrix entries fall back to
# running `mvn clean install` locally as before.
continue-on-error: true
+ outputs:
+ # Pinned so the build job checks out the same snapshot, even if the
branch advances mid-run.
+ head_sha: ${{ steps.resolve-sha.outputs.head_sha }}
env:
HADOOP_PROFILE: ${{ inputs.hadoop }}
HIVE_PROFILE: hive2.3
@@ -73,6 +76,9 @@ jobs:
fetch-depth: 0
repository: apache/spark
ref: ${{ inputs.branch }}
+ - name: Resolve apache/spark HEAD SHA
+ id: resolve-sha
+ run: echo "head_sha=$(git rev-parse HEAD)" >> $GITHUB_OUTPUT
- name: Sync the current branch with the latest in Apache Spark
if: github.repository != 'apache/spark'
run: |
@@ -222,7 +228,8 @@ jobs:
with:
fetch-depth: 0
repository: apache/spark
- ref: ${{ inputs.branch }}
+ # Fall back to the branch when the precompile job failed before
resolving the SHA.
+ ref: ${{ needs.precompile-maven.outputs.head_sha || inputs.branch }}
- name: Sync the current branch with the latest in Apache Spark
if: github.repository != 'apache/spark'
run: |
diff --git a/.github/workflows/python_hosted_runner_test.yml
b/.github/workflows/python_hosted_runner_test.yml
index 1ba0c5529d5a..00ec4fdafc0d 100644
--- a/.github/workflows/python_hosted_runner_test.yml
+++ b/.github/workflows/python_hosted_runner_test.yml
@@ -65,6 +65,9 @@ jobs:
# If this job fails or is cancelled, the matrix entries fall back to
# running the SBT build locally as before.
continue-on-error: true
+ outputs:
+ # Pinned so the build job checks out the same snapshot, even if the
branch advances mid-run.
+ head_sha: ${{ steps.resolve-sha.outputs.head_sha }}
env:
HADOOP_PROFILE: ${{ inputs.hadoop }}
HIVE_PROFILE: hive2.3
@@ -77,6 +80,9 @@ jobs:
fetch-depth: 0
repository: apache/spark
ref: ${{ inputs.branch }}
+ - name: Resolve apache/spark HEAD SHA
+ id: resolve-sha
+ run: echo "head_sha=$(git rev-parse HEAD)" >> $GITHUB_OUTPUT
- name: Sync the current branch with the latest in Apache Spark
if: github.repository != 'apache/spark'
run: |
@@ -185,7 +191,8 @@ jobs:
with:
fetch-depth: 0
repository: apache/spark
- ref: ${{ inputs.branch }}
+ # Fall back to the branch when the precompile job failed before
resolving the SHA.
+ ref: ${{ needs.precompile.outputs.head_sha || inputs.branch }}
- name: Sync the current branch with the latest in Apache Spark
if: github.repository != 'apache/spark'
run: |
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
