This is an automated email from the ASF dual-hosted git repository.
zhengruifeng pushed a commit to branch branch-4.2
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.2 by this push:
new 9a6bd1ef50b9 [SPARK-57388][INFRA] Pin downstream actions/checkout to a
single resolved SHA in maven_test.yml and python_hosted_runner_test.yml
9a6bd1ef50b9 is described below
commit 9a6bd1ef50b9cf0a7bd5d1cec165c5adc637d5f5
Author: Ruifeng Zheng <[email protected]>
AuthorDate: Fri Jun 12 17:01:35 2026 +0800
[SPARK-57388][INFRA] Pin downstream actions/checkout to a single resolved
SHA in maven_test.yml and python_hosted_runner_test.yml
### What changes were proposed in this pull request?
In `.github/workflows/maven_test.yml` and
`.github/workflows/python_hosted_runner_test.yml`, add a step to the precompile
job (`precompile-maven` / `precompile`) that captures `git rev-parse HEAD`
right after the apache/spark checkout, expose it as a `head_sha` job output,
and switch the downstream `build` job's `actions/checkout` from `ref: ${{
inputs.branch }}` to `ref: ${{ needs.<precompile-job>.outputs.head_sha ||
inputs.branch }}`.
This is the same pinning that SPARK-56866 applied to `build_and_test.yml`,
with one adaptation. Unlike the mandatory `precondition` job there (downstream
jobs are skipped when it fails), the precompile jobs here are best-effort: they
run with `continue-on-error: true` and the `build` job proceeds with `if:
(!cancelled())` even when precompile fails. The `|| inputs.branch` fallback
keeps that degraded path intact: if the precompile job dies before resolving
the SHA, the `build` job res [...]
### Why are the changes needed?
These two reusable workflows have the same cross-job checkout race that
SPARK-56866 fixed in `build_and_test.yml`: the precompile job builds Spark and
uploads the compiled output as an artifact, then each `build` matrix entry
independently re-resolves `ref: ${{ inputs.branch }}` at the moment its runner
picks it up. The matrix only starts after the full precompile build finishes
(tens of minutes), so the drift window is structurally long. If the branch
advances in between, a `build` e [...]
Workflows covered by this change:
- `maven_test.yml` is called by 11 workflows: `build_maven.yml`,
`build_maven_java21.yml`, `build_maven_java21_arm.yml`,
`build_maven_java21_macos26.yml`, `build_maven_java25.yml`, and
`build_branch4{0,1,2}_maven*.yml`.
- `python_hosted_runner_test.yml` is called by `build_python_3.12_arm.yml`
and `build_python_3.12_macos26.yml`.
### Does this PR introduce _any_ user-facing change?
No. CI infrastructure only.
### How was this patch tested?
YAML syntax validated locally. These workflows are not exercised by PR CI
(their callers are schedule-triggered daily jobs), so the change takes effect
on their next scheduled runs; it mirrors the `build_and_test.yml` change that
has been running since 2026-05-19.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code (claude-fable-5)
Closes #56450 from zhengruifeng/ci-pin-checkout-sha-other-workflows-dev4.
Authored-by: Ruifeng Zheng <[email protected]>
Signed-off-by: Ruifeng Zheng <[email protected]>
(cherry picked from commit 0e1101aa9349ec5c4dd3abb8fd4d16fb05a25ae7)
Signed-off-by: Ruifeng Zheng <[email protected]>
---
.github/workflows/maven_test.yml | 9 ++++++++-
.github/workflows/python_hosted_runner_test.yml | 9 ++++++++-
2 files changed, 16 insertions(+), 2 deletions(-)
diff --git a/.github/workflows/maven_test.yml b/.github/workflows/maven_test.yml
index bae85c699b9f..99054aa6fa43 100644
--- a/.github/workflows/maven_test.yml
+++ b/.github/workflows/maven_test.yml
@@ -61,6 +61,9 @@ jobs:
# If this job fails or is cancelled, the matrix entries fall back to
# running `mvn clean install` locally as before.
continue-on-error: true
+ outputs:
+ # Pinned so the build job checks out the same snapshot, even if the
branch advances mid-run.
+ head_sha: ${{ steps.resolve-sha.outputs.head_sha }}
env:
HADOOP_PROFILE: ${{ inputs.hadoop }}
HIVE_PROFILE: hive2.3
@@ -73,6 +76,9 @@ jobs:
fetch-depth: 0
repository: apache/spark
ref: ${{ inputs.branch }}
+ - name: Resolve apache/spark HEAD SHA
+ id: resolve-sha
+ run: echo "head_sha=$(git rev-parse HEAD)" >> $GITHUB_OUTPUT
- name: Sync the current branch with the latest in Apache Spark
if: github.repository != 'apache/spark'
run: |
@@ -222,7 +228,8 @@ jobs:
with:
fetch-depth: 0
repository: apache/spark
- ref: ${{ inputs.branch }}
+ # Fall back to the branch when the precompile job failed before
resolving the SHA.
+ ref: ${{ needs.precompile-maven.outputs.head_sha || inputs.branch }}
- name: Sync the current branch with the latest in Apache Spark
if: github.repository != 'apache/spark'
run: |
diff --git a/.github/workflows/python_hosted_runner_test.yml
b/.github/workflows/python_hosted_runner_test.yml
index 1ba0c5529d5a..00ec4fdafc0d 100644
--- a/.github/workflows/python_hosted_runner_test.yml
+++ b/.github/workflows/python_hosted_runner_test.yml
@@ -65,6 +65,9 @@ jobs:
# If this job fails or is cancelled, the matrix entries fall back to
# running the SBT build locally as before.
continue-on-error: true
+ outputs:
+ # Pinned so the build job checks out the same snapshot, even if the
branch advances mid-run.
+ head_sha: ${{ steps.resolve-sha.outputs.head_sha }}
env:
HADOOP_PROFILE: ${{ inputs.hadoop }}
HIVE_PROFILE: hive2.3
@@ -77,6 +80,9 @@ jobs:
fetch-depth: 0
repository: apache/spark
ref: ${{ inputs.branch }}
+ - name: Resolve apache/spark HEAD SHA
+ id: resolve-sha
+ run: echo "head_sha=$(git rev-parse HEAD)" >> $GITHUB_OUTPUT
- name: Sync the current branch with the latest in Apache Spark
if: github.repository != 'apache/spark'
run: |
@@ -185,7 +191,8 @@ jobs:
with:
fetch-depth: 0
repository: apache/spark
- ref: ${{ inputs.branch }}
+ # Fall back to the branch when the precompile job failed before
resolving the SHA.
+ ref: ${{ needs.precompile.outputs.head_sha || inputs.branch }}
- name: Sync the current branch with the latest in Apache Spark
if: github.repository != 'apache/spark'
run: |
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
