zhengruifeng opened a new pull request, #56450:
URL: https://github.com/apache/spark/pull/56450
### What changes were proposed in this pull request?
In `.github/workflows/maven_test.yml` and
`.github/workflows/python_hosted_runner_test.yml`, add a step to the precompile
job (`precompile-maven` / `precompile`) that captures `git rev-parse HEAD`
right after the apache/spark checkout, expose it as a `head_sha` job output,
and switch the downstream `build` job's `actions/checkout` from `ref: ${{
inputs.branch }}` to `ref: ${{ needs.<precompile-job>.outputs.head_sha ||
inputs.branch }}`.
This is the same pinning that SPARK-56866 applied to `build_and_test.yml`,
with one adaptation. Unlike the mandatory `precondition` job there (downstream
jobs are skipped when it fails), the precompile jobs here are best-effort: they
run with `continue-on-error: true` and the `build` job proceeds with `if:
(!cancelled())` even when precompile fails. The `|| inputs.branch` fallback
keeps that degraded path intact: if the precompile job dies before resolving
the SHA, the `build` job resolves `inputs.branch` itself, exactly as today.
Without the fallback, an empty `ref:` would make `actions/checkout` fall back
to the default branch or the (master) event SHA, which is wrong for
`branch-4.x` runs.
### Why are the changes needed?
These two reusable workflows have the same cross-job checkout race that
SPARK-56866 fixed in `build_and_test.yml`: the precompile job builds Spark and
uploads the compiled output as an artifact, then each `build` matrix entry
independently re-resolves `ref: ${{ inputs.branch }}` at the moment its runner
picks it up. The matrix only starts after the full precompile build finishes
(tens of minutes), so the drift window is structurally long. If the branch
advances in between, a `build` entry checks out a newer commit than what was
precompiled and runs tests against stale compile artifacts extracted on top of
it, producing the same class of spurious mixed-commit failures described in
SPARK-56866.
Workflows covered by this change:
- `maven_test.yml` is called by 11 workflows: `build_maven.yml`,
`build_maven_java21.yml`, `build_maven_java21_arm.yml`,
`build_maven_java21_macos26.yml`, `build_maven_java25.yml`, and
`build_branch4{0,1,2}_maven*.yml`.
- `python_hosted_runner_test.yml` is called by `build_python_3.12_arm.yml`
and `build_python_3.12_macos26.yml`.
### Does this PR introduce _any_ user-facing change?
No. CI infrastructure only.
### How was this patch tested?
YAML syntax validated locally. These workflows are not exercised by PR CI
(their callers are schedule-triggered daily jobs), so the change takes effect
on their next scheduled runs; it mirrors the `build_and_test.yml` change that
has been running since 2026-05-19.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code (claude-fable-5)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
