This is an automated email from the ASF dual-hosted git repository.
zhengruifeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new df06f9733ec6 [SPARK-60924][INFRA] Use GraphQL to count commits for
post-merge CI sampling
df06f9733ec6 is described below
commit df06f9733ec65706df2cecae6a66fa7bfc7c5e87
Author: Ruifeng Zheng <[email protected]>
AuthorDate: Fri Nov 6 08:56:45 2026 +0800
[SPARK-60924][INFRA] Use GraphQL to count commits for post-merge CI sampling
### What changes were proposed in this pull request?
Replace the sampling job's full-history checkout and `git rev-list --count
HEAD` with a GitHub GraphQL query for `Commit.history.totalCount` at the
triggering `GITHUB_SHA`.
The job uses the workflow token with the existing `contents: read`
permission. It rejects missing or invalid counts before calculating the
remainder. The one-in-ten sampling rule, job output, summary, and downstream
build conditions are preserved.
### Why are the changes needed?
The sampling job currently downloads Spark's entire Git history on every
master push, including pushes whose build is sampled out. Getting the count
through GraphQL avoids that checkout.
This implements [Nicholas Chammas's suggestion on
#60045](https://github.com/apache/spark/pull/60045#discussion_r4455095349). The
query also works for `branch-4.x` when this optimization is backported.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
- `actionlint` and `git diff --check`.
- Executed the actual workflow script against the live API for sampled and
unsampled master commits and a `branch-4.x` commit; all counts matched `git
rev-list --count` at the same SHA.
- Executed the script with controlled counts 1 through 30; only 10, 20, and
30 selected a build. API failures and empty, null, zero, negative, and
non-integer counts failed without emitting a build-selection output.
- Checked that the sampling job no longer requires checkout and preserves
its repository condition, permissions, output, summary, and dependent job
configuration.
- Confirmed GitHub CLI is included in the `ubuntu-slim` runner image.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Codex (GPT-6)
Closes #60124 from zhengruifeng/infra-ci-graphql-commit-count-20261105.
Authored-by: Ruifeng Zheng <[email protected]>
Signed-off-by: Ruifeng Zheng <[email protected]>
---
.github/workflows/build_main.yml | 24 +++++++++++++++++++-----
1 file changed, 19 insertions(+), 5 deletions(-)
diff --git a/.github/workflows/build_main.yml b/.github/workflows/build_main.yml
index 5016494b049e..72fe7b9c14ae 100644
--- a/.github/workflows/build_main.yml
+++ b/.github/workflows/build_main.yml
@@ -34,14 +34,28 @@ jobs:
outputs:
should-run: ${{ steps.count.outputs.should-run }}
steps:
- - name: Check out the full branch history
- uses: actions/checkout@v6
- with:
- fetch-depth: 0
- name: Check whether this is every tenth commit
id: count
+ env:
+ GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
- count=$(git rev-list --count HEAD)
+ count=$(gh api graphql \
+ -f owner="$GITHUB_REPOSITORY_OWNER" \
+ -f name="${GITHUB_REPOSITORY#*/}" \
+ -f sha="$GITHUB_SHA" \
+ -f query='
+ query($owner: String!, $name: String!, $sha: GitObjectID!) {
+ repository(owner: $owner, name: $name) {
+ object(oid: $sha) {
+ ... on Commit { history { totalCount } }
+ }
+ }
+ }' \
+ --jq '.data.repository.object.history.totalCount')
+ if [[ ! "$count" =~ ^[1-9][0-9]*$ ]]; then
+ echo "::error::Expected a positive commit count, got: $count"
+ exit 1
+ fi
remainder=$((count % 10))
if ((remainder == 0)); then
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]