This is an automated email from the ASF dual-hosted git repository.

zhengruifeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new 13f1e8795b25 [SPARK-60834][INFRA] Run master post-merge builds every 
ten commits
13f1e8795b25 is described below

commit 13f1e8795b25bcccdb16d2f862981036f8fdbe5c
Author: Ruifeng Zheng <[email protected]>
AuthorDate: Wed Nov 4 16:45:03 2026 +0800

    [SPARK-60834][INFRA] Run master post-merge builds every ten commits
    
    ### What changes were proposed in this pull request?
    
    Tracks [SPARK-60834](https://issues.apache.org/jira/browse/SPARK-60834).
    
    Apply the same one-in-ten deterministic commit-count sampling used by 
branch-4.x after #57841 to master post-merge `build_main.yml` builds. A 
lightweight job checks out the full history and starts the existing reusable 
build only when `git rev-list --count HEAD` is divisible by ten. It records the 
decision in the job summary.
    
    The sampling check runs only on upstream master. Fork PR branches continue 
to build on every push, fork master syncs remain skipped, and other upstream 
branches retain their existing behavior. Scheduled workflows are unchanged.
    
    This samples the existing affected-module build; it does not turn the 
selected run into a full-suite build or accumulate the diffs from skipped 
pushes. Changes omitted by sampling may therefore wait for scheduled coverage 
to be exercised.
    
    ### Why are the changes needed?
    
    Reduce Spark's GitHub Actions consumption under the ASF resource limits by 
extending the existing branch-4.x sampling policy to master. For ordinary 
single-commit pushes, this skips nine out of ten expensive master post-merge 
builds.
    
    In the measured calendar week, a proportional one-in-ten cost estimate is 
approximately 233,048 runner-minutes against the 250,000 weekly limit. The 
estimated highest rolling five-day total is 180,643 against the 216,000 limit. 
This leaves approximately 16,952 runner-minutes of weekly headroom before the 
added sampling-check cost. The detailed usage breakdown and assumptions are in 
the PR comments.
    
    These estimates exclude the added sampling-check cost and require 
monitoring as activity changes. The existing workflow concurrency above the ASF 
maximum of 20 jobs remains a separate policy gap; sampling does not cap 
retained runs.
    
    ### Does this PR introduce _any_ user-facing change?
    
    No. This changes CI frequency only.
    
    ### How was this patch tested?
    
    - `actionlint` v1.7.11 passed for `.github/workflows/build_main.yml`.
    - Executed the actual sampling shell step with controlled commit counts 
from 1 through 30; only 10, 20, and 30 selected the build. A simulated Git 
failure failed the check without selecting the build.
    - Checked job conditions for sampled/unsampled master pushes, a failed 
sampling job, fork PR branches, fork master syncs, other upstream branches, and 
cancellation. Confirmed that skipped sampling checks do not suppress fork PR 
builds.
    - `git diff --check` passed.
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    Generated-by: Codex (GPT-6)
    
    Closes #60045 from zhengruifeng/infra-master-post-merge-sampling-20261103.
    
    Authored-by: Ruifeng Zheng <[email protected]>
    Signed-off-by: Ruifeng Zheng <[email protected]>
---
 .github/workflows/build_main.yml | 63 ++++++++++++++++++++++++++++++++++++++--
 1 file changed, 61 insertions(+), 2 deletions(-)

diff --git a/.github/workflows/build_main.yml b/.github/workflows/build_main.yml
index bdc7c21173bb..5016494b049e 100644
--- a/.github/workflows/build_main.yml
+++ b/.github/workflows/build_main.yml
@@ -25,13 +25,72 @@ on:
     - '**'
 
 jobs:
+  check-commit-count:
+    name: Check commit count
+    if: github.repository == 'apache/spark' && github.ref == 
'refs/heads/master'
+    runs-on: ubuntu-slim
+    permissions:
+      contents: read
+    outputs:
+      should-run: ${{ steps.count.outputs.should-run }}
+    steps:
+      - name: Check out the full branch history
+        uses: actions/checkout@v6
+        with:
+          fetch-depth: 0
+      - name: Check whether this is every tenth commit
+        id: count
+        run: |
+          count=$(git rev-list --count HEAD)
+          remainder=$((count % 10))
+
+          if ((remainder == 0)); then
+            should_run=true
+            icon=":white_check_mark:"
+            title="Commit count matched"
+            details="Commit count $count is divisible by 10."
+          else
+            should_run=false
+            icon=":pause_button:"
+            title="Commit count not matched"
+            details="Commit count $count is not divisible by 10."
+          fi
+
+          echo "should-run=$should_run" >> "$GITHUB_OUTPUT"
+          echo "Branch: $GITHUB_REF_NAME"
+          echo "Commit: $GITHUB_SHA"
+          echo "Commit count: $count"
+          echo "Remainder: $remainder"
+          echo "Should run: $should_run"
+          echo "Details: $details"
+          echo "::notice title=$title::$details"
+          echo "### $icon Commit count check" >> "$GITHUB_STEP_SUMMARY"
+          echo "" >> "$GITHUB_STEP_SUMMARY"
+          echo "- Branch: $GITHUB_REF_NAME" >> "$GITHUB_STEP_SUMMARY"
+          echo "- Commit: $GITHUB_SHA" >> "$GITHUB_STEP_SUMMARY"
+          echo "- Commit count: $count" >> "$GITHUB_STEP_SUMMARY"
+          echo "- Remainder: $remainder" >> "$GITHUB_STEP_SUMMARY"
+          echo "- Should run: $should_run" >> "$GITHUB_STEP_SUMMARY"
+          echo "- Details: $details" >> "$GITHUB_STEP_SUMMARY"
+
   call-build-and-test:
     permissions:
       packages: write
     name: Run
+    needs: check-commit-count
     # Skip pushes to `master` on forks: the "Sync fork" button mirrors
     # apache/spark and would otherwise re-run the full build on every sync.
+    # The sampling check is skipped outside apache/spark master. GitHub 
otherwise
+    # applies an implicit success() requirement to dependencies, so 
!cancelled()
+    # allows those builds to run when the sampling job is skipped.
     if: >-
-      github.repository == 'apache/spark'
-      || github.ref != 'refs/heads/master'
+      (!cancelled())
+      && (github.repository == 'apache/spark' || github.ref != 
'refs/heads/master')
+      && (
+        needs.check-commit-count.result == 'skipped'
+        || (
+          needs.check-commit-count.result == 'success'
+          && needs.check-commit-count.outputs.should-run == 'true'
+        )
+      )
     uses: ./.github/workflows/build_and_test.yml


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to