This is an automated email from the ASF dual-hosted git repository.
400Ping pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/mahout.git
The following commit(s) were added to refs/heads/main by this push:
new 49eb81ddf [CI] Fix the main ruleset so the PMC bypass takes effect
(#1529)
49eb81ddf is described below
commit 49eb81ddfe66e267bbf8ff5ee3b3a8d0e4af47cf
Author: Jie-Kai Chang <[email protected]>
AuthorDate: Wed Oct 14 13:43:21 2026 +0800
[CI] Fix the main ruleset so the PMC bypass takes effect (#1529)
* [CI] Fix the main ruleset so the PMC bypass takes effect
Signed-off-by: 400Ping <[email protected]>
* [CI] Run Auto-merge gate on the PR so its check lands on the PR head
Signed-off-by: 400Ping <[email protected]>
---------
Signed-off-by: 400Ping <[email protected]>
---
.asf.yaml | 112 ++++++++--------------
.github/scripts/automerge_gate.py | 103 ++++++++++++++------
.github/workflows/automerge-gate.yml | 52 +++++-----
docs/community/pr-policy-and-review-guidelines.md | 14 ++-
4 files changed, 151 insertions(+), 130 deletions(-)
diff --git a/.asf.yaml b/.asf.yaml
index 9736d574e..c1ca308e7 100644
--- a/.asf.yaml
+++ b/.asf.yaml
@@ -55,44 +55,38 @@ github:
# enable updating head branches of pull requests
allow_update_branch: true
- protected_branches:
- main:
- required_linear_history: true
- required_signatures: false
- # PR policy (docs/community/pr-policy-and-review-guidelines.md): a PR
needs
- # at least one approving review from a committer before it can be merged,
- # unless it was opened by a PMC member (.github/pmc-members.txt).
- # GitHub's own required_approving_review_count (as used by Airflow)
cannot
- # exempt PMC members, so the rule is enforced through this required
status
- # check produced by .github/workflows/pr-approval-check.yml.
- # "Auto-merge gate" (.github/workflows/automerge-gate.yml) keeps
- # auto-merge (allow_auto_merge above) from merging a PR whose CI failed:
- # GitHub auto-merge only waits for *required* checks, and the CI jobs
- # themselves are deliberately not required so that a committer can still
- # merge manually after a failure. The gate waits for all pull_request
- # workflows, disables auto-merge on failure, and then passes.
- #
- # MIGRATION: the same rules are defined as the "main" ruleset below,
- # which additionally lets PMC members bypass them. GitHub applies the
- # most restrictive of classic protection and rulesets, so the bypass
- # only works once this classic block is removed in a follow-up change
- # (after confirming that the ruleset was created).
- required_status_checks:
- strict: false
- checks:
- - context: "Committer approval"
- app_id: 15368 # GitHub Actions
- - context: "Auto-merge gate"
- app_id: 15368 # GitHub Actions
-
- # Branch rules for main, as a ruleset (raw Rulesets API payload) because
- # only rulesets support bypass actors: PMC members can merge a pull request
- # from the GitHub UI ("bypass rules") without waiting for the required
- # checks, e.g. for urgent fixes or when the ASF runner queue is stuck.
- # bypass_mode "pull_request" means they still have to open a pull request
- # (no direct pushes) and every bypass is recorded in the audit log.
- # The actor ids are the GitHub user ids of .github/pmc-members.txt; update
- # both when the PMC roster changes (`gh api users/<login> -q .id`).
+ # Branch protection for main is a ruleset (not classic protected_branches)
+ # because only rulesets support bypass actors. Classic protection and
+ # rulesets are both enforced when both exist, so there must be no
+ # protected_branches entry for main.
+ #
+ # PR policy (docs/community/pr-policy-and-review-guidelines.md): a PR needs
+ # at least one approving review from a committer before it can be merged,
+ # unless it was opened by a PMC member (.github/pmc-members.txt).
+ # GitHub's own required_approving_review_count (as used by Airflow) cannot
+ # exempt PMC members, so the rule is enforced through the required status
+ # check "Committer approval" produced by
.github/workflows/pr-approval-check.yml.
+ # "Auto-merge gate" (.github/workflows/automerge-gate.yml) keeps auto-merge
+ # (allow_auto_merge above) from merging a PR whose CI failed: GitHub
+ # auto-merge only waits for *required* checks, and the CI jobs themselves are
+ # deliberately not required so that a committer can still merge manually
+ # after a failure. The gate waits for all pull_request workflows, disables
+ # auto-merge on failure, and then passes.
+ #
+ # PMC members are bypass actors: on any pull request they get "Merge without
+ # waiting for requirements to be met (bypass rules)", e.g. for urgent fixes
+ # or when the ASF runner queue is stuck. bypass_mode "pull_request" means
+ # they still have to open a pull request (no direct pushes) and every bypass
+ # is recorded in the audit log. GitHub has no notion of a PMC and the ASF org
+ # has no PMC team, so they are listed by GitHub user id, one line per member
+ # of .github/pmc-members.txt; update both when the PMC roster changes
+ # (`gh api users/<login> -q .id`).
+ #
+ # Raw Rulesets API payload: asfyaml only coerces a known set of keys to
+ # integers (actor_id, required_approving_review_count, ...). integration_id
+ # is not one of them and arrives at GitHub as a string, which is rejected
+ # ("Invalid property /rules/4"), so the required checks are not pinned to
+ # the GitHub Actions app (any source).
rulesets:
- name: "main"
target: branch
@@ -103,36 +97,16 @@ github:
- "~DEFAULT_BRANCH"
exclude: []
bypass_actors:
- - actor_id: 692523 # andrewmusselman
- actor_type: User
- bypass_mode: pull_request
- - actor_id: 7681565 # andrewpalumbo
- actor_type: User
- bypass_mode: pull_request
- - actor_id: 105915352 # guan404ming
- actor_type: User
- bypass_mode: pull_request
- - actor_id: 70953 # MaineC
- actor_type: User
- bypass_mode: pull_request
- - actor_id: 43886578 # 400Ping
- actor_type: User
- bypass_mode: pull_request
- - actor_id: 101171023 # rich7420
- actor_type: User
- bypass_mode: pull_request
- - actor_id: 409707 # sscdotopen
- actor_type: User
- bypass_mode: pull_request
- - actor_id: 135417 # magsol
- actor_type: User
- bypass_mode: pull_request
- - actor_id: 176382 # sslavic
- actor_type: User
- bypass_mode: pull_request
- - actor_id: 5852441 # rawkintrevo
- actor_type: User
- bypass_mode: pull_request
+ - { actor_type: User, bypass_mode: pull_request, actor_id: 692523 }
# andrewmusselman
+ - { actor_type: User, bypass_mode: pull_request, actor_id: 7681565 }
# andrewpalumbo
+ - { actor_type: User, bypass_mode: pull_request, actor_id: 105915352 }
# guan404ming
+ - { actor_type: User, bypass_mode: pull_request, actor_id: 70953 }
# MaineC
+ - { actor_type: User, bypass_mode: pull_request, actor_id: 43886578 }
# 400Ping
+ - { actor_type: User, bypass_mode: pull_request, actor_id: 101171023 }
# rich7420
+ - { actor_type: User, bypass_mode: pull_request, actor_id: 409707 }
# sscdotopen
+ - { actor_type: User, bypass_mode: pull_request, actor_id: 135417 }
# magsol
+ - { actor_type: User, bypass_mode: pull_request, actor_id: 176382 }
# sslavic
+ - { actor_type: User, bypass_mode: pull_request, actor_id: 5852441 }
# rawkintrevo
rules:
# Only bypass actors can delete or force-push main.
- type: deletion
@@ -152,9 +126,7 @@ github:
strict_required_status_checks_policy: false
required_status_checks:
- context: "Committer approval"
- integration_id: 15368 # GitHub Actions
- context: "Auto-merge gate"
- integration_id: 15368 # GitHub Actions
environments:
pypi:
diff --git a/.github/scripts/automerge_gate.py
b/.github/scripts/automerge_gate.py
index 9c4a8887e..36214017a 100644
--- a/.github/scripts/automerge_gate.py
+++ b/.github/scripts/automerge_gate.py
@@ -22,25 +22,34 @@ This backs the ``Auto-merge gate`` required status check on
``main`` (see
Background: GitHub auto-merge only waits for *required* status checks. Making
every CI job required would also block committers from merging manually when
they decide a failure is acceptable (e.g. a flaky external check), which is
-not what the project wants. So instead a single required check waits for the
-``pull_request`` workflows of the head commit and then:
+not what the project wants. So instead a single required check, run by a
+``pull_request`` job on the PR itself, polls the other ``pull_request``
+workflows of the head commit until all of them have completed and then:
-* while any of them is still running -> exit 1 ("waiting"); the check is
- re-evaluated every time one of them completes (``workflow_run`` trigger);
* if every one of them passed (or was skipped) -> exit 0; auto-merge, if
enabled, merges the PR;
* if any of them failed -> **disable auto-merge** on the PR (if it was
enabled) and leave a comment, then exit 0 so that a committer can still
merge manually. The failed CI job stays visible on the PR.
+While it is polling the check is "in progress", so auto-merge cannot fire
+early. It gives up after ``MAX_WAIT_MINUTES`` (exit 1; re-run the job).
+
+The job runs on a ``pull_request`` event (not ``workflow_run``) so that its
+check run is attached to the PR head commit; a ``workflow_run`` job's check
+run lands on the default branch commit and never satisfies the required
+check. The price is that ``GITHUB_TOKEN`` is read-only for pull requests from
+forks: if auto-merge cannot be disabled there, the gate fails instead, which
+equally keeps auto-merge from merging.
+
Urgent changes do not go through this script at all: PMC members are bypass
actors of the ``main`` ruleset in ``.asf.yaml`` and can merge a pull request
from the GitHub UI without waiting for any required check.
Only the Python standard library is used so the script runs in CI and locally:
- GITHUB_TOKEN=$(gh auth token) DRY_RUN=1 python
.github/scripts/automerge_gate.py \\
- --repo apache/mahout --sha <head sha>
+ GITHUB_TOKEN=$(gh auth token) DRY_RUN=1 MAX_WAIT_MINUTES=0 \\
+ python .github/scripts/automerge_gate.py --repo apache/mahout --sha
<head sha>
``DRY_RUN=1`` performs no mutation (auto-merge is not disabled, no comment).
"""
@@ -51,6 +60,7 @@ import argparse
import json
import os
import sys
+import time
import urllib.error
import urllib.parse
import urllib.request
@@ -60,8 +70,11 @@ API_URL = os.environ.get("GITHUB_API_URL",
"https://api.github.com";)
GRAPHQL_URL = os.environ.get("GITHUB_GRAPHQL_URL",
"https://api.github.com/graphql";)
# Only workflows triggered by these events are CI for the pull request. In
# particular this excludes "PR Approval Check" (pull_request_target), which is
-# its own required status check, and this gate itself (workflow_run).
+# its own required status check.
PR_EVENTS = {"pull_request"}
+# The gate's own workflow (name: in automerge-gate.yml) must not wait for
itself.
+GATE_WORKFLOW = os.environ.get("GATE_WORKFLOW", "Auto-merge Gate")
+POLL_SECONDS = int(os.environ.get("POLL_SECONDS", "30"))
# Conclusions that do not count as a CI failure.
OK_CONCLUSIONS = {"success", "skipped", "neutral"}
# A run that is waiting for a maintainer to approve it (fork PRs) is pending,
@@ -137,6 +150,8 @@ def latest_pr_runs(runs: list[dict[str, Any]]) -> dict[str,
dict[str, Any]]:
if run.get("event") not in PR_EVENTS:
continue
name = run.get("name") or str(run.get("workflow_id"))
+ if name == GATE_WORKFLOW:
+ continue
if name not in latest or run["id"] > latest[name]["id"]:
latest[name] = run
return latest
@@ -230,31 +245,53 @@ def failure_comment(sha: str, failed: list[dict[str,
Any]], enabled_by: str) ->
return "\n".join(lines)
-def gate(token: str, repo: str, sha: str, dry_run: bool) -> tuple[bool,
list[str]]:
- """Evaluate the gate. Returns ``(ok, log_lines)``."""
- lines: list[str] = []
- runs = latest_pr_runs(workflow_runs_for_sha(token, repo, sha))
- if not runs:
- # Cannot happen when triggered by workflow_run, but be explicit.
- lines.append(f"No pull_request workflow runs found for {sha[:7]}.")
- return False, lines
+def wait_for_runs(
+ token: str, repo: str, sha: str, max_wait_minutes: float
+) -> tuple[list[dict[str, Any]], list[dict[str, Any]], list[str]]:
+ """Poll until no pull_request workflow run of ``sha`` is pending.
- pending: list[dict[str, Any]] = []
- failed: list[dict[str, Any]] = []
+ Returns ``(pending, failed, log_lines)``; ``pending`` is non-empty only
+ when the wait timed out.
+ """
+ deadline = time.monotonic() + max_wait_minutes * 60
+ attempt = 0
+ while True:
+ attempt += 1
+ runs = latest_pr_runs(workflow_runs_for_sha(token, repo, sha))
+ pending = [run for run in runs.values() if classify(run) == "pending"]
+ if runs and not pending:
+ break
+ names = ", ".join(sorted(run["name"] for run in pending)) or "(no runs
yet)"
+ if time.monotonic() >= deadline:
+ lines = [f"Still waiting for: {names}."]
+ return pending, [], lines
+ if attempt == 1 or attempt % 10 == 0:
+ print(f"Waiting for: {names} ...", flush=True)
+ time.sleep(POLL_SECONDS)
+
+ lines = []
+ failed = []
for name in sorted(runs):
run = runs[name]
state = classify(run)
lines.append(
f"{name}: {run.get('status')}/{run.get('conclusion') or '-'} ->
{state}"
)
- if state == "pending":
- pending.append(run)
- elif state == "failed":
+ if state == "failed":
failed.append(run)
+ return [], failed, lines
+
+def gate(
+ token: str, repo: str, sha: str, dry_run: bool, max_wait_minutes: float
+) -> tuple[bool, list[str]]:
+ """Evaluate the gate. Returns ``(ok, log_lines)``."""
+ pending, failed, lines = wait_for_runs(token, repo, sha, max_wait_minutes)
if pending:
- names = ", ".join(run["name"] for run in pending)
- lines.append(f"Waiting for: {names}. Re-evaluated when they complete.")
+ lines.append(
+ f"Gave up after {max_wait_minutes:g} minutes. Re-run this job once
"
+ "the remaining workflows have completed."
+ )
return False, lines
if not failed:
@@ -275,7 +312,18 @@ def gate(token: str, repo: str, sha: str, dry_run: bool)
-> tuple[bool, list[str
"DRY_RUN set, would disable it and comment."
)
continue
- disable_auto_merge(token, node_id)
+ try:
+ disable_auto_merge(token, node_id)
+ except GitHubError as exc:
+ # GITHUB_TOKEN is read-only for PRs from forks. Failing the gate
+ # keeps auto-merge from merging; a committer has to disable
+ # auto-merge (or fix CI and re-run this job).
+ lines.append(
+ f"PR #{number}: auto-merge is enabled by @{enabled_by} but
could "
+ f"not be disabled ({exc}). Failing the gate instead; please "
+ "disable auto-merge manually, or fix CI and re-run this job."
+ )
+ return False, lines
comment(token, repo, number, failure_comment(sha, failed, enabled_by))
lines.append(
f"PR #{number}: auto-merge (enabled by @{enabled_by}) disabled; "
@@ -310,16 +358,15 @@ def main(argv: list[str] | None = None) -> int:
"GITHUB_TOKEN, --repo/$GITHUB_REPOSITORY and --sha/$HEAD_SHA are
required"
)
dry_run = os.environ.get("DRY_RUN", "") not in {"", "0", "false"}
+ max_wait_minutes = float(os.environ.get("MAX_WAIT_MINUTES", "55"))
try:
- ok, lines = gate(token, args.repo, args.sha, dry_run)
+ ok, lines = gate(token, args.repo, args.sha, dry_run, max_wait_minutes)
except GitHubError as exc:
print(f"::error::{exc}")
return 1
- verdict = (
- "PASS" if ok else "WAITING: not all pull_request workflows have
completed."
- )
+ verdict = "PASS" if ok else "FAIL: see above."
lines.append(verdict)
print("\n".join(lines))
@@ -329,7 +376,7 @@ def main(argv: list[str] | None = None) -> int:
summary.write("### Auto-merge gate\n\n")
summary.writelines(f"- {line}\n" for line in lines)
if not ok:
- print(f"::notice::{verdict}")
+ print(f"::error::{lines[-2] if len(lines) > 1 else verdict}")
return 0 if ok else 1
diff --git a/.github/workflows/automerge-gate.yml
b/.github/workflows/automerge-gate.yml
index 9b8ea0d2e..f5a3ee9b4 100644
--- a/.github/workflows/automerge-gate.yml
+++ b/.github/workflows/automerge-gate.yml
@@ -19,12 +19,16 @@
#
# GitHub auto-merge only waits for *required* status checks, and a required
# check blocks manual merges just the same. So the only required CI-related
-# check is the "Auto-merge gate" job below: it is re-run every time one of
-# the pull_request workflows finishes, stays red ("waiting") until all of
-# them have completed, and then
+# check is the "Auto-merge gate" job below. It runs on the PR itself, polls
+# the other pull_request workflows of the same commit until they have all
+# completed (the check is "in progress" meanwhile, so auto-merge cannot fire
+# early), and then
# * passes when everything succeeded (auto-merge proceeds), or
# * disables auto-merge on the PR, comments, and *still passes* when
# something failed, so the committer can merge manually.
+# It has to be a pull_request job: a workflow_run job's check run is attached
+# to the default-branch commit, not to the PR, and never satisfies the
+# required check.
# Urgent PRs do not need this gate: PMC members are bypass actors of the main
# ruleset (.asf.yaml) and can merge from the GitHub UI without waiting for it.
# See .github/scripts/automerge_gate.py for the details.
@@ -32,45 +36,35 @@
name: Auto-merge Gate
on:
- workflow_run:
- # Every workflow that runs on pull_request. A workflow missing from this
- # list still counts for the gate (it lists all pull_request runs of the
- # commit) but will not re-trigger the gate when it finishes, so the gate
- # could stay in "waiting" until another workflow completes.
- workflows:
- - Pre-commit
- - Python Testing
- - CI for Jupyter Notebooks
- - Wheel Build
- - Website Build Check
- - Links
- types: [completed]
+ pull_request:
+ types: [opened, synchronize, reopened, ready_for_review]
permissions:
actions: read
# Both are needed by GITHUB_TOKEN to toggle auto-merge; pull-requests also
- # covers the comment.
+ # covers the comment. (Read-only anyway for PRs from forks; the script then
+ # fails the gate instead of disabling auto-merge.)
contents: write
pull-requests: write
-# Serialise gate runs per commit (no cancellation): the newest run always
-# finishes last, so the status reflects the latest state of CI.
+# A new push supersedes the previous wait.
concurrency:
- group: automerge-gate-${{ github.event.workflow_run.head_sha }}
- cancel-in-progress: false
+ group: automerge-gate-${{ github.event.pull_request.number }}
+ cancel-in-progress: true
jobs:
gate:
# This job name is the status check context referenced in .asf.yaml.
name: Auto-merge gate
- # Only CI of pull requests matters (not pushes to main etc.).
- if: github.event.workflow_run.event == 'pull_request'
runs-on: ubuntu-latest
- timeout-minutes: 10
+ # Must exceed MAX_WAIT_MINUTES below (the longest CI workflow is the
+ # wheel build at roughly 10 minutes; the margin is for runner queueing).
+ timeout-minutes: 60
steps:
- # workflow_run always runs the default-branch workflow; take the script
- # from there as well. No code from the PR head is executed.
+ # The script comes from the PR head (pull_request event). It only reads
+ # workflow runs and toggles auto-merge on the PR's own auto-merge state;
+ # it cannot approve or merge anything.
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
with:
persist-credentials: false
@@ -78,8 +72,10 @@ jobs:
.github/scripts/automerge_gate.py
sparse-checkout-cone-mode: false
- - name: Evaluate CI of the pull request head
+ - name: Wait for the other pull_request workflows and evaluate them
env:
GITHUB_TOKEN: ${{ github.token }}
- HEAD_SHA: ${{ github.event.workflow_run.head_sha }}
+ HEAD_SHA: ${{ github.event.pull_request.head.sha }}
+ GATE_WORKFLOW: ${{ github.workflow }}
+ MAX_WAIT_MINUTES: "55"
run: python3 .github/scripts/automerge_gate.py
diff --git a/docs/community/pr-policy-and-review-guidelines.md
b/docs/community/pr-policy-and-review-guidelines.md
index 17260c9ad..fc8ab65f4 100644
--- a/docs/community/pr-policy-and-review-guidelines.md
+++ b/docs/community/pr-policy-and-review-guidelines.md
@@ -83,16 +83,22 @@ GitHub auto-merge only waits for *required* status checks,
and the CI jobs are
deliberately **not** required on `main`: a required check would also stop a
committer from merging manually when a failure is judged acceptable. Instead
the `Auto-merge gate` status check
([`automerge-gate.yml`](https://github.com/apache/mahout/blob/main/.github/workflows/automerge-gate.yml))
-is required. It is re-evaluated every time a CI workflow of the PR finishes:
+is required. It runs on every push to the PR and waits for the other CI
+workflows of that commit:
-- While any CI workflow is still running, the gate is red ("waiting"), so
- auto-merge cannot fire early.
+- While any CI workflow is still running, the gate is "in progress", so
+ auto-merge cannot fire early. (It gives up after about an hour; re-run the
+ job in that case.)
- When every CI workflow passed (or was skipped), the gate passes and
auto-merge, if enabled, merges the PR.
- When a CI workflow failed, the gate **disables auto-merge** on the PR, leaves
a comment, and then passes. The PR is not merged automatically, but a
committer can still merge it manually. Auto-merge has to be enabled again by
- hand after a fix.
+ hand after a fix. For pull requests from forks the gate cannot disable
+ auto-merge (read-only token) and fails instead; a committer then disables
+ auto-merge by hand or fixes CI and re-runs the job.
+- After fixing a failed CI job, re-run the failed job and then re-run the
+ gate (or push a new commit).
- **Urgent changes (PMC only):** PMC members are bypass actors of the `main`
branch ruleset in
[`.asf.yaml`](https://github.com/apache/mahout/blob/main/.asf.yaml).
On any pull request they can choose "Merge without waiting for requirements
