This is an automated email from the ASF dual-hosted git repository.

potiuk pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git


The following commit(s) were added to refs/heads/main by this push:
     new febd7b45a09 Harden fork CI reconciler state transitions and closure 
rules (#76956)
febd7b45a09 is described below

commit febd7b45a0979a11b8a37d761ac8da6c520a09c5
Author: Ayush Meel <[email protected]>
AuthorDate: Sun Nov 15 20:29:53 2026 +0530

    Harden fork CI reconciler state transitions and closure rules (#76956)
    
    Include PullRequestReview and discussion comments in activity tracking so 
diff comments and reviews count as human activity, reset the inactivity clock 
when a new commit SHA is pushed, and verify head repository availability via 
the REST API before reporting deleted forks as unknown. Case-fold bot logins so 
triage bots do not prevent inactive pull requests from closing, sort 
workflow_dispatch check suites by run attempt, separate timeline control events 
from comments/reviews in the di [...]
---
 .../doc/ci/08_fork_ci_for_external_contributors.md |  33 +--
 .../src/airflow_breeze/utils/fork_ci_messages.py   |   9 +-
 .../src/airflow_breeze/utils/fork_ci_reconcile.py  | 136 ++++++++--
 dev/breeze/tests/test_fork_ci_reconcile.py         | 288 ++++++++++++++++++++-
 4 files changed, 423 insertions(+), 43 deletions(-)

diff --git a/dev/breeze/doc/ci/08_fork_ci_for_external_contributors.md 
b/dev/breeze/doc/ci/08_fork_ci_for_external_contributors.md
index d74f2e6a751..cfee5eae88a 100644
--- a/dev/breeze/doc/ci/08_fork_ci_for_external_contributors.md
+++ b/dev/breeze/doc/ci/08_fork_ci_for_external_contributors.md
@@ -575,7 +575,7 @@ one of their other open pull requests first.
 | released | Usual review with project CI | undraft, except the author's draft 
| unchanged | remove |
 | ungated | Usual review with project CI | undraft, except the author's draft 
| unchanged | remove |
 | closed | Author sets up fork CI and reopens | unchanged | remove | remove |
-| gate_modified | Maintainer reopens and adds `use project ci` | unchanged | 
remove | remove |
+| gate_modified | Author restores the gate and reopens, or maintainer adds 
`use project ci` | unchanged | remove | remove |
 | gate_bypassed | Maintainer reopens and adds `use project ci` | unchanged | 
remove | remove |
 
 The reconciler never undrafts the author's draft. It does not draft a closed 
pull request either,
@@ -589,7 +589,7 @@ since the draft-state job runs after the close and GitHub 
refuses to draft a clo
 | **failed**: conclusion is failure, cancelled, timed out or startup failure | 
`failure` (red), link to the run; for a startup failure the description says 
the run could not start | update with the run link and the jobs that did not 
pass; for a startup failure the text says no job ran and points to the fork's 
Actions settings |
 | **green**: conclusion success and the pull request does not change 
CI-defining files | `success` (green), link to the run | update: ready for 
review, run link; on the author's draft: mark it ready for review when it is |
 | **green_unverified**: conclusion success, but the pull request changes 
CI-defining files (see "Trusting a green fork run") | `pending` (yellow), link 
to the run | update: a maintainer must run project CI; on the author's draft: 
mark it ready for review, then a maintainer runs project CI |
-| **closed**: setup_required or awaiting_push for `close_after_days` | 
unchanged | close with a new comment; reopening restarts the clock |
+| **closed**: setup_required or awaiting_push for `close_after_days` without 
human comments | unchanged | close with a new comment; reopening restarts the 
clock |
 | **released**: the pull request carries `use project ci` | `success` (green), 
link to the pull request's checks | update: project CI is in use; `fork ci 
required` is removed |
 | **ungated**: the base branch has no gate, for example after retargeting to 
an older release branch | `success` (green), link to the pull request's checks 
| update: project CI is in use; `fork ci required` is removed |
 | **gate_modified**: the pull request changes the fork CI gate (see "Gate 
protection") | `error` (red), link to the pull request's changed files | post a 
**new** comment explaining why, then close; checked after gate tampering, 
before the fork state |
@@ -617,12 +617,16 @@ Rules that sit under the table:
   are touched, and adding `use project ci` takes a pull request out of the 
state machine on the
   next run (see "Override labels"). Removing `fork ci required` by hand is not 
enough: it updates
   the pull request, so discovery finds it again and labels it back while it is 
still gated.
-- **Deleted head repository** is reported as unknown (see below) and otherwise 
left alone.
+- **Deleted head repository** is verified via the GitHub REST API to rule out 
transient API
+  errors. If confirmed unavailable, it is reported as unknown (see below) and 
otherwise left alone.
 - **The close clock** starts at the `since` timestamp recorded for the current 
state and
-  resets on any state change. Closing for inactivity records `closed` in the 
marker, so the
-  first run after a reopen sees a state change from `closed` and starts a 
fresh clock rather
-  than closing the pull request again at once. Only a reopen recorded in the 
timeline after
-  the marker counts: an open pull request with a `closed` marker and no such 
reopen is one whose
+  resets on any state change or new commit push. Before closing after 
`close_after_days` (in
+  `setup_required` or `awaiting_push`), the reconciler checks for recent human 
activity
+  (including issue comments, reviews, inline diff comments, and suggestions); 
closure is deferred
+  as long as human discussion has occurred within `close_after_days`. Closing 
for inactivity records
+  `closed` in the marker, so the first run after a reopen sees a state change 
from `closed` and starts
+  a fresh clock rather than closing the pull request again at once. Only a 
reopen recorded in the timeline
+  after the marker counts: an open pull request with a `closed` marker and no 
such reopen is one whose
   closing comment or close failed, and the next run finishes the closure. The 
same holds for a
   `gate_bypassed` marker, where only a reopen by someone other than the author 
counts, so the
   closure is finished even after the offending run has left the tampering 
check's window.
@@ -815,10 +819,9 @@ released to project CI.
 
 The state is `gate_modified`: the pull request loses `ready for maintainer 
review`
 and `needs project ci`, gets an `error` status linking its changed files, and 
the closure is
-reported to Slack as an attention item. Gate tampering is checked first and 
wins. As for gate
-tampering, only a reopen by someone other than the author counts: an open pull 
request whose
-marker says `gate_modified` and that was not reopened since is closed again, 
and a maintainer's
-reopen without restoring the gate closes it again on the next run. A 
maintainer who wants the
+reported to Slack as an attention item. Gate tampering is checked first and 
wins. When closed for `gate_modified`,
+the author can revert or remove the gate change and reopen the pull request; 
if reopened with the
+gate change still present, it is closed again on the next run. A maintainer 
who wants the
 change adds `use project ci`, which releases the pull request before any of 
these checks. Each
 of these files carries a comment saying so.
 
@@ -832,9 +835,9 @@ an HTML marker the reconciler parses:
 ```
 
 followed by the human text for the current state. Editing does not notify, 
which is the
-point: state churn is silent. The three moments that need the author's 
attention, the nudge
-after enabling workflows, the close after inactivity and the close for gate 
tampering, are
-new comments.
+point: state churn is silent. The moments that need the author's attention, 
such as the nudge
+after enabling workflows, the close after inactivity and the close for
+gate tampering, are new comments.
 
 All comment texts live in 
`dev/breeze/src/airflow_breeze/utils/fork_ci_messages.py` as
 templates so wording can be reviewed in one place. Every comment ends with the 
AI-attribution
@@ -1213,7 +1216,7 @@ Unknown means any of:
   conclusion outside the modelled set.
 - A workflow state outside `active` / `disabled_fork` / `disabled_manually` / 
`disabled_inactivity` / not found.
 - A mutation that returned an error (label, comment, close, status).
-- A head repository that no longer exists.
+- A head repository that no longer exists (after confirming via the REST API 
to rule out transient API errors).
 - A state comment whose marker cannot be parsed.
 
 Expected states never alert. The message lists the pull request numbers and 
the reason for
diff --git a/dev/breeze/src/airflow_breeze/utils/fork_ci_messages.py 
b/dev/breeze/src/airflow_breeze/utils/fork_ci_messages.py
index 9d852c7a9fc..7378ad7f368 100644
--- a/dev/breeze/src/airflow_breeze/utils/fork_ci_messages.py
+++ b/dev/breeze/src/airflow_breeze/utils/fork_ci_messages.py
@@ -353,7 +353,7 @@ def is_fork_ci_passed_description(description: str | None) 
-> bool:
 def state_comment(
     marker: Marker,
     *,
-    fork: str,
+    fork: str | None = None,
     run_url: str | None,
     base_ref: str = "main",
     author_draft: bool = False,
@@ -374,7 +374,7 @@ def state_comment(
         parts.append(f"Latest run: {run_url}")
     if marker.state == "failed" and failed_jobs:
         parts.append(build_failed_job_list(failed_jobs))
-    if marker.state == "setup_required":
+    if marker.state == "setup_required" and fork:
         parts.append(_setup_instructions(fork))
     parts.append(f"How this works: [CI in your fork]({CONTRIBUTING_URL}).")
     return "\n\n".join(parts) + FOOTER
@@ -398,8 +398,9 @@ def gate_modified_close_comment(*, change: str | None) -> 
str:
         "monitor, and a pull request whose CI runs in a fork must not change 
it, because the change would "
         "decide where its own CI runs. The one exception is adding, changing 
or removing your own `[pilot]` "
         "entry in `.github/fork-ci-gate.toml`.\n\n"
-        "If the change is wanted, a maintainer can reopen the pull request and 
add the `use project ci` "
-        "label." + FOOTER
+        "If the change was accidental, you can revert or remove the change to 
the gate files and reopen "
+        "the pull request. If the change is wanted, a maintainer can reopen 
the pull request and add the "
+        "`use project ci` label." + FOOTER
     )
 
 
diff --git a/dev/breeze/src/airflow_breeze/utils/fork_ci_reconcile.py 
b/dev/breeze/src/airflow_breeze/utils/fork_ci_reconcile.py
index ed01ce7ad98..61a30366219 100644
--- a/dev/breeze/src/airflow_breeze/utils/fork_ci_reconcile.py
+++ b/dev/breeze/src/airflow_breeze/utils/fork_ci_reconcile.py
@@ -130,7 +130,19 @@ GATED_RUN_JOB_NAMES = 
frozenset[str]().union(*GATED_RUN_JOBS)
 CHECK_RUN_NEEDED_JOB_NAME = "Check if the run is needed"
 PROJECT_CI_NOT_RUN_JOB_NAME = "CI passed (project CI not run)"
 CI_PASSED_CANCELLED_JOB_NAME = "CI passed (run cancelled)"
-BOT_LOGINS = frozenset({"github-actions", "github-actions[bot]"})
+BOT_LOGINS = frozenset(
+    {
+        "github-actions",
+        "github-actions[bot]",
+        "airflow-triage-bot",
+        "codecov",
+        "codecov[bot]",
+        "dependabot",
+        "dependabot[bot]",
+        "asf-ci[bot]",
+        "asfbot",
+    }
+)
 DISCOVERY_WINDOW = timedelta(hours=24)
 PASS_TWO_PAGE_SIZE = 25
 FORK_BATCH_SIZE = 50
@@ -303,6 +315,7 @@ class PullRequestSnapshot:
     last_reopened_at: datetime | None = None
     last_reopened_by_anyone_at: datetime | None = None
     last_base_change_at: datetime | None = None
+    last_comment_at: datetime | None = None
     # (changed at, previous base, new base) for each retarget, oldest first.
     base_changes: tuple[tuple[datetime, str, str], ...] = ()
     current_status: tuple[str, str, str] | None = None
@@ -436,7 +449,10 @@ def _workflow_file(resource_path: str | None) -> str:
 
 
 def _is_bot(login: str | None) -> bool:
-    return login in BOT_LOGINS
+    if not login:
+        return False
+    normalized = login.casefold()
+    return normalized in BOT_LOGINS or normalized.endswith("[bot]")
 
 
 def observe_fork_state(fork: ForkObservation, marker: Marker | None) -> 
tuple[ForkState | None, str | None]:
@@ -649,6 +665,7 @@ def classify(
     head_shared_across_bases: bool = False,
     find_gate_change: Callable[[], str | None] = lambda: None,
     find_failed_jobs: Callable[[], list[FailedJob] | None] = lambda: [],
+    verify_head_repo_unavailable: Callable[[], bool] = lambda: False,
 ) -> Plan:
     """
     Decide the mutations for one gated pull request.
@@ -710,6 +727,9 @@ def classify(
         return release(ForkState.UNGATED, functools.partial(ungated_comment, 
base_ref=pr.base_ref))
 
     if pr.head_repo is None:
+        if not verify_head_repo_unavailable():
+            plan.unknown.append("head repository is unavailable due to GitHub 
API error; retrying")
+            return plan
         plan.unknown.append("head repository no longer exists")
         return plan
 
@@ -739,7 +759,10 @@ def classify(
     reopened_since_closing = {
         ForkState.CLOSED.value: pr.last_reopened_by_anyone_at,
         ForkState.GATE_BYPASSED.value: pr.last_reopened_at,
-        ForkState.GATE_MODIFIED.value: pr.last_reopened_at,
+        ForkState.GATE_MODIFIED.value: max(
+            (t for t in (pr.last_reopened_by_anyone_at, pr.last_reopened_at) 
if t is not None),
+            default=None,
+        ),
     }
     if marker and marker.state in reopened_since_closing:
         reopened = reopened_since_closing[marker.state]
@@ -841,7 +864,7 @@ def classify(
                     "so `CI passed` waits for the next run"
                 )
     plan.state = state
-    since = marker.since if marker and marker.state == state.value else now
+    since = marker.since if marker and marker.state == state.value and 
marker.sha == pr.head_sha else now
 
     if marker and marker.state == ForkState.SETUP_REQUIRED.value and state == 
ForkState.AWAITING_PUSH:
         plan.new_comments.append(nudge_comment(fork=pr.head_repo))
@@ -856,7 +879,14 @@ def classify(
             workflow_disabled_comment(fork=pr.head_repo, 
workflow_state=fork.workflow_state)
         )
 
-    if state in CLOSABLE_STATES and now - since >= 
timedelta(days=config.close_after_days):
+    inactive_for_closing = pr.last_comment_at is None or now - 
pr.last_comment_at >= timedelta(
+        days=config.close_after_days
+    )
+    if (
+        state in CLOSABLE_STATES
+        and now - since >= timedelta(days=config.close_after_days)
+        and inactive_for_closing
+    ):
         _enter_state(plan, pr, labels, ForkState.CLOSED)
         _set_marker_comment(
             plan,
@@ -867,7 +897,8 @@ def classify(
                 run_url=None,
             ),
         )
-        
plan.new_comments.append(inactivity_close_comment(days=config.close_after_days, 
fork=pr.head_repo))
+        comment = inactivity_close_comment(days=config.close_after_days, 
fork=pr.head_repo or "")
+        plan.new_comments.append(comment)
         plan.close = True
         return plan
 
@@ -965,6 +996,18 @@ fragment PassTwoFields on PullRequest {
           ... on BaseRefChangedEvent { createdAt previousRefName 
currentRefName }
         }
       }
+      latestComments: comments(last: 10) {
+        nodes { createdAt author { login } }
+      }
+      latestReviews: reviews(last: 10) {
+        nodes {
+          createdAt
+          author { login }
+          comments(last: 10) {
+            nodes { createdAt author { login } }
+          }
+        }
+      }
       commits(last: 1) { nodes { commit {
         oid
         status {
@@ -975,7 +1018,7 @@ fragment PassTwoFields on PullRequest {
       } } }
       comments(first: 100) {
         pageInfo { hasNextPage endCursor }
-        nodes { databaseId body author { login } }
+        nodes { databaseId body createdAt author { login } }
       }
 }
 """
@@ -1044,15 +1087,17 @@ def parse_pull_request(node: dict[str, Any]) -> 
PullRequestSnapshot:
     last_reopened = None
     last_reopened_by_anyone = None
     last_base_change = None
+    last_comment_at = None
     base_changes = []
     for item in (node.get("timelineItems") or {}).get("nodes") or []:
         if item["__typename"] == "ConvertToDraftEvent":
             last_draft_actor = _login(item.get("actor"))
         elif item["__typename"] == "BaseRefChangedEvent":
             last_base_change = parse_github_datetime(item["createdAt"])
-            base_changes.append(
-                (last_base_change, item.get("previousRefName") or "", 
item.get("currentRefName") or "")
-            )
+            if last_base_change is not None:
+                base_changes.append(
+                    (last_base_change, item.get("previousRefName") or "", 
item.get("currentRefName") or "")
+                )
         elif item["__typename"] == "ReopenedEvent":
             last_reopened_by_anyone = parse_github_datetime(item["createdAt"])
             # Only a reopen by someone else counts for the gate tampering 
check: an author closing
@@ -1064,6 +1109,23 @@ def parse_pull_request(node: dict[str, Any]) -> 
PullRequestSnapshot:
             if label == LABEL_USE_PROJECT_CI:
                 last_unlabel = parse_github_datetime(item["createdAt"])
 
+    for c in (node.get("latestComments") or {}).get("nodes") or []:
+        if not _is_bot(_login(c.get("author"))) and c.get("createdAt"):
+            t = parse_github_datetime(c["createdAt"])
+            if t is not None:
+                last_comment_at = max(filter(None, (last_comment_at, t)), 
default=t)
+
+    for r in (node.get("latestReviews") or {}).get("nodes") or []:
+        if not _is_bot(_login(r.get("author"))) and r.get("createdAt"):
+            t = parse_github_datetime(r["createdAt"])
+            if t is not None:
+                last_comment_at = max(filter(None, (last_comment_at, t)), 
default=t)
+        for rc in (r.get("comments") or {}).get("nodes") or []:
+            if not _is_bot(_login(rc.get("author"))) and rc.get("createdAt"):
+                t = parse_github_datetime(rc["createdAt"])
+                if t is not None:
+                    last_comment_at = max(filter(None, (last_comment_at, t)), 
default=t)
+
     current_status = None
     status_created_at = None
     ci_passed_state = None
@@ -1088,6 +1150,10 @@ def parse_pull_request(node: dict[str, Any]) -> 
PullRequestSnapshot:
             status_created_at = parse_github_datetime(context.get("createdAt") 
or "")
 
     comments = node.get("comments") or {}
+    for c in comments.get("nodes") or []:
+        if not _is_bot(_login(c.get("author"))) and c.get("createdAt"):
+            t = parse_github_datetime(c["createdAt"])
+            last_comment_at = max(filter(None, (last_comment_at, t)), 
default=t)
     found = find_marker_comment(comments.get("nodes") or [])
     page_info = comments.get("pageInfo") or {}
     comments_cursor = page_info.get("endCursor") if found is None and 
page_info.get("hasNextPage") else None
@@ -1113,6 +1179,7 @@ def parse_pull_request(node: dict[str, Any]) -> 
PullRequestSnapshot:
         last_reopened_at=last_reopened,
         last_reopened_by_anyone_at=last_reopened_by_anyone,
         last_base_change_at=last_base_change,
+        last_comment_at=last_comment_at,
         base_changes=tuple(base_changes),
         current_status=current_status,
         status_created_at=status_created_at,
@@ -1754,13 +1821,19 @@ def fetch_fork_observations(
             suites = [
                 suite
                 for suite in (commit.get("checkSuites") or {}).get("nodes") or 
[]
-                if (run := suite.get("workflowRun") or {}).get("event") == 
"push"
+                if (run := suite.get("workflowRun") or {}).get("event") in 
("push", "workflow_dispatch")
                 and _workflow_file((run.get("workflow") or 
{}).get("resourcePath")) == CI_WORKFLOW_FILE
             ]
             if not suites:
                 observations[pr.number] = ForkObservation()
                 continue
-            latest = max(suites, key=lambda suite: suite["createdAt"])
+            latest = max(
+                suites,
+                key=lambda suite: (
+                    suite["createdAt"],
+                    (suite.get("workflowRun") or {}).get("runAttempt") or 1,
+                ),
+            )
             observations[pr.number] = ForkObservation(
                 suite_status=latest["status"],
                 suite_conclusion=latest["conclusion"],
@@ -1998,7 +2071,7 @@ def load_workflow_triggers(workflows_dir: Path) -> 
dict[str, frozenset[str]]:
     """Map each workflow file name of a checkout to the events that trigger 
it."""
     triggers = {}
     for path in sorted(workflows_dir.glob("*.y*ml")):
-        workflow = yaml.safe_load(path.read_text()) or {}
+        workflow = yaml.safe_load(path.read_text(encoding="utf-8")) or {}
         triggers[path.name] = get_workflow_triggers(workflow if 
isinstance(workflow, dict) else {})
     return triggers
 
@@ -2148,12 +2221,18 @@ class BypassFinder:
             paginate=True,
         )
         window_start = self._now - TAMPER_WINDOW
-        return [
-            pull["number"]
-            for pull in pulls
-            if pull["number"] != pr.number
-            and (pull.get("closed_at") is None or 
parse_github_datetime(pull["closed_at"]) >= window_start)
-        ]
+        active_pulls = []
+        for pull in pulls:
+            if pull["number"] == pr.number:
+                continue
+            closed_at_str = pull.get("closed_at")
+            if closed_at_str is None:
+                active_pulls.append(pull["number"])
+            else:
+                closed_at = parse_github_datetime(closed_at_str)
+                if closed_at is not None and closed_at >= window_start:
+                    active_pulls.append(pull["number"])
+        return active_pulls
 
     def find(self, pr: PullRequestSnapshot) -> str | None:
         if not pr.head_repo:
@@ -2404,7 +2483,7 @@ query($owner: String!, $name: String!, $number: Int!, 
$cursor: String) {
   repository(owner: $owner, name: $name) { pullRequest(number: $number) {
     comments(first: 100, after: $cursor) {
       pageInfo { hasNextPage endCursor }
-      nodes { databaseId body author { login } }
+      nodes { databaseId body createdAt author { login } }
     }
   } }
 }
@@ -2419,6 +2498,10 @@ def find_state_comment(client: GitHubClient, repo: str, 
pr: PullRequestSnapshot)
         comments = client.graphql(
             COMMENTS_QUERY, {"owner": owner, "name": name, "number": 
pr.number, "cursor": cursor}
         )["repository"]["pullRequest"]["comments"]
+        for c in comments.get("nodes") or []:
+            if not _is_bot(_login(c.get("author"))) and c.get("createdAt"):
+                t = parse_github_datetime(c["createdAt"])
+                pr = replace(pr, last_comment_at=max(filter(None, 
(pr.last_comment_at, t)), default=t))
         found = find_marker_comment(comments["nodes"])
         if found is not None:
             marker, comment_id, body, error = found
@@ -2509,6 +2592,16 @@ def compare_with_base(client: GitHubClient, repo: str, 
pr: PullRequestSnapshot)
     return BaseCheck(merge_base, behind, tuple(sorted(overlap)))
 
 
+def check_head_repo_unavailable(client: GitHubClient, repo: str, pr_number: 
int) -> bool:
+    try:
+        pr_data = client.rest_get(f"repos/{repo}/pulls/{pr_number}")
+        if not isinstance(pr_data, dict):
+            return False
+        return (pr_data.get("head") or {}).get("repo") is None
+    except Exception:
+        return False
+
+
 def find_heads_shared_across_bases(prs: Iterable[PullRequestSnapshot]) -> 
set[str]:
     """Head commits of more than one pull request whose bases differ."""
     bases_by_head: dict[str, set[str]] = {}
@@ -2868,6 +2961,9 @@ def run_reconcile(
                 head_shared_across_bases=pr.head_sha in shared_heads,
                 find_gate_change=functools.partial(gate_change_finder.find, 
pr),
                 find_failed_jobs=functools.partial(read_failed_jobs, client, 
pr, fork, read_jobs),
+                verify_head_repo_unavailable=functools.partial(
+                    check_head_repo_unavailable, client, repo, pr.number
+                ),
             )
             # A revocation to `pending` always goes through: it is idempotent 
and must not depend on
             # `report_project_ci_passed` succeeding later in this run.
diff --git a/dev/breeze/tests/test_fork_ci_reconcile.py 
b/dev/breeze/tests/test_fork_ci_reconcile.py
index fa1d432977e..4e2ddd2bf90 100644
--- a/dev/breeze/tests/test_fork_ci_reconcile.py
+++ b/dev/breeze/tests/test_fork_ci_reconcile.py
@@ -48,6 +48,7 @@ from airflow_breeze.utils.fork_ci_messages import (
 )
 from airflow_breeze.utils.fork_ci_reconcile import (
     DEFERRED_RUNS_BEFORE_ALERT,
+    GATE_PROTECTED_PATHS,
     JOB_STATUS_FAILURE_RUNS_BEFORE_ALERT,
     JOB_STATUSES_DEFERRED_NOTE,
     JOB_STATUSES_FAILED_REASON,
@@ -68,10 +69,12 @@ from airflow_breeze.utils.fork_ci_reconcile import (
     Projection,
     PullRequestChangedError,
     PullRequestSnapshot,
+    _is_bot,
     apply_plan,
     build_job_statuses,
     build_slack_payload,
     changes_ci_definition,
+    check_head_repo_unavailable,
     classify,
     compare_with_base,
     compute_projection,
@@ -156,6 +159,7 @@ def _classify(
     base=CURRENT_BASE,
     head_shared_across_bases=False,
     gate_change=None,
+    verify_head_repo_unavailable=lambda: False,
 ):
     find_bypass = mock.Mock(return_value=bypass_run)
     changes = mock.Mock(return_value=changes_ci_files)
@@ -170,6 +174,7 @@ def _classify(
         compare_base=mock.Mock(return_value=base),
         head_shared_across_bases=head_shared_across_bases,
         find_gate_change=lambda: gate_change,
+        verify_head_repo_unavailable=verify_head_repo_unavailable,
     )
     return plan, find_bypass, changes
 
@@ -822,6 +827,31 @@ def test_close_clock(marker, should_close):
         assert plan.state == ForkState.SETUP_REQUIRED
 
 
[email protected](
+    ("marker", "last_comment_at", "should_close"),
+    [
+        pytest.param(_marker("failed", days_ago=7), None, True, 
id="failed-seven-days-no-comments"),
+        pytest.param(
+            _marker("failed", days_ago=7), NOW - timedelta(days=8), True, 
id="failed-seven-days-old-comments"
+        ),
+        pytest.param(
+            _marker("failed", days_ago=7),
+            NOW - timedelta(days=2),
+            False,
+            id="failed-seven-days-recent-comments",
+        ),
+        pytest.param(_marker("failed", days_ago=6.9), None, False, 
id="failed-under-seven-days"),
+    ],
+)
+def test_failing_pull_request_inactivity_clock(marker, last_comment_at, 
should_close):
+    pr = _pr(marker=marker, last_comment_at=last_comment_at)
+    failed_fork = ForkObservation(suite_status="COMPLETED", 
suite_conclusion="FAILURE")
+    plan, _, _ = _classify(pr, failed_fork)
+
+    assert plan.close is False
+    assert plan.state == ForkState.FAILED
+
+
 @pytest.mark.parametrize(
     "reopened_at",
     [pytest.param(None, id="never-reopened"), pytest.param(NOW - 
timedelta(days=31), id="reopened-before")],
@@ -959,12 +989,46 @@ def test_both_override_labels_are_reported():
     assert len(plan.attention) == 1
 
 
-def test_deleted_head_repository_is_unknown():
-    plan, _, _ = _classify(_pr(head_repo=None), GREEN)
+def test_deleted_head_repository_reported_as_unknown():
+    plan, _, _ = _classify(_pr(head_repo=None), GREEN, 
verify_head_repo_unavailable=lambda: True)
 
+    assert plan.state is None
+    assert plan.close is False
     assert plan.unknown == ["head repository no longer exists"]
-    assert plan.status is None
-    assert plan.draft is None
+    assert plan.new_comments == []
+
+
+def test_deleted_head_repository_default_retries_safely():
+    plan, _, _ = _classify(_pr(head_repo=None), GREEN)
+
+    assert plan.state is None
+    assert plan.close is False
+    assert plan.unknown == ["head repository is unavailable due to GitHub API 
error; retrying"]
+    assert plan.new_comments == []
+
+
+def test_deleted_head_repository_retries_on_api_error():
+    plan, _, _ = _classify(_pr(head_repo=None), GREEN, 
verify_head_repo_unavailable=lambda: False)
+
+    assert plan.state is None
+    assert plan.close is False
+    assert plan.unknown == ["head repository is unavailable due to GitHub API 
error; retrying"]
+    assert plan.new_comments == []
+
+
+def test_check_head_repo_unavailable():
+    client = mock.Mock()
+    client.rest_get.return_value = {"head": {"repo": None}}
+    assert check_head_repo_unavailable(client, "apache/airflow", 123) is True
+
+    client.rest_get.return_value = {"head": {"repo": {"full_name": 
"user/repo"}}}
+    assert check_head_repo_unavailable(client, "apache/airflow", 123) is False
+
+    client.rest_get.return_value = None
+    assert check_head_repo_unavailable(client, "apache/airflow", 123) is False
+
+    client.rest_get.side_effect = GitHubError("500 Internal Server Error")
+    assert check_head_repo_unavailable(client, "apache/airflow", 123) is False
 
 
 def test_malformed_marker_is_reported_and_rewritten():
@@ -2362,6 +2426,44 @@ def 
test_fork_observations_use_the_fork_name_and_latest_suite(client):
     assert client.graphql.call_args.args[1] == {"o0": "someone", "n0": 
"my-airflow", "s0": "sha1"}
 
 
+def test_fetch_fork_observations_accepts_workflow_dispatch(client):
+    pr = parse_pull_request(_pr_node(1, head_repo="someone/my-airflow", 
head_sha="sha1"))
+    client.graphql.return_value = {
+        "pr0": {
+            "object": {
+                "checkSuites": {
+                    "nodes": [
+                        {
+                            "createdAt": "2026-11-02T00:00:00Z",
+                            "updatedAt": "2026-11-02T00:40:00Z",
+                            "status": "COMPLETED",
+                            "conclusion": "SUCCESS",
+                            "workflowRun": {
+                                "databaseId": 88,
+                                "runAttempt": 1,
+                                "url": "dispatch-run",
+                                "event": "workflow_dispatch",
+                                "workflow": {"resourcePath": "/x/ci-amd.yml"},
+                            },
+                        },
+                    ]
+                }
+            }
+        }
+    }
+
+    observations = fetch_fork_observations(client, [pr])
+
+    assert observations[1] == ForkObservation(
+        suite_status="COMPLETED",
+        suite_conclusion="SUCCESS",
+        run_url="dispatch-run",
+        run_id=88,
+        run_attempt=1,
+        completed_at=datetime(2026, 11, 2, 0, 40, tzinfo=UTC),
+    )
+
+
 def test_state_comment_is_found_beyond_the_first_page(client):
     marker = _marker("running")
     first_page = {
@@ -3432,6 +3534,34 @@ def test_unfinished_gate_change_closure_is_retried():
     assert "Closing this pull request because it changes the fork CI gate." in 
plan.new_comments[0]
 
 
+def test_author_reopen_allowed_when_gate_change_removed():
+    pr = _pr(
+        marker=_marker("gate_modified", days_ago=1),
+        last_reopened_by_anyone_at=NOW,
+    )
+    plan, _, _ = _classify(pr, GREEN, gate_change=None)
+
+    assert plan.state == ForkState.GREEN
+    assert plan.close is False
+
+
+def test_author_reopen_reclosed_when_gate_change_remains():
+    pr = _pr(
+        marker=_marker("gate_modified", days_ago=1),
+        last_reopened_by_anyone_at=NOW,
+    )
+    plan, _, _ = _classify(
+        pr, GREEN, gate_change="it changes 
`dev/breeze/src/airflow_breeze/utils/fork_ci_gate.py`"
+    )
+
+    assert plan.state == ForkState.GATE_MODIFIED
+    assert plan.close is True
+    assert (
+        "changes the fork CI gate: it changes 
`dev/breeze/src/airflow_breeze/utils/fork_ci_gate.py`"
+        in plan.new_comments[0]
+    )
+
+
 PILOT_BASE = "enabled = true\n[pilot]\nother = 2026-10-01T00:00:00Z\n"
 
 
@@ -3526,6 +3656,12 @@ def _gate_change_finder(client, changed, head_files, 
base_files=None, head_sha="
             "it changes `.github/workflows/fork-ci-monitor.yml`",
             id="monitor-moved-away",
         ),
+        pytest.param(
+            [{"filename": 
"dev/breeze/src/airflow_breeze/utils/fork_ci_reconcile.py"}],
+            {},
+            "it changes 
`dev/breeze/src/airflow_breeze/utils/fork_ci_reconcile.py`",
+            id="reconciler-code-protected",
+        ),
         pytest.param(
             [{"filename": ".github/fork-ci-gate.toml"}],
             {".github/fork-ci-gate.toml": "enabled = false\n"},
@@ -3934,6 +4070,150 @@ def test_mark_older_pilot_pull_request(
     assert (marked.ci_run_at, marked.pilot_joined_at) == (expected_ci_run_at, 
expected_pilot_joined_at)
 
 
+def test_gate_protected_paths_includes_all_five_files():
+    expected = {
+        "dev/breeze/src/airflow_breeze/utils/fork_ci_gate.py",
+        "dev/breeze/src/airflow_breeze/utils/fork_ci_github.py",
+        "dev/breeze/src/airflow_breeze/utils/fork_ci_messages.py",
+        "dev/breeze/src/airflow_breeze/utils/fork_ci_reconcile.py",
+        ".github/workflows/fork-ci-monitor.yml",
+    }
+    assert expected == GATE_PROTECTED_PATHS
+
+
[email protected](
+    ("login", "expected"),
+    [
+        ("github-actions", True),
+        ("github-actions[bot]", True),
+        ("airflow-triage-bot", True),
+        ("Airflow-Triage-Bot", True),
+        ("codecov", True),
+        ("codecov[bot]", True),
+        ("Codecov[Bot]", True),
+        ("dependabot", True),
+        ("dependabot[bot]", True),
+        ("asf-ci[bot]", True),
+        ("asfbot", True),
+        ("ASFBot", True),
+        ("custom-linter[bot]", True),
+        ("ayushmeel", False),
+        ("potiuk", False),
+        ("", False),
+        (None, False),
+    ],
+)
+def test_is_bot(login, expected):
+    assert _is_bot(login) is expected
+
+
+def test_setup_required_inactivity_clock_resets_on_new_commit_sha():
+    pr = _pr(
+        marker=_marker("setup_required", days_ago=7, sha="old_sha"),
+        head_sha="new_sha",
+    )
+    plan, _, _ = _classify(pr, ForkObservation(workflow_state="disabled_fork"))
+
+    assert plan.close is False
+    assert plan.state == ForkState.SETUP_REQUIRED
+
+
+def test_setup_required_inactivity_clock_resets_on_recent_human_comment():
+    pr = _pr(
+        marker=_marker("setup_required", days_ago=7),
+        last_comment_at=NOW - timedelta(days=2),
+    )
+    plan, _, _ = _classify(pr, ForkObservation(workflow_state="disabled_fork"))
+
+    assert plan.close is False
+    assert plan.state == ForkState.SETUP_REQUIRED
+
+
+def test_parse_pull_request_tracks_reviews_and_comments():
+    node = _pr_node(
+        1,
+        latestReviews={
+            "nodes": [
+                {
+                    "createdAt": "2026-11-05T00:00:00Z",
+                    "author": {"login": "reviewer"},
+                    "comments": {
+                        "nodes": [
+                            {
+                                "createdAt": "2026-11-06T12:00:00Z",
+                                "author": {"login": "code-suggester"},
+                            },
+                            {
+                                "createdAt": "2026-11-09T00:00:00Z",
+                                "author": {"login": "codecov[bot]"},
+                            },
+                        ]
+                    },
+                }
+            ]
+        },
+        latestComments={
+            "nodes": [
+                {
+                    "createdAt": "2026-11-07T00:00:00Z",
+                    "author": {"login": "human-author"},
+                },
+                {
+                    "createdAt": "2026-11-08T00:00:00Z",
+                    "author": {"login": "Airflow-Triage-Bot"},
+                },
+            ]
+        },
+    )
+    pr = parse_pull_request(node)
+    assert pr.last_comment_at == datetime(2026, 11, 7, 0, 0, tzinfo=UTC)
+
+
+def test_fetch_fork_observations_orders_by_created_at_and_run_attempt(client):
+    pr = parse_pull_request(_pr_node(1, head_repo="someone/my-airflow", 
head_sha="sha1"))
+    client.graphql.return_value = {
+        "pr0": {
+            "object": {
+                "checkSuites": {
+                    "nodes": [
+                        {
+                            "createdAt": "2026-11-02T00:00:00Z",
+                            "updatedAt": "2026-11-02T00:30:00Z",
+                            "status": "COMPLETED",
+                            "conclusion": "FAILURE",
+                            "workflowRun": {
+                                "databaseId": 88,
+                                "runAttempt": 1,
+                                "url": "attempt-1",
+                                "event": "workflow_dispatch",
+                                "workflow": {"resourcePath": "/x/ci-amd.yml"},
+                            },
+                        },
+                        {
+                            "createdAt": "2026-11-02T00:00:00Z",
+                            "updatedAt": "2026-11-02T00:50:00Z",
+                            "status": "COMPLETED",
+                            "conclusion": "SUCCESS",
+                            "workflowRun": {
+                                "databaseId": 88,
+                                "runAttempt": 2,
+                                "url": "attempt-2",
+                                "event": "workflow_dispatch",
+                                "workflow": {"resourcePath": "/x/ci-amd.yml"},
+                            },
+                        },
+                    ]
+                }
+            }
+        }
+    }
+
+    observations = fetch_fork_observations(client, [pr])
+    assert observations[1].run_attempt == 2
+    assert observations[1].suite_conclusion == "SUCCESS"
+    assert observations[1].run_url == "attempt-2"
+
+
 def 
test_older_pilot_pull_request_stays_checked_for_tampering_after_its_gated_run_is_deleted():
     config = dataclasses.replace(CONFIG, enabled=False, pilot={"someone": 
JOINED})
 

Reply via email to