potiuk opened a new pull request, #114:
URL: https://github.com/apache/airflow-steward/pull/114
## Summary
Two related improvements to the `classify-and-act.md` decision logic, both
prompted by misclassifications hit during a triage sweep on `apache/airflow`:
1. **CANCELLED contexts should not count as failures.** A check-run with
`conclusion: "CANCELLED"` is almost always caused by a newer push superseding
the run, GitHub Actions concurrency-cancellation, or the contributor cancelling
manually — none of these signal that the PR's current head SHA is broken.
Cancelled contexts can still pull `statusCheckRollup.state` to `FAILURE`, so
trusting the rollup state alone is wrong.
2. **Rollup `FAILURE` with no extractable failed-check names should route to
row 22 (data anomaly skip), not the row 17 draft fallback.** When the rollup
has rolled to FAILURE but no contributing context is visible (only CANCELLED,
or the failing check-run hasn't propagated yet), the existing logic fell
through `has_deterministic_signal` → `ci_failures_only` (no match because
`failed_checks` empty) → ... → row 17 (draft fallback). The result would be a
draft conversion with an empty violations comment.
## Empirical evidence
During a triage sweep on `apache/airflow` today:
- **#65993** (@shaealh): 2 CANCELLED checks (Breeze unit tests + Breeze
integration tests) + 6 unresolved review threads from a maintainer. Treating
CANCELLED as failure made `ci_only` false, `threads_only` false,
`has_det_signal` true via both legs, and the PR landed in row 17 draft. With
CANCELLED excluded, `threads_only` is true and the PR correctly routes to row
15 ping (the maintainer's ask was "address my review feedback", not "convert to
draft").
- **#65751** (@MdSadiqMd): rollup `FAILURE` with zero failed checks visible.
Existing logic lands in row 17 draft and would have produced a violations
comment with no violations. Should route to row 22 skip — wait for the rollup
to settle.
## Changes
`classify-and-act.md`:
- Add a new precondition glossary entry **`failed_checks`** that defines the
canonical "real failure" set (`CheckRun.conclusion ∈ {FAILURE, TIMED_OUT}` or
`StatusContext.state ∈ {FAILURE, ERROR}`) and explicitly excludes CANCELLED,
NEUTRAL/SKIPPED/STALE/STARTUP_FAILURE/ACTION_REQUIRED, and pending checks.
Notes that implementations may track `cancelled_checks` and `pending_checks`
separately for diagnostics, but only `failed_checks` feeds the decision table.
- Tighten `has_deterministic_signal` to require `failed_checks` non-empty
(in addition to `rollup.state == FAILURE`) for the CI-failure leg. The
non-empty clause naturally routes the rollup-FAILURE-with-empty-`failed_checks`
case to row 22 by failing every other row.
- Broaden Row 22 to cover both directions of rollup-vs-context inconsistency
(rollup SUCCESS + non-empty `failed_checks`, OR rollup FAILURE + empty
`failed_checks`).
- Update the row 22 hard rule to reflect that it now fires before rows 17 +
19-20 (not just 19-20), and to spell out which direction of inconsistency is
enforced naturally vs. needs an explicit pre-row-19 check.
## Test plan
- [x] Re-classified the 80-PR page from today's `apache/airflow` triage
sweep with the tightened logic. #65993 now routes to row 15 ping (correct),
#65751 routes to row 22 skip (correct), no other classifications change.
- [x] `prek run --files` passes (markdownlint, skill-validate, typos, etc.).
- [x] No mechanical references to "failed_checks" elsewhere needed updating
— existing references to "failed check name" in `static_check`,
`recent_main_failures`, etc. naturally feed off the now-defined `failed_checks`
set.
---
##### Was generative AI tooling used to co-author this PR?
- [X] Yes — Claude Code (Opus 4.7)
Generated-by: Claude Code (Opus 4.7) following [the
guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
