This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new 1b9a32b5283 Do not fail CI passed for a cancelled duplicate CI run
(#75578)
1b9a32b5283 is described below
commit 1b9a32b528300a545ca71daee3d161da1468347c
Author: Jarek Potiuk <[email protected]>
AuthorDate: Sat Oct 24 23:43:32 2026 +0200
Do not fail CI passed for a cancelled duplicate CI run (#75578)
* Do not fail CI passed for a cancelled duplicate CI run
A run that concurrency cancels still runs the always() CI passed job, and
that job failed the required check. When a stacked pull request's base and
head are force-pushed together, GitHub sends two synchronize events for the
same commit, so one run is cancelled seconds in and a failed CI passed sits
next to the passing one of its twin. A cancelled run now reports under a
name the rulesets do not require, so nothing is red, while a run that was
only cancelled still never satisfies CI passed and the merge stays blocked.
Generated-by: Claude Opus 5
* Accept the cancelled-run CI passed name in the fork CI tampering check
A cancelled ci-amd.yml run now reports its CI passed job as "CI passed (run
cancelled)". Without the name in the gated-run jobs, a run of a gated pull
request that a newer push cancelled would be taken for project CI work and
close the pull request as tampering with the gate.
Generated-by: Claude Opus 5
* Keep failing CI passed when a CI job times out
A job that hits its timeout is cancelled while the run is not, so the
cancelled-run rename alone would turn a hung job into a passing check
and drop it from the scheduled canary's Slack report.
Generated-by: Claude Opus 5
* Make the Slack canary report wait for the CI passed gate job
Generated-by: Claude Opus 5
---
.github/workflows/ci-amd.yml | 34 ++++++++++++++++++----
.github/workflows/ci-arm.yml | 34 ++++++++++++++++++----
dev/README_AIRFLOW3_DEV.md | 10 +++++--
.../src/airflow_breeze/utils/fork_ci_reconcile.py | 9 +++---
dev/breeze/tests/test_fork_ci_reconcile.py | 10 +++++++
5 files changed, 80 insertions(+), 17 deletions(-)
diff --git a/.github/workflows/ci-amd.yml b/.github/workflows/ci-amd.yml
index 2f7fde8dffd..92f1903a1b7 100644
--- a/.github/workflows/ci-amd.yml
+++ b/.github/workflows/ci-amd.yml
@@ -1461,11 +1461,20 @@ jobs:
# pull request to CI in the contributor's fork, and every job here is
skipped. Reporting
# "CI passed" then would let it merge on no CI, so the job reports under
another name and
# the fork CI reconciler posts "CI passed" from the fork run instead.
+ # A run with cancelled jobs and no failed one -- usually a duplicate run
of the same commit that
+ # concurrency cancelled, such as the two `synchronize` events GitHub sends
when a stacked pull
+ # request's base and head are force-pushed together -- reports under
another name too, so it
+ # never puts a failed "CI passed" next to the passing one of its twin. "CI
passed" is then
+ # missing for that run, which keeps the merge blocked until another run of
the commit reports it.
+ # A job cancelled by its timeout gets that name too, since `name` cannot
test whether the run
+ # itself was cancelled, but the job still fails.
name: >-
- ${{ needs.build-info.result == 'skipped' && 'CI passed (project CI not
run)' || 'CI passed' }}
+ ${{ (!contains(needs.*.result, 'failure') && contains(needs.*.result,
'cancelled') &&
+ 'CI passed (run cancelled)') ||
+ (needs.build-info.result == 'skipped' && 'CI passed (project CI not
run)') || 'CI passed' }}
# The one check main's ruleset requires, so auto-merge waits for CI to
finish. GitHub treats a
# skipped required check as passed, so this job always runs and fails
itself when any job it
- # needs failed or was cancelled. Jobs skipped by selective checks count as
passed.
+ # needs failed or timed out. Jobs skipped by selective checks count as
passed.
if: always()
needs:
- additional-ci-image-checks
@@ -1508,12 +1517,25 @@ jobs:
- tests-with-lowest-direct-resolution-providers
runs-on: ["ubuntu-slim"]
steps:
- - name: "Fail if any CI job failed or was cancelled"
- if: contains(needs.*.result, 'failure') || contains(needs.*.result,
'cancelled')
+ - name: "Fail if any CI job failed"
+ if: contains(needs.*.result, 'failure')
run: |
- echo "At least one CI job failed or was cancelled."
+ echo "At least one CI job failed."
exit 1
+ # A job that hits its timeout is cancelled while the run is not, and
must fail this check, as
+ # the scheduled canary's Slack report lists only failed jobs.
+ - name: "Fail if a CI job was cancelled without the run being cancelled"
+ if: contains(needs.*.result, 'cancelled') && !cancelled()
+ run: |
+ echo "A CI job was cancelled, likely by its timeout, while the run
was not cancelled."
+ exit 1
+ - name: "Report that CI jobs were cancelled"
+ if: contains(needs.*.result, 'cancelled') && cancelled()
+ run: |
+ echo "At least one CI job was cancelled, so this run does not report
CI passed."
+ echo 'CI jobs were cancelled: this run does not report `CI passed`.'
>> "${GITHUB_STEP_SUMMARY}"
- name: "All CI jobs passed or were skipped"
+ if: "!contains(needs.*.result, 'cancelled')"
run: echo "All CI jobs passed or were skipped."
notify-slack:
@@ -1521,6 +1543,8 @@ jobs:
needs:
- build-info
- check-run-needed
+ # Waits for the gate job, which is the only failed job when a CI job
timed out.
+ - ci-passed
- finalize-tests
- tests-agent-frameworks
# A no-op canary (`run-needed` is 'false') does not post; a failed check
does. A failing gate
diff --git a/.github/workflows/ci-arm.yml b/.github/workflows/ci-arm.yml
index 5eab899803e..180182647a6 100644
--- a/.github/workflows/ci-arm.yml
+++ b/.github/workflows/ci-arm.yml
@@ -1343,11 +1343,20 @@ jobs:
# pull request to CI in the contributor's fork, and every job here is
skipped. Reporting
# "CI passed" then would let it merge on no CI, so the job reports under
another name and
# the fork CI reconciler posts "CI passed" from the fork run instead.
+ # A run with cancelled jobs and no failed one -- usually a duplicate run
of the same commit that
+ # concurrency cancelled, such as the two `synchronize` events GitHub sends
when a stacked pull
+ # request's base and head are force-pushed together -- reports under
another name too, so it
+ # never puts a failed "CI passed" next to the passing one of its twin. "CI
passed" is then
+ # missing for that run, which keeps the merge blocked until another run of
the commit reports it.
+ # A job cancelled by its timeout gets that name too, since `name` cannot
test whether the run
+ # itself was cancelled, but the job still fails.
name: >-
- ${{ needs.build-info.result == 'skipped' && 'CI passed (project CI not
run)' || 'CI passed' }}
+ ${{ (!contains(needs.*.result, 'failure') && contains(needs.*.result,
'cancelled') &&
+ 'CI passed (run cancelled)') ||
+ (needs.build-info.result == 'skipped' && 'CI passed (project CI not
run)') || 'CI passed' }}
# The one check main's ruleset requires, so auto-merge waits for CI to
finish. GitHub treats a
# skipped required check as passed, so this job always runs and fails
itself when any job it
- # needs failed or was cancelled. Jobs skipped by selective checks count as
passed.
+ # needs failed or timed out. Jobs skipped by selective checks count as
passed.
if: always()
needs:
- additional-ci-image-checks
@@ -1389,12 +1398,25 @@ jobs:
- tests-with-lowest-direct-resolution-providers
runs-on: ["ubuntu-slim"]
steps:
- - name: "Fail if any CI job failed or was cancelled"
- if: contains(needs.*.result, 'failure') || contains(needs.*.result,
'cancelled')
+ - name: "Fail if any CI job failed"
+ if: contains(needs.*.result, 'failure')
run: |
- echo "At least one CI job failed or was cancelled."
+ echo "At least one CI job failed."
exit 1
+ # A job that hits its timeout is cancelled while the run is not, and
must fail this check, as
+ # the scheduled canary's Slack report lists only failed jobs.
+ - name: "Fail if a CI job was cancelled without the run being cancelled"
+ if: contains(needs.*.result, 'cancelled') && !cancelled()
+ run: |
+ echo "A CI job was cancelled, likely by its timeout, while the run
was not cancelled."
+ exit 1
+ - name: "Report that CI jobs were cancelled"
+ if: contains(needs.*.result, 'cancelled') && cancelled()
+ run: |
+ echo "At least one CI job was cancelled, so this run does not report
CI passed."
+ echo 'CI jobs were cancelled: this run does not report `CI passed`.'
>> "${GITHUB_STEP_SUMMARY}"
- name: "All CI jobs passed or were skipped"
+ if: "!contains(needs.*.result, 'cancelled')"
run: echo "All CI jobs passed or were skipped."
notify-slack:
@@ -1402,6 +1424,8 @@ jobs:
needs:
- build-info
- check-run-needed
+ # Waits for the gate job, which is the only failed job when a CI job
timed out.
+ - ci-passed
- finalize-tests
- tests-agent-frameworks
# A no-op canary (`run-needed` is 'false') does not post; a failed check
does. A failing gate
diff --git a/dev/README_AIRFLOW3_DEV.md b/dev/README_AIRFLOW3_DEV.md
index 51216378097..c094c9dc2fb 100644
--- a/dev/README_AIRFLOW3_DEV.md
+++ b/dev/README_AIRFLOW3_DEV.md
@@ -196,9 +196,13 @@ Auto-merge is enabled for the whole repository, so every
branch that PRs target
then, its PRs never report the check, and a committer merges them by bypassing
the rule.
The `main` ruleset requires two checks. **CI passed** is a job in the `Tests
(AMD)` workflow that always
-runs and fails when any other CI job failed or was cancelled; jobs that
selective checks skip count as
-passed, so PRs that run only part of the tests still get a result. The
`check-ci-gate-needs` prek hook
-makes sure every new CI job is added to it. `Tests (ARM)`, which runs on
pushes and on a schedule rather
+runs and fails when any other CI job failed; jobs that selective checks skip
count as passed, so PRs that
+run only part of the tests still get a result. A run with cancelled jobs and
no failed one reports as
+**CI passed (run cancelled)** instead, so a duplicate run of the same commit
that concurrency cancelled
+does not show a failed **CI passed** next to its twin's result. Such a run
does not report **CI passed**
+at all, so the PR stays blocked until another run of the commit, or a re-run,
reports it. A job that
+hits its timeout is cancelled while the run is not, so that run's check fails,
under the same name. The
+`check-ci-gate-needs` prek hook makes sure every new CI job is added to it.
`Tests (ARM)`, which runs on pushes and on a schedule rather
than on PRs, has the same job, kept identical by the
`check-ci-workflows-in-sync` hook. For a PR that the
fork CI gate (AIP-120) leaves to CI in the contributor's fork, project CI does
not run, so the job reports
as **CI passed (project CI not run)** instead, and the fork CI reconciler
posts **CI passed** once the
diff --git a/dev/breeze/src/airflow_breeze/utils/fork_ci_reconcile.py
b/dev/breeze/src/airflow_breeze/utils/fork_ci_reconcile.py
index 35189f1a8f4..3c4eeafb70d 100644
--- a/dev/breeze/src/airflow_breeze/utils/fork_ci_reconcile.py
+++ b/dev/breeze/src/airflow_breeze/utils/fork_ci_reconcile.py
@@ -79,12 +79,13 @@ GATE_JOB_NAME = "Fork CI gate"
# Jobs that also run, without needing the gate, in a run the gate left to fork
CI: main's
# check-run-needed (no steps outside schedule and push runs) and the "CI
passed" gate job of the
# merge ruleset, which reports as "CI passed (project CI not run)" when
project CI was skipped and
-# as "CI passed" when a newer push cancelled the run before build-info
started. Like the gate, each
-# must finish within GATE_MAX_DURATION, so project CI work moved under one of
these names counts.
-# Each runs once, so the names of one job are grouped: repeating short jobs
under them counts too.
+# as "CI passed (run cancelled)" when a newer push cancelled the run before
build-info started --
+# "CI passed" in a workflow from before that name. Like the gate, each must
finish within
+# GATE_MAX_DURATION, so project CI work moved under one of these names counts.
Each runs once, so
+# the names of one job are grouped: repeating short jobs under them counts too.
GATED_RUN_JOBS = (
frozenset({"Check if the run is needed"}),
- frozenset({"CI passed (project CI not run)", "CI passed"}),
+ frozenset({"CI passed (project CI not run)", "CI passed (run cancelled)",
"CI passed"}),
)
GATED_RUN_JOB_NAMES = frozenset[str]().union(*GATED_RUN_JOBS)
BOT_LOGINS = frozenset({"github-actions", "github-actions[bot]"})
diff --git a/dev/breeze/tests/test_fork_ci_reconcile.py
b/dev/breeze/tests/test_fork_ci_reconcile.py
index be4e1dcb15f..3f7d56f7c14 100644
--- a/dev/breeze/tests/test_fork_ci_reconcile.py
+++ b/dev/breeze/tests/test_fork_ci_reconcile.py
@@ -719,6 +719,16 @@ def _job(
False,
id="cancelled-before-the-gate-started",
),
+ pytest.param(
+ [
+ _job("Check if the run is needed", "cancelled", started=False),
+ _job("Fork CI gate", "cancelled", started=False),
+ _job("Build info", "cancelled", started=False),
+ _job("CI passed (run cancelled)", "success"),
+ ],
+ False,
+ id="cancelled-run-reporting-ci-passed-run-cancelled",
+ ),
pytest.param(
[_job("Fork CI gate", "cancelled", minutes=30, started=False)],
False,
