potiuk commented on code in PR #73378: URL: https://github.com/apache/airflow/pull/73378#discussion_r4330951476
########## dev/breeze/doc/adr/0020-run-ci-for-external-contributions-in-forks.md: ########## @@ -0,0 +1,167 @@ +<!-- + Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. + --> + +<!-- START doctoc generated TOC please keep comment here to allow auto update --> +<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE --> + +- [20. Run CI for external contributions in contributor forks](#20-run-ci-for-external-contributions-in-contributor-forks) + - [Status](#status) + - [Context](#context) + - [Decision](#decision) + - [Consequences](#consequences) + +<!-- END doctoc generated TOC please keep comment here to allow auto update --> + +# 20. Run CI for external contributions in contributor forks + +Date: 2026-09-19 + +## Status + +Proposed (pending the outcome of the AIP-120 vote; the implementation ships dormant) + +## Context + +[AIP-120](https://cwiki.apache.org/confluence/spaces/AIRFLOW/pages/451974711/AIP-120+Run+CI+for+external+contributions+in+contributor+forks) +proposes that pull requests from contributors who are not committers, collaborators or +members of an exempted organisation get their CI feedback from GitHub Actions running in the +contributor's own fork, and that project CI in ``apache/airflow`` is reserved for the rest. +The motivation is in the AIP: Airflow is routinely the largest consumer of the shared ASF +GitHub Actions pool by job count, roughly a fifth of externally authored pull requests close +without merging after consuming CI and reviewer attention, and the wait maintainers observe on +their own pull requests is queueing in that shared pool, not execution time. GitHub Actions is +free and unmetered on public repositories, so the cost moved to the contributor is elapsed +time on a 20-job concurrency cap, not money. + +Mechanising this touches four places at once: the trigger and selective-check logic of the +main test workflow, an automated lifecycle for gated pull requests, a way for contributors +to check their setup before pushing, and the contribution documentation. The design is in +[Fork CI for external contributors](../ci/08_fork_ci_for_external_contributors.md). This +record captures the decisions that shape it and the alternatives rejected. + +## Decision + +1. **One eligibility function, one configuration file.** ``fork_ci_gate.decide`` in breeze is + a pure, standard-library-only function over the fields both the workflow event and the + GraphQL API expose. Its configuration is ``.github/fork-ci-gate.toml`` in the repository: + an ``enabled`` switch, an ``enabled_since`` cut-off, ``close_after_days``, the + exempted-organisation allowlist and the ``[pilot]`` table of volunteer contributors. A repository file rather than an Actions variable because + ASF projects cannot self-serve variables and exemption changes deserve a reviewed pull + request. The gate reads the file from the base branch so a pull request cannot alter its + own eligibility. Two labels that only people with triage rights can set override the + automatic decision per pull request: ``use project ci`` and ``use fork ci``. They take + effect whether or not the gate is enabled, which makes ``use fork ci`` the pilot switch. + +2. **Project CI is stopped by a gate job, not by cancellation.** ``ci-amd.yml`` gets a + ``gate`` job that executes the eligibility file directly with the runner's ``python3`` and + makes ``build-info`` conditional on it. Cancelling the run is impossible with the read-only + token a fork pull request gets, and failing the run would attach a misleading red status. + Skipping is silent and cheap. + +3. **Fork pushes run CI with pull-request semantics.** The push trigger becomes ``'**'`` and + the previous branch patterns move into the eligibility function so behaviour in + ``apache/airflow`` is unchanged. In a fork, selective checks compute the change set with the + cross-repository compare API against ``apache/airflow`` (merge-base semantics, no fork sync Review Comment: Added. When a head commit turns green, the reconciler counts the commits it is behind its base branch (`behind_by` of the compare, once per head commit, stored in the marker). From 200 commits behind, about four days of `main`, the state comment says so and asks for a rebase. It's advisory and doesn't change the state. Implemented in #74822, described under "Far behind the base branch" in the design doc. --- Drafted-by: Claude Code (Opus 5); reviewed by @potiuk before posting ########## dev/breeze/doc/ci/08_fork_ci_for_external_contributors.md: ########## @@ -0,0 +1,769 @@ +<!-- + Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. + --> + +<!-- START doctoc generated TOC please keep comment here to allow auto update --> +<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE --> + +- [Fork CI for external contributors (AIP-120)](#fork-ci-for-external-contributors-aip-120) + - [Goals and non-goals](#goals-and-non-goals) + - [Overview](#overview) + - [Configuration](#configuration) + - [Eligibility](#eligibility) + - [Project CI gate](#project-ci-gate) + - [Fork mode CI](#fork-mode-ci) + - [Reconciler](#reconciler) + - [Monitor workflow](#monitor-workflow) + - [`breeze ci audit`](#breeze-ci-audit) + - [Documentation changes](#documentation-changes) + - [Rollout and rollback](#rollout-and-rollback) + - [Security considerations](#security-considerations) + - [Known limitations](#known-limitations) + - [Testing](#testing) + +<!-- END doctoc generated TOC please keep comment here to allow auto update --> + +# Fork CI for external contributors (AIP-120) + +This document is the design of the CI changes implementing +[AIP-120: Run CI for external contributions in contributor forks](https://cwiki.apache.org/confluence/spaces/AIRFLOW/pages/451974711/AIP-120+Run+CI+for+external+contributions+in+contributor+forks). +The policy itself and its rationale live in the AIP and in the dev list discussions +([first](https://lists.apache.org/thread/w74655o8y1jzrzcnygntyyovocf7zmzj), +[second](https://lists.apache.org/thread/ky392mrrcmj312y1r276r30p584n804c), which agreed to +start with an opt-in pilot). +This document describes how the policy is mechanised. + +## Goals and non-goals + +Goals: + +- External contributors get CI feedback from a plain `git push` to their fork. No pull request + against their own fork, no manual workflow dispatch, no sorting through workflow files. +- Project CI (`apache/airflow` runners) does not run for gated pull requests. The decision is + made by one function with one configuration file. +- A single scheduled reconciler owns the lifecycle of gated pull requests: labelling, drafting, + undrafting, the state comment, the nudge after workflows are enabled, closing after + inactivity, and closing a pull request that tampered with the gate to get project CI. It is + idempotent, so missed or duplicated scheduled runs are harmless. +- Maintainers and contributors see the fork CI state where they already look: a commit status + on the pull request head, red / yellow / green, linking to the run in the fork. +- A contributor can verify their whole setup locally with `breeze ci audit` before pushing. +- Everything ships dormant and is switched on by a one-line configuration change. +- Before it is switched on for everyone, the whole flow can be piloted on real traffic by + committers on single pull requests and by volunteer external contributors on all their new + pull requests, without affecting anyone else. + +Non-goals: + +- The one-time reset of the open pull request backlog (AIP-120 §9.1). The design makes it a + configuration change, but it is decided separately. +- Any change to how committer, collaborator or exempted-organisation pull requests are tested. +- Event-driven mutation of pull requests. Nothing in this design uses `pull_request_target`. + All writes come from the scheduled reconciler. + +## Overview + +``` + contributor fork apache/airflow + ┌──────────────────────────────────┐ ┌──────────────────────────────────────────┐ + │ git push feature-branch │ │ pull_request event │ + │ └─ ci-amd.yml (push trigger) │ │ └─ ci-amd.yml │ + │ gate job: fork -> run │ │ gate job: eligibility -> skip │ + │ build-info: fork mode │ │ (build-info and everything after │ + │ changed files from │ │ are skipped for gated PRs) │ + │ compare API vs upstream │ │ │ + │ public runners only │ │ fork-ci-monitor.yml (every 15 minutes) │ + └──────────────────────────────────┘ │ └─ breeze ci fork-ci-reconcile │ + ▲ │ discover new gated PRs -> label │ + │ GraphQL: checkSuites │ per labelled PR: fork state -> │ + │ REST: workflow state │ draft/undraft, labels, comment, │ + └────────────────────────│ commit status with run link, │ + │ close after 7 days or when the │ + │ PR tampered with the gate │ + │ unknown failures -> Slack │ + └──────────────────────────────────────────┘ +``` + +Four pieces, in dependency order: + +1. **Eligibility** — `dev/breeze/src/airflow_breeze/utils/fork_ci_gate.py`, one pure function + and one configuration file. Used by the gate job and by the reconciler. +2. **Project CI gate** — a `gate` job at the head of `ci-amd.yml`. +3. **Fork mode CI** — a push trigger plus fork-aware selective checks. +4. **Reconciler** — `breeze ci fork-ci-reconcile`, run by `fork-ci-monitor.yml` on a schedule. + +Plus `breeze ci audit` for the contributor's local setup, and documentation. + +## Configuration + +One file, `.github/fork-ci-gate.toml`, parsed with the standard library `tomllib`: + +```toml +# Fork CI gate (AIP-120). See dev/breeze/doc/ci/08_fork_ci_for_external_contributors.md + +# Master switch. When false the gate job lets every pull request run project CI, except +# pull requests labelled `use fork ci` and pull requests of [pilot] participants. Push +# filtering for apache/airflow branches applies regardless. +enabled = false + +# Pull requests created before this instant are never gated, whatever their author. +# Moving the date back is how the backlog reset (AIP-120 §9.1) would be applied. +enabled_since = 2026-01-01T00:00:00Z + +# Days without any fork CI activity after which a gated pull request is closed. +close_after_days = 7 + +# Organisations exempted under AIP-120 §2.2. Keys are display names used in comments, +# values are GitHub logins. Changes go through a reviewed pull request. +[allowlist] +# "Example Org" = ["login-one", "login-two"] + +# Opt-in pilot participants. Keys are GitHub logins, values are the instant they joined. +# Their pull requests created at or after that instant are gated whether or not `enabled` +# is set. Contributors opt in and out with a pull request editing this table. +[pilot] +# "login-three" = 2026-10-20T00:00:00Z +``` + +The file lives in the repository rather than in an Actions variable because ASF projects +cannot self-serve Actions variables, and because exemption changes should be reviewed like +any other change. The gate job reads it from the pull request's *base* branch, never from +the pull request's own tree, so a pull request cannot change its own eligibility. + +## Eligibility + +`fork_ci_gate.decide(event, config) -> Decision` is a pure function. `event` carries the +fields both callers have: event name, repository, ref, head repository, author login, +author association, pull request creation time and the pull request's current labels. +`Decision` is `run` (a bool) plus a `reason` from a fixed set, used in logs, comments and by +the reconciler. + +The rules, first match wins. The push rules are independent of `enabled` because the `'**'` +push trigger exists whether or not the gate is on, and today no CI runs for pushes to +`main` or to ad-hoc branches in `apache/airflow` (post-merge coverage is the cron canary). +The label and pilot rules are also independent of `enabled`; see "Override labels" and +"Contributor pilot" below. + +The `fork_schedule` rule exists because enabling `ci-amd.yml` in a fork also enables its +cron trigger, and selective checks force the full matrix for `schedule` events. Without it +every enabled fork would run the full canary matrix twice a day on the contributor's +concurrency cap. A manual `workflow_dispatch` in a fork still runs. + +| Rule | Decision | Reason | +|---|---|---| +| Event is `schedule` and repository is not `apache/airflow` | skip | `fork_schedule` | +| Event is `schedule` or `workflow_dispatch` | run | `scheduled` | +| Event is `push` and repository is not `apache/airflow` | run | `fork_push` | +| Event is `push` to `apache/airflow` and the branch matches the pre-existing push patterns (`v*-test`, `providers-*/v*`, `chart/v*x-test`, `airflow-ctl/v*-test`) | run | `canonical_branch` | +| Event is `push` to `apache/airflow`, any other branch | skip | `adhoc_branch` | +| Pull request carries `use project ci` | run | `label_project_ci` | +| Pull request carries `use fork ci` | skip | `label_fork_ci` | +| Author login is in `[pilot]` and the pull request was created at or after that login's join instant | skip | `pilot` | +| `enabled` is false | run | `gate_disabled` | Review Comment: Synced: this PR now carries the design doc and ADR exactly as they are at the top of the stack, including `head_is_upstream` before the label rules, `base_without_gate`, `needs project ci` and the new `CI passed` status. `green_unverified` no longer goes to Slack, so the ADR's "two expected outcomes" is true again. --- Drafted-by: Claude Code (Opus 5); reviewed by @potiuk before posting -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
