This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch fork-ci-5-docs in repository https://gitbox.apache.org/repos/asf/airflow.git
commit 5bfcd262e95522f5b16db902be5f99222e696f15 Author: Jarek Potiuk <[email protected]> AuthorDate: Thu Oct 15 23:20:13 2026 +0100 Document fork CI for contributors and maintainers Contributors need to know how to set up CI in their fork, how to check that setup, and how to join or leave the pilot, and maintainers need to know which parts of a gated pull request's lifecycle the reconciler owns so triage does not fight it. The previous one-line advice to enable GitHub Actions in the fork did not say any of that. Generated-by: Claude Opus 5 --- contributing-docs/03_contributors_quick_start.rst | 13 +++++ .../03a_contributors_quick_start_beginners.rst | 5 ++ contributing-docs/05_pull_requests.rst | 9 +++- contributing-docs/10_working_with_git.rst | 3 ++ contributing-docs/18_contribution_workflow.rst | 63 +++++++++++++++++++++- contributing-docs/25_maintainer_pr_triage.md | 27 ++++++++++ dev/breeze/doc/ci/05_workflows.md | 21 ++++++++ .../doc/ci/08_fork_ci_for_external_contributors.md | 1 - 8 files changed, 138 insertions(+), 4 deletions(-) diff --git a/contributing-docs/03_contributors_quick_start.rst b/contributing-docs/03_contributors_quick_start.rst index 5adc87ad9dc..61b1af39148 100644 --- a/contributing-docs/03_contributors_quick_start.rst +++ b/contributing-docs/03_contributors_quick_start.rst @@ -237,6 +237,19 @@ Forking and cloning Project to clone the repo locally (you can also do it in your IDE - see the `Using your IDE`_ chapter below) +4. Add ``apache/airflow`` as the ``upstream`` remote, following the + `Git remote naming conventions <10_working_with_git.rst>`__: + + .. code-block:: bash + + git remote add upstream https://github.com/apache/airflow.git + + If you use SSH with GitHub, add ``[email protected]:apache/airflow.git`` instead; ``breeze ci audit`` + recognises both forms. + + Once Breeze is installed, run ``breeze ci audit`` before your first push to check your remotes + and the CI in your fork; see `CI in your fork <18_contribution_workflow.rst#ci-in-your-fork>`__. + .. note:: For windows based machines, on cloning, the Git line endings may be different from unix based systems and might lead to unexpected behaviour on running breeze tooling. Manually setting a property will mitigate this issue. diff --git a/contributing-docs/03a_contributors_quick_start_beginners.rst b/contributing-docs/03a_contributors_quick_start_beginners.rst index a8c333233ad..fdbea14df69 100644 --- a/contributing-docs/03a_contributors_quick_start_beginners.rst +++ b/contributing-docs/03a_contributors_quick_start_beginners.rst @@ -71,6 +71,11 @@ Option A – Breeze on Your Laptop git clone https://github.com/<you>/airflow.git cd airflow + git remote add upstream https://github.com/apache/airflow.git + +Before your first push, run ``breeze ci audit`` (after installing Breeze below) to check that your +remotes and the CI in your fork are set up; see +`CI in your fork <18_contribution_workflow.rst#ci-in-your-fork>`__. 2. **Set up Git hooks and install Breeze** diff --git a/contributing-docs/05_pull_requests.rst b/contributing-docs/05_pull_requests.rst index 4052986d8ba..d4e548f370a 100644 --- a/contributing-docs/05_pull_requests.rst +++ b/contributing-docs/05_pull_requests.rst @@ -94,8 +94,13 @@ these guidelines: The Airflow repo uses `GitHub Actions <https://help.github.com/en/actions>`_ to run the tests and `codecov <https://codecov.io/gh/apache/airflow>`_ to track - coverage. You can set up both for free on your fork. It will help you make sure you do not - break the build with your PR and that you help increase coverage. + coverage. Unless you are a committer, a collaborator or a member of an exempted organisation, + enable the ``Tests (AMD)`` workflow in your fork and every push to your fork runs CI for free + (committers and collaborators keep it disabled, so their pushes do not run CI twice). Pull + requests labelled ``fork ci required`` take their CI result from your fork + and stay in draft until that run is green; see + `CI in your fork <18_contribution_workflow.rst#ci-in-your-fork>`__ and run ``breeze ci audit`` + to check your setup. Also we advise to install locally `prek hooks <08_static_code_checks.rst#prek-hooks>`_ to apply various checks, code generation and formatting at the time you make a local commit - which gives you near-immediate feedback on things you need to fix before you push your code to the PR, or in diff --git a/contributing-docs/10_working_with_git.rst b/contributing-docs/10_working_with_git.rst index e51cb1a1f21..edb6b54188d 100644 --- a/contributing-docs/10_working_with_git.rst +++ b/contributing-docs/10_working_with_git.rst @@ -37,6 +37,9 @@ you should configure your local checkout to match: * ``origin`` — your personal fork of ``apache/airflow`` (where you push branches for PRs). +``breeze ci audit`` checks these names, offers the commands below when they differ, and also checks +the CI in your fork; see `CI in your fork <18_contribution_workflow.rst#ci-in-your-fork>`__. + Always push PR branches to ``origin``; don't push to ``upstream`` (the branch protection on ``apache/airflow`` will reject it anyway). Working on dedicated branches is recommended, though developing directly on your fork's ``main`` is diff --git a/contributing-docs/18_contribution_workflow.rst b/contributing-docs/18_contribution_workflow.rst index b96f4f16238..112a825f332 100644 --- a/contributing-docs/18_contribution_workflow.rst +++ b/contributing-docs/18_contribution_workflow.rst @@ -43,7 +43,8 @@ In general, your contribution includes the following stages: 2. Create a `local virtualenv <07_local_virtualenv.rst>`_, initialize the `Breeze environment <../dev/breeze/doc/README.rst>`__, and install `prek tool <08_static_code_checks.rst#prek-hooks>`__. - If you want to add more changes in the future, set up your fork and enable GitHub Actions. + Set up the ``upstream`` and ``origin`` remotes, enable CI in your fork and check the result + with ``breeze ci audit``, as described in `CI in your fork`_. 3. Join `devlist <https://lists.apache.org/[email protected]>`__ and set up a `Slack account <https://s.apache.org/airflow-slack>`__. @@ -126,6 +127,66 @@ to make them immediately visible in the environment. 6. Open your IDE (for example, PyCharm) and select the virtual environment you created as the project's default virtual environment in your IDE. +CI in your fork +............... + +Pull requests from contributors who are not committers or collaborators are moving to running +their CI in the contributor's own fork, on GitHub's free runners, instead of on the shared CI +capacity of the Apache Software Foundation (`AIP-120 +<https://cwiki.apache.org/confluence/spaces/AIRFLOW/pages/451974711/AIP-120+Run+CI+for+external+contributions+in+contributor+forks>`__). +CI on a public fork costs you nothing and has no minute limit; it runs on every push to your +fork, so you get feedback before a maintainer looks at your pull request. The change is being +piloted: until it is switched on for everyone, it applies to pilot participants (see below) and +to pull requests a maintainer labels ``use fork ci``. + +Set up your checkout and fork once: + +1. Name your remotes as described in `Git remote naming conventions <10_working_with_git.rst>`__: + ``upstream`` for ``apache/airflow`` and ``origin`` for your fork. + +2. Unless you are a committer, a collaborator or a member of an exempted organisation, enable the + ``Tests (AMD)`` workflow in your fork. GitHub disables workflows in new forks. Committers and + collaborators keep it disabled, so their pushes do not run CI twice, unless they joined the pilot: + + .. code-block:: bash + + gh workflow enable ci-amd.yml -R <you>/airflow + + If the workflow is missing, your fork's ``main`` is older than it; sync it first with + ``gh repo sync <you>/airflow --branch main``. + +3. Check everything at once. ``breeze ci audit`` prints one row per check and the command that fixes + anything missing; ``--fix`` runs those commands after asking: + + .. code-block:: bash + + breeze ci audit + +When fork CI applies to your pull request, it is labelled ``fork ci required`` and kept in draft +until the ``Tests (AMD)`` run in your fork for its latest commit is green. A +``Fork CI / Tests (AMD)`` status on the pull request links to that run, and one comment on the pull +request explains the current state. Once the run is green the pull request is labelled +``ready for maintainer review``, and marked ready for review if the monitor drafted it; a pull +request you drafted yourself stays a draft until you mark it ready. A pull request whose fork CI does not run for 7 days is +closed with an explanation; you can reopen it after setting up fork CI. If your pull request changes +files that define what CI runs, such as ``.github/``, ``dev/breeze/`` or ``scripts/ci/``, a green +fork run is not enough: a maintainer reviews the change and runs project CI for it. + +**Joining the pilot.** Open a pull request that adds your GitHub login to the ``[pilot]`` table in +``.github/fork-ci-gate.toml``, with a join instant after the expected merge, for example the next day +at 00:00 UTC: + +.. code-block:: toml + + [pilot] + "your-login" = 2026-10-20T00:00:00Z + +Every pull request you open after that instant uses fork CI; your earlier pull requests keep project +CI. To leave the pilot, open a pull request that removes the line. A pilot entry gates your pull +requests whatever your role, so a committer in the pilot also enables the workflow; +``breeze ci audit`` tells you which applies to you. The design is described in +`Fork CI for external contributors <../dev/breeze/doc/ci/08_fork_ci_for_external_contributors.md>`__. + Step 3: Connect with People --------------------------- diff --git a/contributing-docs/25_maintainer_pr_triage.md b/contributing-docs/25_maintainer_pr_triage.md index 046a1b20fa2..2196f8db177 100644 --- a/contributing-docs/25_maintainer_pr_triage.md +++ b/contributing-docs/25_maintainer_pr_triage.md @@ -242,6 +242,33 @@ The triage workflow uses the following labels and states: | `suspicious changes detected` | PR (and all open PRs by the same author) was closed because the diff contained suspicious patterns (secret exfiltration, malicious CI modifications, etc.). | | `closed because of open PR limit` | PR was closed in the one-time closure that introduced the [open PR limit](32_open_pull_request_limit.rst). Applied by `dev/close_prs_over_open_pr_limit.py`, never by the triage skill. If the author reopens the PR, triage it like any other. | | **Draft status** | PR was converted to draft because it does not meet quality criteria. The contributor is asked to fix the listed issues and mark the PR as "Ready for review" once done. | +| `fork ci required` | PR runs its CI in the contributor's fork (AIP-120). Applied and managed by the fork CI reconciler, never by the triage skill. | +| `needs project ci` | Fork CI is green, but the PR changes CI-defining files, so a maintainer must review them and add `use project ci`. Applied and removed by the fork CI reconciler. Filter on it to find PRs waiting only on a maintainer. | +| `use project ci` / `use fork ci` | Set by a maintainer to override the fork CI decision for one PR: run project CI, or use fork CI for a PR that would otherwise run project CI. | + +## Pull requests gated to fork CI + +Pull requests labelled `fork ci required` run their CI in the contributor's fork. The scheduled +fork CI reconciler (`breeze ci fork-ci-reconcile`, run by `fork-ci-monitor.yml`) owns four things +for them, and the triage skill must treat those pull requests as out of scope for each: + +- **Draft state.** The reconciler keeps the pull request in draft until the fork run for its + latest commit is green, and undrafts only what it drafted. +- **`ready for maintainer review`.** The reconciler adds it when the fork run is green and removes + it when a new push starts a new run. +- **`needs project ci`.** The reconciler adds it when the fork run is green but the pull request + changes CI-defining files, and removes it once the state is anything else. These pull requests + stay in draft and wait for a maintainer to review the CI change and add `use project ci`. +- **Closing after inactivity.** The reconciler closes a pull request whose fork CI has not run for + `close_after_days`. + +Everything else the skill does (quality criteria, pings, suspicious-change flags) applies as +usual. The `Fork CI / Tests (AMD)` commit status shows the fork CI state and links to the run in +the fork. To move a single pull request back to project CI, add `use project ci`; the reconciler +releases it on its next run. Pull requests that change CI-defining files stay in draft even when +the fork run is green, and are reported to `internal-airflow-ci-cd` for a maintainer to review and +run project CI with `use project ci`. See +[Fork CI for external contributors](../dev/breeze/doc/ci/08_fork_ci_for_external_contributors.md). ## For contributors diff --git a/dev/breeze/doc/ci/05_workflows.md b/dev/breeze/doc/ci/05_workflows.md index e62dc41a283..3a417bbc3c3 100644 --- a/dev/breeze/doc/ci/05_workflows.md +++ b/dev/breeze/doc/ci/05_workflows.md @@ -23,6 +23,7 @@ - [CI run types](#ci-run-types) - [Pull request run](#pull-request-run) + - [Fork push run](#fork-push-run) - [Canary run](#canary-run) - [Workflows](#workflows) - [Workflow Architecture Overview](#workflow-architecture-overview) @@ -92,6 +93,26 @@ all dependencies to latest versions and build images with them. This way we chec dependencies are upgraded. This can also be forced by setting the `upgrade to newer dependencies` label in the PR if you are a committer and want to force dependency upgrade. +Every pull request run of `ci-amd.yml` starts with the `Fork CI gate` job. For pull requests +whose CI runs in the contributor's fork (AIP-120), the gate skips `build-info` and therefore +everything after it, so the run finishes in seconds; see +[Fork CI for external contributors](08_fork_ci_for_external_contributors.md). + +## Fork push run + +These runs are triggered by a push to any branch of a contributor's fork in which the +`Tests (AMD)` workflow is enabled. They run in the fork, on GitHub's public runners and with the +fork's token, and are how pull requests labelled `fork ci required` get their CI result. They are +selected like a pull request run: the changed files come from comparing the pushed commit with +`apache/airflow` (`main`, or the release branch the pushed branch name starts with), so a fork +whose `main` is behind does not make the run bigger, and they never use self-hosted runners. The +`schedule` trigger is skipped in forks, so enabling the workflow does not start the canary +matrix there. The fork CI reconciler reads the result of these runs and reports it on the pull +request as the `Fork CI / Tests (AMD)` status. + +In `apache/airflow` itself, pushes still only run CI for the release-prep and providers branches +listed in `ci-arm.yml`; a push to any other branch produces a run in which only the gate executes. + ## Canary run This workflow is triggered when a pull request is merged into the `main` diff --git a/dev/breeze/doc/ci/08_fork_ci_for_external_contributors.md b/dev/breeze/doc/ci/08_fork_ci_for_external_contributors.md index 79f2358e476..5bfec73929e 100644 --- a/dev/breeze/doc/ci/08_fork_ci_for_external_contributors.md +++ b/dev/breeze/doc/ci/08_fork_ci_for_external_contributors.md @@ -801,7 +801,6 @@ as a failed check rather than a crash. - `contributing-docs/10_working_with_git.rst`: cross-link from the naming conventions. - `contributing-docs/25_maintainer_pr_triage.md`: the ownership split with the reconciler. - `dev/breeze/doc/ci/05_workflows.md`: add the fork push run type. -- `dev/breeze/doc/ci/03_github_variables.md`: the new gate job outputs. - `dev/breeze/doc/08_ci_tasks.rst`: `breeze ci audit` and `breeze ci fork-ci-reconcile`. - `dev/breeze/doc/ci/README.md`: index this document.
