One thought, however: Following Christos' comments, AIP-120 gives us full freedom when to decide when to run CI in PR and when to run it from a fork. The "default" logic for that can be arbitrarily complex.
We could very easily - for example switch from "fork" to "project" CI when there is a single approval from a maintainer. However, I wouldn't start with that complexity (simpler is better). What I have already added to the design is our "standard" approach where we can add labels to the PR to essentially "force" the behaviour. One other note - of course someone **could** modify our gate in their PR and switch to "project" PR - but the reconciler could easily detect and cancel / close such PRs. J. . On Sat, Sep 19, 2026 at 5:28 PM Jarek Potiuk <[email protected]> wrote: > > I'm in favor of encouraging contributors to run the CI on their fork and > making sure that it's green before opening a PR > > I think it is not mutually exclusive with AIP-120 proposal - and in fact > AIP-120 > already accommodates it - what AIP-120 is **exactly** what you propose but > automatically and "nudged" when workflows are disabled in the fork and PR > to airflow is opened anyway. > > The issue with "telling" people to open PRs to their own forks is that > this is > **not** the default GH behavior. When you have a branch pushed to your > repo, > opening PR **automaticallly** select the forked repository as the target > - not your > own fork —and you have to manually change the base for the PR before you > create it. > > But AIP-120 already does what you ask. When people will enable workflows, > any "push" in their fork will **immediately** start the CI. Without them > even having to > open PR. This is what you wanted to ask them to do: enable workflows, and > AIP-120 doesn't "ask people to do it every time manually"; instead, it > enables it so that > this CI run happens automatically every time they push a branch. > > Basically the first time they open a PR to Airflow, and enable the > workflows in their fork, any push > in their fork will start CI jobs, That job is not as visible as in PR: you > will see it in 'Actions` of your > fork. However, when you open a PR (and we can do this regardless of > whether it is an "airflow" > PR or "fork" PR - they will get a link to the run posted as a comment in > their PR. > > So .. in a way AIP-120 enforces what you wanted to ask people to do in the > docs. Which of course > we can do anyway. > > J. > > > > > > On Sat, Sep 19, 2026 at 4:50 PM Christos Bisias <[email protected]> > wrote: > >> Hello everyone, >> >> I'm in favor of encouraging contributors to run the CI on their fork and >> making sure that it's green but BEFORE moving forward and opening a PR. It >> should be part of doing your due diligence and making sure that your >> changes aren't going to waste the reviewer's time. In my mind, not knowing >> if the CI is green isn't any different than opening a PR for code changes >> without any new tests. >> >> In some other Apache Open Source projects that I've contributed in the >> past, this was part of the contributing docs. Along with instructions for >> setting up the project, there was a section for enabling the Github >> actions >> in your fork and usually there was another part in the docs with good >> practices followed by the project that was referencing this. Part of the >> project standards was to make sure that you have a green CI before opening >> a PR. In Airflow's case, this would go next to the info about `prek` and >> it >> would say something like "prek makes sure that all static checks are >> passing but you should also check for any regressions in the tests by >> running the full CI in your fork". >> >> When I started contributing to Airflow I found it strange that I had to >> create a branch with a special name (v<X>-<Y>-test) just to run the CI on >> my fork. I asked on slack and the advice that I got from a few people was >> to just open a PR and if there are any failures in the CI, fix them >> afterwards. So after a while, I started doing the same as everyone else >> and >> continued launching a CI on my fork only for certain occasions. >> >> I think it's a good idea for contributors to run the CI on their forks but >> it should become widely known that this is the standard process and that >> it >> should happen before they open a PR. If that happens and they open a PR >> without having Github actions enabled on their fork or without a green CI, >> then it would make sense to convert their PR into a draft and explain why. >> This is a good practice for everyone, not just new contributors. >> Additionally, I think that it would make sense for the PR owner to be >> responsible for taking it out of draft when it's ready. >> >> Christos >> >> On Sat, Sep 19, 2026 at 5:37 PM Jarek Potiuk <[email protected]> wrote: >> >> > Also to give more details >> > >> > Here https://github.com/apache/airflow/pull/73378 - is draft design >> > proposal for the change. >> > >> > The PR explains design for the scope of change. It's actually very >> small: >> > one more workflow, one more gate job in ci-amd.yaml and two breeze >> commands >> > - "ci audit" and "ci reconcile" - first used by contributors to >> > check/set-up their fork, second used by CI to check >> status/relabel/comment >> > on the PRs. >> > >> > * Committer's/Collaborators/ Approved repos workflow remains unchanged >> > * For other contributors, >> > * If their "fork" workflows are not enabled : they get >> notification >> > on their PR that they should enable them (with `breeze ci audit` that >> does >> > a bit more checks - for standard remote names etc. or the single `gh >> > workflow enable` command and rebase their PR). Their PR is drafted >> (unless >> > it was already a draft). >> > * If their "fork" has workflows enabled: CI runs (slower for >> bigger >> > PR because of concurrency) - using their fork's CI (free, unlimited for >> > minutes) and once green, they got un-drafted (unless it was manually >> > drafted) and get "ready for maintainer review" label - no comments >> posted >> > * we can have different behaviour on what happens when they >> "loose" >> > any of that - i.e. if their CI becomes red etc. >> > >> > The whole thing is designed so that maintainers should practically never >> > need to look at a contributor's PR until the CI running in their >> workflow >> > is "green." We will mirror the PR's state (via API calls) from their >> fork's >> > CI status: for PRs without workflows, the status will be "Red", for >> those >> > in progress, the PR status will reflect that it is in progress, failed >> CI >> > in the fork will result in a failed PR; green will result in a green >> PR. We >> > will include a link to the run as an updated comment in the PR. >> > >> > >> > >> ┌───────────────────────────────────┬───────────┬────────────────┬─────────────────────────┬───────────────────────────────────────────┐ >> > │ │ │ ready for │ >> > │ │ >> > │ Observed fork state │ Draft │ maintainer │ >> > Commit status │ Comment │ >> > │ │ │ review │ >> > │ │ >> > >> > >> ├───────────────────────────────────┼───────────┼────────────────┼─────────────────────────┼───────────────────────────────────────────┤ >> > │ setup_required: workflow state is │ │ │ error >> > (red), link to │ create or update: how to enable, with the │ >> > │ disabled_fork, or the fork has │ draft │ remove │ the >> > fork's Actions page │ exact gh workflow enable ci-amd.yml -R │ >> > │ no ci-amd.yml │ │ │ for >> > ci-amd.yml │ <fork> line, and breeze ci audit │ >> > >> > >> ├───────────────────────────────────┼───────────┼────────────────┼─────────────────────────┼───────────────────────────────────────────┤ >> > │ awaiting_push: enabled, no run │ │ │ >> pending >> > (yellow), link │ on transition from setup_required: post a │ >> > │ for the head SHA │ draft │ remove │ to >> the >> > fork's Actions │ new comment asking to push again (an │ >> > │ │ │ │ page >> > │ edit would not notify) │ >> > >> > >> ├───────────────────────────────────┼───────────┼────────────────┼─────────────────────────┼───────────────────────────────────────────┤ >> > │ running: run in progress │ draft │ remove │ >> pending >> > (yellow), link │ update state and run link │ >> > │ │ │ │ to >> the >> > run │ │ >> > >> > >> ├───────────────────────────────────┼───────────┼────────────────┼─────────────────────────┼───────────────────────────────────────────┤ >> > │ failed: conclusion is failure, │ draft │ remove │ >> failure >> > (red), link to │ update with the run link │ >> > │ cancelled or timed out │ │ │ the >> run >> > │ │ >> > >> > >> ├───────────────────────────────────┼───────────┼────────────────┼─────────────────────────┼───────────────────────────────────────────┤ >> > │ green: conclusion success │ undraft* │ add │ >> success >> > (green), link │ update: ready for review, run link │ >> > │ │ │ │ to >> the >> > run │ │ >> > >> > >> ├───────────────────────────────────┼───────────┼────────────────┼─────────────────────────┼───────────────────────────────────────────┤ >> > │ setup_required or awaiting_push │ unchanged │ unchanged │ >> > unchanged │ close with a new comment; reopening │ >> > │ for close_after_days │ │ │ >> > │ restarts the clock │ >> > >> > >> └───────────────────────────────────┴───────────┴────────────────┴─────────────────────────┴───────────────────────────────────────────┘ >> > >> > * only if the last convert-to-draft event on the pull request was by >> > github-actions[bot]; a pull request the author drafted stays a draft. >> > >> > >> > I think - after the experience of pr triage, commenting etc. -> that >> sounds >> > pretty light-weight, and almost surgical change. >> > >> > J. >> > >> > >> > >> > On Sat, Sep 19, 2026 at 3:19 PM Jarek Potiuk <[email protected]> wrote: >> > >> > > > No limit for public forks does change things- i will re-read with >> that >> > > in mind >> > > >> > > Yeah. Precisely - the table they show is a bit ambiguous in that >> regard. >> > > >> > > In our initial PMC slack conversations **I remembered** hitting the >> > limits >> > > myself, so my initial proposal was including it, but I double checked >> and >> > > verified that what I remembered was hitting the limits in our private >> > > "airflow-s" repository, where we keep security issues, which is not >> part >> > of >> > > the "apache" organisation. BTW. This is one of the reason why GitHub >> > > Private Security Reporting does not have CI enabled - because the >> repos >> > > where you can open issues are private, and it would be a very easy >> > > "workaround" if it was possible to fork your public repo and have >> > unlimited >> > > CI in its private fork :) >> > > >> > > So I completely removed the 2000 minutes from the reasoning. It's in >> the >> > > AIP only as historical reference - but yeah I see how it can be >> confusing >> > > (and I can remove it entirely if it is confusing). Sorry if that was >> > > creating confusion. >> > > >> > > J. >> > > >> > > >> > > On Sat, Sep 19, 2026 at 3:13 PM Ash Berlin-Taylor <[email protected]> >> > wrote: >> > > >> > >> No limit for public forks does change things- i will re-read with >> that >> > in >> > >> mind >> > >> >> > >> > On 19 Sep 2026, at 14:01, Jarek Potiuk <[email protected]> wrote: >> > >> > >> > >> > Thanks Ash and Dennis - this is exactly the kind of pushback I >> hoped >> > >> for, >> > >> > and I'd love to hear from more people, especially those who are not >> > >> > committers. I am looking at the concerns seriously, but I think >> both >> > of >> > >> > you overestimate the actual impact on contributors. Let me explain >> > why. >> > >> > >> > >> > Dennis: >> > >> > >> > >> > It's really not *that* bad. The contributor experience is a single >> > >> > command, run once per fork: >> > >> > >> > >> > gh workflow enable ci-amd.yml -R <your-user>/airflow >> > >> > >> > >> > That's it. No sorting through 50 workflow files, no fork settings >> > pages, >> > >> > and the comment we post on the drafted PR will contain that exact >> > line. >> > >> > We can even wrap it in a breeze command that enables the workflows >> and >> > >> > audits the fork - checks that Actions are on, the right workflows >> are >> > >> > active, the branch is rebased - and tells you exactly what is >> missing >> > >> > before you push. We will also adjust triggers so that a plain push >> to >> > a >> > >> > branch in a fork runs CI there - no PR against your own fork >> needed. >> > >> > Everything else we *already* expect from contributors today: green >> CI, >> > >> > following the guidelines. The only new thing is "enable workflows >> in >> > >> > your fork". >> > >> > >> > >> > The monitor is not hand-wavy - it's deterministic and cheap. The >> > GitHub >> > >> > API tells us whether workflows in a fork are enabled (state >> > >> > "disabled_fork" vs "active") and whether the run for a given SHA is >> > >> > green. We can check it, and when someone enables workflows, ping >> them >> > >> > to rebase and undraft. A few REST/GraphQL queries per PR, no build >> > time. >> > >> > >> > >> > On "my PRs sit in CI for an hour": that is precisely the problem. >> Your >> > >> > jobs are not *running* for an hour - they are *waiting* in the ASF >> > >> > queue. I have a script that checks ASF-wide GitHub Actions usage >> and >> > >> > Airflow is routinely at the very top by number of jobs. We stay >> within >> > >> > ~6 full-time runners, which is below the limits Infra set, so we >> don't >> > >> > *have* to do anything - but this change would cut the number of >> jobs >> > we >> > >> > run significantly, and that is what shortens your hour. >> > >> > >> > >> > You are right on the ratio - 4392/31 is ~142x, not 60x. I'll fix >> the >> > >> AIP. >> > >> > >> > >> > Ash: >> > >> > >> > >> > "Want to contribute? pay" - you are not paying. You are just going >> > >> slower. >> > >> > I was wrong on the 2000 minutes earlier in slack and corrected it >> in >> > >> the AIP and >> > >> > in my mail: there are *no* minute limits for public repos, so a >> flaky >> > >> > test or a broken main costs you a re-run, not quota. The only >> limit is >> > >> 20 >> > >> > concurrent runners per account. That has a nice property: someone >> > >> > opening five large core PRs in a row waits a long time for all of >> them >> > >> to >> > >> > go green, while a one-line docs fix is unaffected. This is >> > >> "self-regulate >> > >> > the pace", not "pay to play". Please do read the AIP section on >> this - >> > >> > time is the only factor, money is not involved at all. >> > >> > >> > >> > On "lots of work before we might reject the idea": the work is >> > enabling >> > >> > workflows once with single command. The risk of a PR being rejected >> > >> > on design grounds exists today in exactly the same way. >> > >> > >> > >> > On 6.3 (gate on triaged issues): I am fully supportive and would be >> > >> > happy to see you write it up as a separate AIP - I'd even propose >> > doing >> > >> it >> > >> > spec-driven. But note that it is a far more impactful change to >> how we >> > >> > work than AIP-120: AIP-120 changes nothing in our process except >> where >> > >> CI >> > >> > runs, while issue-gating changes how features and fixes get >> approved. >> > >> The >> > >> > two are independent and complementary. >> > >> > >> > >> > The other side - the queue itself - is AIP-118, which Shahar and I >> are >> > >> > working on; feedback there is very welcome too. The root cause of >> the >> > >> > queue is that there is no prioritisation in the ASF pool - not >> between >> > >> > committers and external contributors, and not between projects. >> Infra >> > >> > is exploring GitHub Enterprise multi-tenancy, but that is months >> away >> > >> and >> > >> > comes with its own limits (a fixed runner count for Airflow shared >> by >> > >> > everyone could make things worse). With 118 and 120 together we get >> > >> three >> > >> > tiers: committers and canary on bigger self-hosted runners, >> approved >> > >> > contributors on the ASF pool, new/occasional contributors on their >> own >> > >> > free runners. If it works, the plan is to share the approach with >> > other >> > >> > PMCs, which makes the middle tier far less prone to queueing. >> > >> > >> > >> > J. >> > >> > >> > >> > >> --------------------------------------------------------------------- >> > >> > To unsubscribe, e-mail: [email protected] >> > >> > For additional commands, e-mail: [email protected] >> > >> > >> > >> >> > >> --------------------------------------------------------------------- >> > >> To unsubscribe, e-mail: [email protected] >> > >> For additional commands, e-mail: [email protected] >> > >> >> > >> >> > >> >
