Well, my first experience is not good https://github.com/ashb/airflow/actions/runs/42013825789 just says “Startup Failure”. There is unfortunately no indication on the PR itself that this has failed — no link, no status etc.
(These are just teething problems for sure, so not letting this sway my voting too much.) > On 5 Nov 2026, at 10:51, Ash Berlin-Taylor via dev <[email protected]> > wrote: > > Thanks, testing it here https://github.com/apache/airflow/pull/76536 > > The main thing I want to see around is the experience of “fork CI is green, > now what do we have to do to merge it”. > >> On 5 Nov 2026, at 04:17, Jarek Potiuk <[email protected]> wrote: >> >> Yes. >> >> What Shahar wrote - that was also from the beginning part of the AIP-120 >> proposal - having labels, way to test it - and we already tried it over >> last 2 weeks, made a number of improvements, have a few edge cases to >> handle (all described in the VOTE email) and thread open in #contributors >> where we discussed all that. The email in the devlist above is just a >> summary of it. >> >> So - absolutely, Ash - add "use fork ci" on your PRs and you will get the >> experience. Even more - you can actually, review, rebase, re-run, review, >> and approve any of the PRs from the people listed in >> https://github.com/apache/airflow/blob/main/.github/fork-ci-gate.toml - who >> offered to test it during the pilot. Report issues or raise concerns if you >> want, and we (and even some of the contributors who volunteered to pilot >> it) are addressing any issues and ironing the wrinkles within hours. We >> have a whole team of people who are able to help address any of your >> concerns there (and they did so over the last 2 weeks). >> >> J. >> >> On Wed, Nov 4, 2026 at 12:40 PM Shahar Epstein via dev < >> [email protected]> wrote: >> >>> Ash - to test it out, simply label your own PR with "use fork ci" and >>> trigger the workflow (push any change). >>> The participants of the pilot get this auto-labeled, but maintianers could >>> apply it manually as well. >>> >>> >>> https://github.com/apache/airflow/blob/0ae09c32182e2fd0f4f50b1ce423dd1b42f77343/contributing-docs/18_contribution_workflow.rst#ci-in-your-fork >>> >>> --- >>> >>> +1 (binding) from me, this is really needed at this point of time. >>> >>> >>> Shahar >>> >>> On Wed, Nov 4, 2026 at 2:52 PM Ash Berlin-Taylor via dev < >>> [email protected]> wrote: >>> >>>> Before I vote on this, is there a way I can test out the proposed >>>> model/changes so I can see what the flow looks like, both from a >>> maintainer >>>> and a contributor? >>>> >>>>> On 4 Nov 2026, at 11:06, Jarek Potiuk <[email protected]> wrote: >>>>> >>>>> Hello everyone, >>>>> >>>>> I am calling a vote on AIP-120, "Your PR, your CI" [1]. As I said when >>>>> I withdrew the earlier [RESULT] and reopened the discussion [2], this >>>>> goes to a vote rather than [LAZY CONSENSUS], because not everyone who >>>>> objected has withdrawn their objection. >>>>> >>>>> AIP-120 will change to ADIP-12 if the separate [LAZY CONSENSUS] on the >>>>> new ADIP wiki section passes [3]. That only changes where the page >>>>> lives and what it is called. This vote does not depend on it. >>>>> >>>>> What we are voting on >>>>> --------------------- >>>>> >>>>> Whether to switch on CI in contributors' forks for all external >>>>> contributors, as described in AIP-120 [1] and the design [4]: >>>>> >>>>> * PRs from external contributors run CI in the contributor's own fork, >>>>> on GitHub's free runners, instead of on the ASF runners. Committers, >>>>> collaborators and exempted organisations keep project CI, unchanged. >>>>> >>>>> * A contributor enables one workflow in their fork, once. After that a >>>>> plain "git push" runs CI. No PR against their own fork is needed, and >>>>> "breeze ci audit" checks the setup. >>>>> >>>>> * The PR shows the fork run as a commit status linking to the run. A PR >>>>> is drafted when the fork needs setup or the run failed, and undrafted >>>>> when the run is green. A run in progress does not draft it. >>>>> >>>>> * A green fork run is not enough when the PR changes what decides which >>>>> jobs run and on which runners (workflows, breeze sources and similar). >>>>> Such PRs stay visible for review, but wait for a maintainer to run >>>>> project CI with the "use project ci" label. Maintainers can use that >>>>> label on any PR they want verified in project CI before merging. >>>>> >>>>> * A gated PR with no fork CI activity for 7 days is closed, and can be >>>>> reopened. >>>>> >>>>> Not part of this vote: >>>>> >>>>> * The one-time reset of the open PR backlog. It is not proposed here. >>>>> * Which organisations are exempted. That is agreed separately on this >>>>> list. >>>>> >>>>> Where things stand >>>>> ------------------ >>>>> >>>>> The implementation is merged and switched off for everyone except the >>>>> pilot: "enabled = false" in .github/fork-ci-gate.toml. If the vote >>>>> passes, I will post a final pilot update on this list and then switch >>>>> it on in a separate PR. Rollback is setting "enabled = false" again. If >>>>> the vote does not pass, it is not switched on for all external >>>>> contributors. >>>>> >>>>> What the pilot showed >>>>> --------------------- >>>>> >>>>> Seven external contributors joined the opt-in pilot on 2026-10-31. So >>>>> far 16 new PRs carry the "fork ci required" label, 9 of them merged, >>>>> plus some older PRs that were rebased. Feedback is in the >>>>> #internal-airflow-ci-cd Slack channel. Some quotes: >>>>> >>>>> "So far, this has been far less inconvenient that I expected. The >>>>> only downside is that the CI workflow takes a bit longer on my own >>>>> fork that when using the projects runners." - Sameer Mesiah >>>>> >>>>> "Total run time was ~30 minutes, which is basically on par with how >>>>> long it was taking on the shared Apache CI (for this PR >>>>> specifically)." - Justin Pakzad >>>>> >>>>> "I was actually quite surprised that the runtimes were pretty close >>>>> (in some cases quicker) on my fork. I only tested a provider/task-sdk >>>>> PRs, I'm assuming larger core changes will def take longer." - Justin >>>>> Pakzad >>>>> >>>>> "I love it that I can finally rerun failed jobs since they're running >>>>> on the fork" - Iliya Romm >>>>> >>>>> That last one was a benefit we did not plan: contributors can restart >>>>> a single failed job themselves, without asking a maintainer. >>>>> >>>>> It was not all smooth, and that is what the pilot was for: >>>>> >>>>> "this had a green (main) CI run, with two maintainer approvals, then >>>>> I rebased and the fork CI failed, which auto-marked the PR as draft. >>>>> [...] I think the auto-draft rules should be more careful." - Iliya >>>>> Romm >>>>> >>>>> "My PR only changes documentation [...] but fork CI converted it to >>>>> draft automatically." - Aaron Chen >>>>> >>>>> "GitHub just disabled it without any notification, so my workflows on >>>>> fork silently stopped" - Andrew Chang >>>>> >>>>> Improvements made after the pilot started >>>>> ----------------------------------------- >>>>> >>>>> * Rebased pilot PRs are now gated, not only new ones, and a required >>>>> "CI run started" check shows from the moment of the push (#76148). >>>>> * Cancelled project CI runs no longer leave a mixed red/green status >>>>> on the PR (#76148). >>>>> * A run in progress no longer drafts a PR, so rebasing an approved PR >>>>> does not draft it unless the new run fails (#76151). >>>>> * One pending status links to the fork run, replacing a wall of pending >>>>> jobs. This also spends fewer GitHub API writes, and we now watch the >>>>> rate limits (#76154). >>>>> * Scheduled workflows are skipped in forks. They were the likely reason >>>>> GitHub disabled Actions on some forks (#76243). >>>>> * "breeze ci audit" checks that the fork's GitHub Actions can actually >>>>> run (#76136, by Andrew). >>>>> * Only changes that can steer CI are held for a maintainer, and they >>>>> are no longer drafted, so docs under dev/breeze/ pass normally >>>>> (#76282). >>>>> * Fork CI PRs that change CI files no longer get the misleading "need >>>>> workflow approval" label (#76289, by Andrew). >>>>> * A re-run that passes now replaces the failed job statuses on the PR >>>>> (#76327). >>>>> * Fork image names are lowercased in the PROD image example tests >>>>> (#76143), the contributor docs were improved (#76272, by Aaron), and >>>>> transient GitHub errors no longer cause repeated alerts (#76185). >>>>> >>>>> Still open: >>>>> >>>>> * Status on the PR lags, because the monitor runs every 15 minutes. >>>>> Aaron saw the first bot action after about 23 minutes. >>>>> * The monitor's budget for job status writes still fills up with >>>>> several large fork runs active. We need to watch this as more >>>>> contributors join. >>>>> * A failed run after a rebase still drafts the PR, which can cancel >>>>> auto-merge when the failure is a flaky test. Iliya also suggested that >>>>> maintainer approvals could count towards running project CI. >>>>> * The labels are confusing and could use better names. >>>>> * Pilot PRs were mostly small to medium. Large core changes have had >>>>> little testing. >>>>> >>>>> The objections >>>>> -------------- >>>>> >>>>> You should vote knowing what was raised. Please read the original >>>>> thread [5] and the [RESULT] thread [6] rather than rely on my summary. >>>>> >>>>> * Ash's main objection is that this optimises for the wrong thing: the >>>>> problem is PRs not being reviewed, and making contributors run CI >>>>> does nothing about that backlog. Ash also pointed out that allowing >>>>> contributors to run CI in forks and forcing them to are separate >>>>> things. This vote is about requiring it. Ash has not withdrawn this >>>>> objection. >>>>> >>>>> My answer: it does not add review capacity, and I do not claim it >>>>> does. The aim is to slow down iteration on big PRs, so that fewer of >>>>> them compete for review, and to move the cost of getting a PR green >>>>> to its author. How much that helps is my expectation, not a measured >>>>> result. >>>>> >>>>> * Ash and Dennis both raised the effect on new contributors. Dennis >>>>> described it as a big barrier to entry, and Ash the case of someone >>>>> doing all the setup only to have the PR rejected afterwards. >>>>> >>>>> My answer: the setup is now one step, with no second PR and no >>>>> sorting through workflow files, but it is still an extra step that a >>>>> new contributor did not have before. That cost is real. The pilot >>>>> participants are experienced contributors, so the pilot does not tell >>>>> us how a first-time contributor will find it. >>>>> >>>>> * Slower CI for big PRs is intended, but it should be stated plainly. >>>>> On 2026-10-05 I estimated about 85 minutes or more for a full matrix >>>>> run in a fork. The first real full run (#75695) took about 2 hours 40 >>>>> minutes. >>>>> >>>>> On the other side, Arrow, DataFusion and SeaTunnel already run PR CI in >>>>> contributors' forks, or are moving to it, mainly to reduce pressure on >>>>> the shared ASF runners. For us that is a side benefit. >>>>> >>>>> How the vote works >>>>> ------------------ >>>>> >>>>> This is a procedural vote, for the reasons in [2]: where CI runs for >>>>> contributors' PRs is a decision about how the project operates, not a >>>>> change to Airflow itself or its public interfaces. Hussein asked about >>>>> this on the [RESULT] thread, and nobody objected to it in the reopened >>>>> discussion. >>>>> >>>>> It passes with at least 3 binding +1 votes and more binding +1 than -1 >>>>> votes [7]. Everyone is welcome to vote; votes from PMC members are >>>>> binding. A single -1 does not block it, but I will respond to every -1. >>>>> >>>>> The vote runs until 2026-11-09 12:00 UTC. That is longer than the >>>>> 72-hour minimum because of the Summit Reconnect on 4-5 November and the >>>>> weekend. >>>>> >>>>> [ ] +1 Switch on CI in contributors' forks for external contributors, >>>>> after a final pilot update >>>>> [ ] 0 No opinion >>>>> [ ] -1 Do not do this, because... >>>>> >>>>> [1] >>>> >>> https://cwiki.apache.org/confluence/spaces/AIRFLOW/pages/451974711/AIP-120+Run+CI+for+external+contributions+in+contributor+forks >>>>> [2] https://lists.apache.org/thread/h5qsfqthmsqoh5j7wyy12r90mnnvz8kt >>>>> [3] https://lists.apache.org/thread/qxjx3lnwm8gro0dstqr3cvgyj5jg6hyk >>>>> [4] >>>> >>> https://github.com/apache/airflow/blob/main/dev/breeze/doc/ci/08_fork_ci_for_external_contributors.md >>>>> [5] https://lists.apache.org/thread/w74655o8y1jzrzcnygntyyovocf7zmzj >>>>> [6] https://lists.apache.org/thread/krl1bzszlfgfz70m494rm8jq1ph6dr01 >>>>> [7] https://www.apache.org/foundation/voting.html >>>>> >>>>> J. >>>>> >>>>> --------------------------------------------------------------------- >>>>> To unsubscribe, e-mail: [email protected] >>>>> For additional commands, e-mail: [email protected] >>>>> >>>> >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: [email protected] >>>> For additional commands, e-mail: [email protected] >>>> >>>> >>> > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] >
