I would love to hear any comments for that one. I think that should help us to deal with PR overload and also it will have a nice effect of freeing a lot of resources in shared Apache Runners Queue. I will share this with other PMCs as well and ask them to do similar changes (perhaps even running a campaign to help them open PRs, as I did with Trust models) - but I need to know if this is something we would like to try and also see the effects of it before.
I think we already see the effects of the PR limits step: the number of PRs created by non-committers modestly decreased around/after PR limit introduction. Since then, the number of open PRs has hovered around 800 and has not grown as it used to for many previous weeks, so I would cautiously say it worked as expected (in addition to the 200+ drop when I closed the PRs of the "over-limited" contributors). A short summary table and AI-assisted assessment below [1]. So, unless there is strong opposition to trying it, I would like to proceed with (hopefully) LAZY CONSENSUS for implementation next week. I can do this before the workshops we are planning next week in Glasgow, which might also give a unique opportunity to test the workflow of new contributors from scratch - using these workflows. Together with a few other committers, we could quickly fix any issues new contributors might have. J. --------------- [1] Analysis of creation of external PRs (AI assisted): What this longer view suggests: external PRs varied substantially before the limit discussion—roughly 126–197 per week in the weeks shown. They were at 132 in the last full week before discussion, 145 in the mixed discussion/closure week, and 118 in the first full week after closure. That’s a modest decrease from the pre-discussion week, not a clear drop at discussion time. The latest partial week has 76 so far, but it’s too early to compare directly with full weeks. So, with bots removed, I’d describe the pattern as a post-closure dip in external PRs, against a noisy and variable pre-existing baseline—not yet enough to establish that the limit caused a sustained reduction. Week (2026) Committer* External PR* Jul 13–19 100 127 Jul 20–26 113 171 Jul 27–Aug 2 162 184 Aug 3–9 114 144 Aug 10–16 85 142 Aug 17–23 95 138 Aug 24–30 81 126 Aug 31–Sep 6 37 185 Sep 7–13 181 197 Sep 14–20 113 132 Sep 21–27† 161 145 Sep 28–Oct 4‡ 185 118 Oct 5–8‡ 120 76 J, On Mon, Oct 5, 2026 at 1:01 PM Jarek Potiuk <[email protected]> wrote: > Hello everyone, > > It has been 10 days since we introduced the limit of 5 open PRs for > contributors without write access. I want to share early results and > propose the next step. > > TL;DR > > * The PR limit is doing what we hoped. The data is early (9 days), and > I will redo the analysis in a few weeks. > * Proposal: PRs from external contributors run their CI in the > contributor's own fork, on GitHub's free runners, instead of on the > ASF runners. Nothing changes for committers or for a few stakeholder > repositories the PMC decides on. > * This is NOT "pay-to-play": CI on public forks is free and has no > limit on minutes. The only limit is 20 jobs running at the same time. > Small PRs are practically unaffected; big, Airflow-wide PRs take > longer to get a CI result. > * If there is general consensus, I will start a [LAZY CONSENSUS] next > week and start rolling it out right after. I already have a design. > > 1. Early results of the PR limit > > Charts and numbers are here: > https://gist.github.com/potiuk/a2048d6b5e67c5c2eb07a089c335d02c > > I compared PRs opened from January to Oct 4, excluding bots and > backports. The window after the limit is only 9 days, so these are > early observations, not conclusions: > > * Bulk submissions dropped sharply. PRs from contributors who opened > more than 5 PRs in a week went from ~63/week (the 4 weeks before) > and ~42/week (Jul-Aug) to 17 in the first week after. That is back > to the level of the first half of the year. > * We did not scare people away. 74 different contributors opened PRs > in the first week after the limit, compared with 82/week just before > and 76/week in Jul-Aug. The PRs are spread across more people. > * Contributor PRs per day are ~22% lower than in the 4 weeks before, > but the same as in the 9 days just before the limit. Part of the > decrease happened while we were discussing the limit. > * The open queue of contributor PRs went from ~814 to ~590. This is > mostly the one-time closure of PRs above the limit. Before that, the > queue had grown from ~260 in January to ~870 in early September. It > has not grown since (!) > * Committers' PRs were not affected. > * The share of PRs opened as drafts, which do not count towards the > limit, rose slightly (4% -> 7%). Too early to tell, but worth > watching. > > I will re-run the same analysis in 2-3 weeks and share the results. > > 2. Why a next step is needed > > AI changed the balance between creating code and reviewing it. > Producing a PR now costs almost nothing. Reviewing it costs the same > as before - and maintainer time is the one resource we cannot scale. > More reviewers would help, but we cannot expect 10x more review time. > > Today, the whole cost of iterating on a PR is on our side: our review > time, and our (ASF-donated) CI capacity, used for every push of every > PR. The PR limit caps how many PRs one person can have open. It does > not change who carries the cost of getting a PR into a reviewable > state. That is what this proposal is about. > > 3. The proposal: "your PR, your CI" > > * For contributors without write access, the CI runs in their own fork > (on push to the branch), using GitHub's free runners. To get it, a > contributor only has to enable GitHub Actions in their fork - once. > * The ASF runners keep running CI for: > - committers, > - a small number of stakeholder repositories the PMC decides on (for > example forks/repos used by Astronomer, AWS and Google, which > contribute a lot of the code and run shared development), > - anyone else the PMC decides to add. > For all of them, the experience does not change at all. > * The PR shows the result of the CI from the fork, so reviewers see > the same green/red status as today. PRs without a CI result get a > clear, friendly message explaining what to do. > > 4. What it means for contributors - iteration speed > > GitHub's free plan gives public repositories unlimited minutes on > standard runners. The limit is concurrency: at most 20 jobs at the > same time per account. Our CI uses "selective checks" - it runs only > what a change can affect - so the impact depends on the size of the > change. I measured real runs from today and replayed them on 20 > concurrent runners (these are estimates, slightly optimistic): > > Type of PR Jobs Job-minutes ASF runners 20 runners > UI translations only 33 132 ~49 min ~49 min > Single provider (Trino) 50 300 ~31 min ~34 min > Core fix (selective) 37 275 ~35 min ~35 min > Full matrix (CI change) 138 1307 ~49 min ~85 min+ > > The full canary run on main (309 jobs) would take ~4h instead of ~2h, > but that does not run for contributor PRs. > > So small, focused PRs - a single provider, docs, UI, a localized core > fix - are practically unaffected. Only large PRs that touch "all of > Airflow" and need the full matrix will take longer to get a result, > roughly 1.5-2x - so it will be slower to iterate on them. > > I think this is exactly in line with what we want. Big, cross-cutting > PRs are also the ones that cost maintainers the most review time. > Making their author's iteration a bit slower aligns the incentives: > smaller, focused PRs get the fastest feedback, both from CI and from > reviewers. It is also the path we already recommend to new > contributors. And paired with PR limits - it further limits possibility > of one person producing a "mountain of reviews". > > 5. Next steps > > * Please share your thoughts in this thread. > * If there is a general consensus, I will start a [LAZY CONSENSUS] > next week and put it in motion next week as well. I already have a > design of how it can be implemented with our current workflows. > * More steps will follow. I am using Apache Magpie to iterate on and > test some of them, but none of those changes will require anyone to > use Magpie. > > A note about Magpie, because the approach changed based on the > feedback we got. Magpie is now focused on individual maintainers, who > can experiment and build flows that help them do things faster - > without needing project changes and without others having to use > it. What benefits the whole community should then become deterministic > things: CI, GitHub/repository configuration, documentation - plus > analysis of the results, like the one above. This proposal is an > example: Magpie helped me prototype and analyze it, but the result is > plain GitHub Actions configuration. > > 6. FAQ - concerns raised so far > > Q: Isn't this "pay-to-play" / "pay-to-contribute"? > > A: No. Nobody has to pay anything. CI for public repositories on > GitHub's free plan has no minute limit. A contributor gets the same > CI, for free, in their fork. The only difference is that at most 20 > jobs run at the same time, which affects only large PRs, and only > how fast they get a result. Contributors who need more can use > runners of their employer, and the PMC can add repositories to the > list that uses ASF runners. > > Q: Will contributors run out of the 2,000 free minutes per month? > > A: No. That limit applies to private repositories. Public forks of > Airflow have no minute limit on standard GitHub runners. > > Q: How does this reduce the number of PRs? Isn't the real problem that > PRs are not reviewed? > > A: The main goal is not fewer PRs - it is fewer PRs that are not > ready to review, and moving the cost of getting a PR green to the > author. The review problem is real, and we keep working on it in > parallel (review rotation, better triage, surfacing what needs a > review). These are complementary, not alternatives. > > Q: Can we trust the result of CI running in a contributor's fork? They > could modify the workflows. > > A: Contributors can already modify workflow files in their PRs today. > We review the code changes, not CI logs, and every merged change > runs the full canary build on main. If we want, maintainers can > keep an option to run the CI on ASF runners before merging a PR. > > Q: What about secrets, variables and other integrations? > > A: None are needed. PRs from forks already run without secrets today, > and our CI was designed this way. > > Q: Will cancelled or missing CI runs confuse contributors? > > A: Each PR will get a clear message and a link to documentation > explaining what to do: enable Actions in your fork, push again. > This is a one-time step. > > Q: Why not ask the ASF for more CI resources, or use donated cloud > credits? > > A: We are doing that too, and it will make CI faster for everyone. But > more CI capacity does not create more review time. Free CI for > unlimited PRs is part of what makes producing PRs cheaper than > reviewing them. > > Q: Why not run a smaller matrix in PRs (e.g. only the oldest Python)? > > A: That is a good idea and complementary. Selective checks already do > a lot of that, and we can trim further. It makes CI cheaper; it > does not change who carries the cost of iterating. > > Q: Why not require an accepted issue before opening a PR? > > A: It would move the load from PRs to issues - many agents already > open an issue and a PR solving it at the same time - and someone > would still need to triage and accept those issues. We can still > discuss it separately though - again this is complementary not > replacement. > > Q: Does this affect committers or our regular stakeholders? > > A: No. Committers and the repositories the PMC adds to the list keep > using ASF runners, with no change in experience. > > Q: Do I need to use Magpie for any of this? > > A: No. It is plain GitHub Actions and repository configuration. > > Looking forward to your thoughts. > > J. > > -- > Drafted with Claude Code (Opus 5); reviewed by Jarek before sending. >
