How many of these PRs are for new contributors where we need to approve running CI already?
> On 18 Sep 2026, at 15:24, Jarek Potiuk <[email protected]> wrote: > > Hi all, > > I would like to propose a change to how CI runs for external > contributions—specifically, > hope that it will help us deal with maintainer overload caused by too many PRs > that need our attention. > > Basically the idea is to switch where CI runs for non-approved > external contributors. > > Since this affects the contribution experience for everyone who is not > a committer or collaborator, I want to discuss it thoroughly before > moving forward. > > I have written this up as AIP-120 on the wiki: > https://cwiki.apache.org/confluence/spaces/AIRFLOW/pages/451974711/AIP-120+Run+CI+for+external+contributions+in+contributor+forks > > Below is a summary of the data and arguments. > > The Problem (Data from the last 90 days) > > - 4,009 PRs opened (~45/day). > - 65% came from external contributors. > - External PRs close without merging at 19.5%, compared to 5.2% for > committers and collaborators (~3.8x higher). > - That accounts for 508 external PRs in 90 days (~5.6/day) that > consumed CI resources and reviewer attention without producing any > code changes. > > CI capacity is limited and costly for the foundation. A full matrix > run requires 4,392 job-minutes across 308 jobs, whereas a docs-only > change takes 31 job-minutes (a 60x difference). > > The Proposal > > 1. External CI in Forks: External contributions run their CI within > the contributor's own fork. Project CI is reserved for maintainers and > approved organizational stakeholders. > > 2. Automated Flow: > - Workflows run in the contributor's fork on push. > - Runs originating from non-approved forks are stopped in apache/airflow. > - The PR is automatically set to draft status, labeled, and > accompanied by an explanatory comment for the author. > - A monitor undrafts the PR once the fork CI turns green. > - PRs with no CI activity after a specified period are > automatically closed with an explanation (and can be reopened later). > > 3. Fully Automated: The entire process is deterministic, requires no > manual maintainer action, and adds no administrative overhead. > > Note: Apache SeaTunnel already uses a similar workflow, and several > ASF projects do not run PR CI at all. > > Correction Regarding GitHub Actions Costs > > I previously suggested this model would work because contributors > would spend their own free-tier Actions minutes, creating a financial > rate-limit. This was incorrect. > > GitHub Actions is free and unmetered for public repositories; the > 2,000-minute monthly cap applies only to private repos. Because a fork > of apache/airflow is public, running CI costs the contributor nothing. > > The actual limiting factor is concurrency: > > - Free GitHub accounts are capped at 20 concurrent jobs, whereas our > project runs on enterprise-level concurrency. > - Docs changes: ~8 minutes in either setup (no difference). > - Large core changes: ~88 minutes in a fork vs. ~35 minutes on > project infrastructure (repeated on every push). > > While this creates a time-based gradient rather than a financial one, > asking contributors to spend time verifying their own work is a > reasonable expectation. It discourages iterating on large, unverified > changes directly against project resources while leaving small or > documentation contributions largely unaffected. > > Addressing Objections > > - "This won't reduce total PRs opened": Correct. The goal is to > reduce unready PRs that require maintainer attention. A PR that is > automatically drafted, fails to go green, and closes automatically > consumes zero reviewer time. > - Alternative: Gating PRs on triaged issues: While this has merit in > separating feature approval from implementation review, requiring > maintainers to approve issues first shifts the burden onto our already > large issue backlog rather than eliminating it. (These two approaches > are also not mutually exclusive.) > > Additional Question: Existing Backlog > > I would also like to discuss whether we should close currently open > external PRs when switching to this model, asking authors to reopen > them under the new workflow to establish a clean baseline. > > I view this as a separable proposal. Mass-closing contributor work > carries a risk of negative feedback, PR reopen surges, or abandoning > valid work. If we choose to do this, messaging must clearly explain > that CI capacity is foundation-funded and finite, rather than implying > GitHub Actions costs money for the contributor. > > Discussion Points > > I am seeking feedback on the general proposal, as well as specific input on: > > 1. Is a time-based concurrency gate effective enough to pursue? > 2. Should we reset or mass-close the existing external PR backlog? > 3. How long should a PR sit without CI activity before being > automatically closed? > 4. What criteria should qualify an organization for a CI exemption? > > Best regards, > Jarek > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
