Thanks for bringing this up! Overall, I like this idea, but it's worth testing it for a bit before we enforce it, especially the LLM-verify part.
> * The author ensures the PR passes ALL the checks and tests (i.e. green). > It might sometimes mean we have to - even more quickly to `main` breakages, > and probably provide some "status" info and exceptions when we know main is > broken. Probably, we should exempt some checks that might be flaky? > * All PRs that do not meet this requirement will be converted to Drafts > with automated suggestions (reviewed quickly and efficiently by a > triager) provided to the author on the next steps. This will be super helpful! I also do it manually from time to time. Best, Wei > On Mar 3, 2026, at 9:34 PM, Jarek Potiuk <[email protected]> wrote: > > *TL;DR; I propose a stricter (automation-assisted) approach for the "ready > for review" state and clearer expectations for contributors regarding when > maintainers review PRs of non-collaborators.* > > Following the > https://lists.apache.org/thread/8tzwwwd7jmtmfo4j9pzg27704g10vpr4 where I > showcased a tool that I claude-coded, I would like to have a (possibly > short) discussion on this subject and reach a stage where I can attempt to > try the tool out. > > *Why? * > > Because we maintainers are overwhelmed and burning out, we no longer see > how our time invested in Airflow can bring significant returns to us > (personally) and the community. > > While some of us spend a lot of time reviewing, commenting on, and merging > code, with the current rate of AI-generated PRs and other things we do, > this is not sustainable. Also there is a mismatch—or lack of > clarity—regarding the quality expectations for the PRs we want to review. > > *Social Contract Issue* > > We are a good (I think) open source project with a thriving community and a > great group of maintainers who are also friends and like to work with each > other but also are very open to bringing new community members in. As > maintainers, we are willing to help new contributors grow and generally > willing to spend some of our time doing so. This is the social contract we > signed up for as OSS maintainers and as committers for the Apache Software > Foundation PMC. Community Over Code. > > However, this social contract - this community-building aspect is currently > heavily imbalanced because AI-generated content takes away time, focus and > energy from the maintainers. Instead of having meaningful discussions in > PRs about whether changes are needed and communicating with people, we > start losing time talking to - effectively - AI agents about hundreds of > smaller and bigger things that should not be there in a first place. > > Currently - collaboration and community building suffer. Even if real > people submit code generated by agents (which is becoming really good, fast > and cheap to produce), we simply lack the time as maintainers to have > meaningful conversations with the people behind those agents. > > Sometimes we lose time talking to agents. Sometimes we lose time on talking > to people who have 0 understanding of what they are doing and submitt > continuous crap, and we should not be having that conversation at > all. Sometimes, we just look at the number of PRs opened in a given day in > despair, dreading even trying to bring order to them. > > And many of us also have some "work" to do or a "feature" to work on top of > that. > > I think we need to reclaim the maintainers' collective time to focus on > what matters: delegating more responsibility to authors so they meet our > expected quality bar (and efficiently verifying it with tools without > losing time and focus). > > *What do we have now?* > > We have already done a lot to help with it - AGENTS.The PR guidelines, > overhauled by Kaxil and updated by others, will certainly help clarify > expectations for agents in the future. I know Kaxil is also exploring a way > to enable automated copilot code reviews in a manner that will not be too > "dehumanizing" and will work well. This is all good. The better the agents > people use and the more closely they follow those instructions, the higher > the quality of incoming PRs will be. But we also need to help maintainers > easily identify what to focus on—distinguishing work in progress and > unfinished PRs that need work from those truly "Ready for (human) review." > > *How?* > > My proposal has two parts: > > * Define and communicate expectations for PRs that maintainers can manage. > * Relentlessly automate it to ensure expectations are met and that > maintainers can easily focus on those PRs that "Ready for review." > > My tool (needs a bit more fine-tuning and refinement): > https://github.com/apache/airflow/pull/62682 `*breeze pr auto-triage*` is > designed to do exactly this: automate those expectations by auto-triaging > the PRs. It not only converts them to Draft when they are not yet "Ready > For Review," but also provides actionable, automated (deterministic + LLM) > comments to the authors. A concrete maintainer (the current triager) is > using the tool very efficiently. > > *Proposed expectations (for non-collaborators):* > > Those are not "new" expectations. Really, I'm proposing we completely > delegate the responsibility for fulfilling those expectations to the author > (with helpful, automated comments - reviewed and confirmed by a human > triager for now). And simply be very clear that generally no maintainer > will look at a PR until: > > * The author ensures the PR passes ALL the checks and tests (i.e. green). > It might sometimes mean we have to - even more quickly to `main` breakages, > and probably provide some "status" info and exceptions when we know main is > broken. > > * The author follows all PR guidelines (LLM-verified) regarding > description, content, quality, and presence of tests. > > * All PRs that do not meet this requirement will be converted to Drafts > with automated suggestions (reviewed quickly and efficiently by a > triager) provided to the author on the next steps. > > * Drafts with no activity will be more aggressively pruned by our stalebot. > > The triager is there mostly to quickly assess and generate comments—with > tool/AI assistance. The triager won't be the one who actually reviews those > PRs when they are "ready for review." > > * Only after that do we mark the PR as "*ready for maintainer review*" > (label) > > * Only such PRs should be reviewed and it is entirely up to the author to > make them ready. > > Note: This approach is only for non-collaborators. For collaborators: we > might have just one expectation - mark your PR with "ready for maintainer > review" when you think it's ready. > We accept people as committers and collaborators because we already know > they generally know and follow the rules; automating this step isn't > necessary. > > This is nothing new; we've already been doing this with humans handling all > the heavy lifting without much of strictness or organization, but this is > no longer sustainable. > > I propose we make the expectations explicit, communicate them clearly, and > relentlessly automate their execution. > > I would love to hear what y'all think. > > J. --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
