Just a comment: while many of us are busy with Airflow 3 - so we should not distract those who are—maybe it's a good idea to aim to work out an approach for handling incoming PRs for a cleaner start to 3.4?
There were a few ideas here and in other places. I am happy to review them and create a shared doc where we can discuss them in the meantime. Perhaps together we can - together - come up with some consensus on how to progress. I think the tension around releasing Airflow hinders discussions and reaching consensus, but possibly we could still get there. Broadly speaking I think the following approach is the next step: * Make a distinction between areas of contribution, regarding our "expectations". For example rules for the scheduler might differ from those for most providers, but the "standard" or "common" set of providers will follow yet another set of rules. * Limit the number of PRs a non-collaborator can have open; there is a feature released in GitHub last week we can use for it. * Limit the **SIZE** of a PR a non-collaborator can open - for example, <10 lines (excluding tests) of "core" contributions will be okay; anything bigger will be rejected by CI * Limit the overall number of collaborator PRs that can be opened "per area" - this could be rejected by CI * For all defined areas, we **must** have maintainers who commit to reviewing those area PRs relatively quickly - this is very important * Write it down as a "Social Contract" we will follow - very important as well I think this approach has a number of benefits: * not closing for external contributions * but limiting them so that maintainers can find time for them * Teaching new contributors that small PRs are way better than bigger If this direction seems good, I'd be happy to run simulations on the currently "Ready for maintainer review" PRs to see the effect on our "incoming PRs" state. The outcome will naturally be different - the fact that we will start measuring it, will change the output - but I think it might be a good indication of the volume - per area - that will need a maintainer taking a look. We have all tooling in place to first experiment with it (Magpie) and then turn it into CI (Magpie rules -> Deterministic CI) is going to be a first-class citizen in Magpie. J. On Wed, Jun 24, 2026 at 5:26 PM Sameer Mesiah <[email protected]> wrote: > That is actually a fairly good heuristic. To expand on that idea, I think > we should exclude PRs that have been explicitly labelled as AI spam. I have > done a few reviews where I have written dozens of comments only to realize > that the author is just using an Agent to spam commits. And then the PR > gets closed without any of the comments being incorporated. We can also > account for what I call 'non-blocking' suggestions i.e. it's the author's > choice to implement. Does anybody else here have any ideas on how we > further build on this idea? Slop reviews are certainly a problem (though > not as endemic as Slop PRs). In some ways, they are worse, because you feel > gaslit but at the same time, you want feedback because you might be unsure > of something. > > On Tue, 23 Jun 2026 at 02:30, Ghaeli, Sean <[email protected]> wrote: > > > A way to prevent slop reviews is to weight an author's review score by > the > > extent to which it became incorporated in the implementation. A review > that > > results in no changes is given no weight. > > > > >
