I agree with the points Jon and Josh are making. In general, llms can be effectively used for triage. Testing a PR against a certain standard before any humans have to do the same. There are currently 597 PRs that go back to 2014. I wonder if, as a test, we can see how it performs when assisting with cleanup?
Patrick On Fri, Feb 20, 2026 at 1:37 PM Jon Haddad <[email protected]> wrote: > While this might not be universally true, I think it's probably a safe bet > that the overwhelming majority of AI-generated work will arrive as GitHub > PRs, not as patches attached to a JIRA. To help reduce cognitive overhead > for reviewers, we could set up a GitHub agent to perform an initial review > of all incoming PRs. I've found them to be pretty damn good at assessing > code quality, especially when given specific criteria. We could have it > analyze test results, JaCoCo reports, SpotBugs reports, etc, and give users > immediate feedback. Having that would probably be helpful in general, and > it can even be a skill we maintain in the repo itself. > > We can also have it detect new features using a few different methods and > help write the docs. I have delegated most of my writing to agents too. > We could also put this on a GitHub agent. If a user submits a PR without > documentation, the agent could suggest a documentation PR to the branch, > which the user can accept or refine. This should help raise the bar for > patches if the user intends to make an actual contribution. If they're > not, and it's just slop, they'll probably go away. > > Jon > > > > > On Fri, Feb 20, 2026 at 7:03 AM Josh McKenzie <[email protected]> > wrote: > >> If: >> >> 1. we're clear about and evolve a shared set of expectations on >> testing (unit + property + integration + cluster-distributed + harry + >> coverage (not as a target but as awareness of what is and isn't exercised >> and is human curated)) >> 2. design is thorough (public API surface area reviewed by humans + >> doesn't shotgun blast across abstraction boundaries) >> 3. CI is in a place where people can run it and make sure things work >> before merging, and >> 4. a human has reviewed every single line of code that goes in >> >> >> I don't care if a sentient trash can writes the code personally, >> copyrights and licenses respected from models used and legalities >> permitting of course. :) Which is basically just saying "If the code is >> well designed, structured, reviewed, and tested the way it's written isn't >> that important." >> >> We're not there yet on community alignment or availability on 1-3 above, >> but I think we could be if we put some effort into it this year. And I also >> argue that getting a more hygienic position on 1-3 above would make things >> better for all of us as humans on the project, LLM's or not. >> >> On Thu, Feb 19, 2026, at 6:18 PM, Štefan Miklošovič wrote: >> >> jesus ... >> >> ... even it will be explicitly forbidden to merge the code which has >> not gone through non-AI review ... >> >> On Fri, Feb 20, 2026 at 12:17 AM Štefan Miklošovič >> <[email protected]> wrote: >> > >> > .... even it will be explicitly forbidden to merge the code WITHOUT A >> > REVIEW which was done by AI .... >> > >> > On Fri, Feb 20, 2026 at 12:01 AM Štefan Miklošovič >> > <[email protected]> wrote: >> > > >> > > In the other thread it seems to me we went over positives only when it >> > > comes to usage of AI during reviews. There are also negatives of >> > > embracing AI / reviews / prompts. I think we should also look at the >> > > other side - how it might affect us negatively. I will put on a >> > > "negativist" cap for a while to see also the ugly side of this. >> > > >> > > "PR slop" - by enabling this officially, I expect we will see >> > > substantial rise in the volume of PRs of very low quality and >> > > reviewers would spend time on PRs which are just nonsensical or highly >> > > under-developed. An original author might think that any review issues >> > > will be addressed by his / her AI again when some issues arise. >> > > >> > > This also means that we might grow a generation of authors who do not >> > > code anymore the classical style and do not actually know too much >> > > about how the code works in a broader context which I think might be >> > > detrimental in the long run. >> > > >> > > Risk of a "lazy reviewer" - committer / reviewer, as busy as they are, >> > > even it will be explicitly forbidden to merge the code which was done >> > > by AI, will just review it by AI and being under a lot of stress, they >> > > might merge it anyway. I think this puts more responsibility on >> > > committers to be honest. Not saying we are not doing our best already, >> > > but my gut feeling is that once it will be officially in the repo then >> > > somebody might take some kind of a "mental shortcut" and not review >> > > the classical way. >> > > >> > > I do not trust AI, inherently. It might guide you at best. Sometimes >> > > even that is not true. I believe that by introducing these prompts / >> > > context files we will make it nonsensical less often. >> > > >> > > Anyway I think that by introducing this the bar for quality reviews >> > > will be even higher because on the other side there will be AI >> > > sitting, not a human anymore. >> >> >>
