Hi Dmitri, Thanks for the feedback. I completely agree. Committers remain the shepherds of approval and are responsible for taking a PR to a mergeable status. I mentioned that the review bot does not approve PRs.
Yufei On Tue, Oct 20, 2026 at 3:41 PM Dmitri Bourlatchkov <[email protected]> wrote: > Hi Yufei, > > I'm open to trying Orca for PR reviews with the reservation that its > quality will be subject to "review" (by the Polaris community) as we > experience it in real-life situations. > > However, I do not think it will reduce the load on human reviewers. > > LLMs can help greatly with identifying logical errors and hidden mistakes. > However, I'm sure many people already use local agents for this purpose. > > Still, at this time, LLMs cannot guide the PR author on the overall Polaris > design and, more importantly, on the long term evolution that the community > is discussing / envisioning. > > Human input will still be critical because of the latter, IMHO. > > Cheers, > Dmitri. > > On Tue, Oct 20, 2026 at 6:28 PM Yufei Gu <[email protected]> wrote: > > > Hi folks, > > > > As PR volume grows, review is becoming a bottleneck. I’d like to propose > > using Orca, an AI review bot, to help identify potential issues in > Polaris > > PRs and reduce the load on human reviewers. Other communities(e.g., > > Spark[1], Airflow[2]) are working on a similar idea. > > > > For Polaris, the initial Orca review results look more useful than > Copilot. > > Orca uses additional project context from a Polaris LLM wiki covering > > architecture, conventions, and review guidance to help produce more > > relevant feedback. Check Polaris PR #5532 > > <https://github.com/apache/polaris/pull/5532> to see how it works. > > > > A few principles for the proposed workflow: > > > > - *Clear attribution:* Every review would explicitly state that it is > > AI-generated. > > - *No approvals or LGTMs:* The bot would provide comments only. It > would > > not approve PRs or make merge decisions. > > - *A separate GitHub account:* Reviews would come from > orca-review-bot, > > clearly distinguishable from human reviews. > > > > Reviews are triggered manually today. As the next step, I'm working on > > automating execution so that someone can explicitly request a review from > > Orca on a PR. This would keep reviews opt-in while removing the manual > work > > needed to run them. > > > > There are two limitations to keep in mind: > > > > - *False positives:* Like a human reviewer, the bot can misunderstand > > code or flag issues that aren't problems. Please push back on > incorrect > > findings. > > - *False negatives:* The bot will miss issues, just as human reviewers > > do. An absence of findings does not mean a PR is ready to merge. > > Committers > > remain responsible for the final review and merge decision. > > > > I'm starting an opt-in trial today. Feel free to share any feedback on > > review quality, the triggering workflow, and ways to keep comment noise > > manageable; this feedback would be especially helpful. > > 1. https://www.mail-archive.com/dev%40spark.apache.org/msg36376.html > > 2. > https://www.mail-archive.com/[email protected]/msg521119.html > > > > Thanks! > > Yufei > > >
