Hi folks, As PR volume grows, review is becoming a bottleneck. I’d like to propose using Orca, an AI review bot, to help identify potential issues in Polaris PRs and reduce the load on human reviewers. Other communities(e.g., Spark[1], Airflow[2]) are working on a similar idea.
For Polaris, the initial Orca review results look more useful than Copilot. Orca uses additional project context from a Polaris LLM wiki covering architecture, conventions, and review guidance to help produce more relevant feedback. Check Polaris PR #5532 <https://github.com/apache/polaris/pull/5532> to see how it works. A few principles for the proposed workflow: - *Clear attribution:* Every review would explicitly state that it is AI-generated. - *No approvals or LGTMs:* The bot would provide comments only. It would not approve PRs or make merge decisions. - *A separate GitHub account:* Reviews would come from orca-review-bot, clearly distinguishable from human reviews. Reviews are triggered manually today. As the next step, I'm working on automating execution so that someone can explicitly request a review from Orca on a PR. This would keep reviews opt-in while removing the manual work needed to run them. There are two limitations to keep in mind: - *False positives:* Like a human reviewer, the bot can misunderstand code or flag issues that aren't problems. Please push back on incorrect findings. - *False negatives:* The bot will miss issues, just as human reviewers do. An absence of findings does not mean a PR is ready to merge. Committers remain responsible for the final review and merge decision. I'm starting an opt-in trial today. Feel free to share any feedback on review quality, the triggering workflow, and ways to keep comment noise manageable; this feedback would be especially helpful. 1. https://www.mail-archive.com/dev%40spark.apache.org/msg36376.html 2. https://www.mail-archive.com/[email protected]/msg521119.html Thanks! Yufei
