Hi guys, Thanks Yufei for starting this discussion.
I'm +1 to try Orca as an opt-in trial (it's funny, Orca is also the name of devices I'm using https://getorca.com/ :)). I think we all agree it's a great help for catching mistakes, but the design and final merge decision remain subject to human review. I would like to remind us to stay aligned with the ASF Generative Tooling guidance (https://www.apache.org/legal/generative-tooling.html). Two parts of it apply here: - The guidance says that AI-generated text published publicly must either be reveiwed before publication by a person who take reponsability for it, or clearly state it was generated by AI (EU AI Act, in force since August 2026). A bot posting review comments directly on public PRs fails the second case. Your "clear attribution" prniciple covers this, and on PR #5332 both the review summary and each inline comment already carry the label. Let's keep it that way on every comment, as inline comments are often read on their own (notifications, emails). - If Orca proposes code (for instance GitHub suggestion blocks) and the PR author applies it, that code becomes an AI-generated contriibution. The guidance then asks the contributor to ensure that the tool's terms of use don't restrict the output in a way inconsistent with the Open Source Definition and grant enough rights to contribute it under the Apache License 2.0. Also the output must not include third-party material (or only material used under a compatible license). I think we should check and share which model/provider Orca is using, with a pointer to its terms of use. With that contributors can check without guessing. The guidance also recommends a "Generated-by: <tool> <version>" (or "Co-authored-by:") in commit message, which fits the disclosure we already encourage in our CONTRIBUTING.md ( https://github.com/apache/polaris/blob/main/CONTRIBUTING.md#guidelines-for-ai-assisted-contributions ). Once the trial gives us some feedback, I think we should update CONTRIBUTING.md with model/provider with the tool used by contributors. Regards JB On Wed, Oct 21, 2026 at 12:28 AM Yufei Gu <[email protected]> wrote: > Hi folks, > > As PR volume grows, review is becoming a bottleneck. I’d like to propose > using Orca, an AI review bot, to help identify potential issues in Polaris > PRs and reduce the load on human reviewers. Other communities(e.g., > Spark[1], Airflow[2]) are working on a similar idea. > > For Polaris, the initial Orca review results look more useful than Copilot. > Orca uses additional project context from a Polaris LLM wiki covering > architecture, conventions, and review guidance to help produce more > relevant feedback. Check Polaris PR #5532 > <https://github.com/apache/polaris/pull/5532> to see how it works. > > A few principles for the proposed workflow: > > - *Clear attribution:* Every review would explicitly state that it is > AI-generated. > - *No approvals or LGTMs:* The bot would provide comments only. It would > not approve PRs or make merge decisions. > - *A separate GitHub account:* Reviews would come from orca-review-bot, > clearly distinguishable from human reviews. > > Reviews are triggered manually today. As the next step, I'm working on > automating execution so that someone can explicitly request a review from > Orca on a PR. This would keep reviews opt-in while removing the manual work > needed to run them. > > There are two limitations to keep in mind: > > - *False positives:* Like a human reviewer, the bot can misunderstand > code or flag issues that aren't problems. Please push back on incorrect > findings. > - *False negatives:* The bot will miss issues, just as human reviewers > do. An absence of findings does not mean a PR is ready to merge. > Committers > remain responsible for the final review and merge decision. > > I'm starting an opt-in trial today. Feel free to share any feedback on > review quality, the triggering workflow, and ways to keep comment noise > manageable; this feedback would be especially helpful. > 1. https://www.mail-archive.com/dev%40spark.apache.org/msg36376.html > 2. https://www.mail-archive.com/[email protected]/msg521119.html > > Thanks! > Yufei >
