Hi all, Thanks Yufei for starting the trial, and Dmitri, JB and Robert for the points so far. Sharing some feedback from real usage.
I went through the Orca review on PR #5717 [1]. A few comments were valid on their own, like the Kafka mapper duplication, "type" becoming the contract, and the "data.source" vs CE "source" naming. What it missed was the context around the PR: the dev list discussion, the sequencing with #5084 [2], and that this PR is meant to be mapper-only. In Polaris a lot of design direction is agreed on the list and in syncs, and the bot can't see that yet. *My concern is newer or part-time contributors treating bot comments asmust-do and growing the PR scope. Reviewers would then be reviewingboth the diff and the bot thread.* A suggestion for the trial: treat Orca comments like any other nit. The author can reply, but only changes code once a human reviewer agrees the finding is valid. It may also help to link related dev threads or issues in the PR description, so the bot (and humans) get that context.. Robert's idea of comparing it with Copilot on a few recent PRs sounds like a good way to judge it. Happy to share more as I see it on other PRs. Thanks, Vignesh [1] https://github.com/apache/polaris/pull/5717 [2] https://github.com/apache/polaris/pull/5084
