Ok .. I managed to enable the bots. Please try `ai-review-requested` label out (only commiters can do). and please let me know if there are anything suspicious or not working well 👍
On Tue, 20 Oct 2026 at 08:31, Dongjoon Hyun <[email protected]> wrote: > Thank you so much for keeping working and sharing the status, Hyukjin. > > Dongjoon. > > On 2026/10/19 21:50:36 Hyukjin Kwon wrote: > > update: I am still struggling / testing. `ai-review-requested` label does > > not work for now. > > > > On Fri, 16 Oct 2026 at 07:05, Hyukjin Kwon <[email protected]> wrote: > > > > > headsup. I will stop the bot today. Will likely introduce > > > `ai-review-requested` label (only committers can set) to trigger the > review > > > most likely today for testing it out, see > > > https://github.com/apache/spark/pull/59502 > > > > > > On Mon, 12 Oct 2026 at 08:14, Hyukjin Kwon <[email protected]> > wrote: > > > > > >> Another quick heads up. > > >> > > >> - I am basically stopping (or rather not posting) the reviews when > > >> there are no findings. > > >> - If there are updates from the author, the review will be > triggered. > > >> To minimize the cost, I am not setting the effort as max but > rather trigger > > >> multiple times whenever the author pushes a newer commit. > > >> - I am looking especially through P3 review comments or not. Please > > >> let me know if you guys are getting too many P3 nits, and if that > bugs you. > > >> > > >> > > >> > > >> On Thu, 8 Oct 2026 at 10:54, Hyukjin Kwon <[email protected]> > wrote: > > >> > > >>> Just quick headup, I am having some trouble with automation right > now. I > > >>> would need to test/run manually so it would be triggered like > everyday for > > >>> now. > > >>> > > >>> On Sat, 3 Oct 2026 at 01:53, Dongjoon Hyun <[email protected]> > wrote: > > >>> > > >>>> +1 > > >>>> > > >>>> Thanks, Hyukjin. > > >>>> > > >>>> Could we introduce a few safeguards for the new review bot? > > >>>> > > >>>> Specifically: > > >>>> - A GitHub label to opt out or disable the bot per PR. > > >>>> - Protections against excessive ASF Jira API calls. > > >>>> > > >>>> This would prevent the bot from being blocked/banned by ASF Jira > > >>>> and allow reviews to proceed even if Jira is down. It also helps > > >>>> manage worst-case scenarios where the bot leaves repetitive or > > >>>> unhelpful nitpicks. > > >>>> > > >>>> Best, > > >>>> Dongjoon > > >>>> > > >>>> On 2026-10-01 21:58 UTC Hyukjin Kwon wrote: > > >>>> > Hi all, > > >>>> > > > >>>> > I've been running a trial of automated AI code review on new > > >>>> apache/spark > > >>>> > PRs, and I'd like to share how it works and get your feedback. > > >>>> > > > >>>> > For now the reviews are posted from my own GitHub account > > >>>> (@HyukjinKwon) > > >>>> > while I check that it behaves well. Once it's proven, I plan to > move > > >>>> it to > > >>>> > the https://github.com/spark-test account, so the reviews are > clearly > > >>>> > separate from my personal ones. I plan to run this for the next > week. > > >>>> > > > >>>> > *What it does* > > >>>> > > > >>>> > Each review is a full, multi-pass review of the PR diff, run by > Claude > > >>>> > (via Claude Code). Findings are checked against the code before > > >>>> they're > > >>>> > posted. They're posted as a normal GitHub review: a short summary > plus > > >>>> > inline comments, each labeled with a severity, e.g. "Non-blocking > > >>>> (P2)" > > >>>> > or "Nit (P3)". Example: > > >>>> > > > >>>> > https://github.com/apache/spark/pull/59184#pullrequestreview-5379759787 > > >>>> > > > >>>> > *No LGTM, no approvals* > > >>>> > > > >>>> > - It never approves a PR and never posts "LGTM". Every review is a > > >>>> plain > > >>>> > "Comment" review, so it doesn't count as an approval or a change > > >>>> > request, and it has no effect on merging. > > >>>> > - If a review finds nothing new, it posts nothing at all: no "no > > >>>> > findings" or "all comments addressed" message. > > >>>> > - It doesn't push commits or edit anyone's PR. > > >>>> > - Human review and committer judgement are unchanged. Please > treat its > > >>>> > comments like any other reviewer's: feel free to push back or > > >>>> ignore. > > >>>> > > > >>>> > *When a review is triggered* > > >>>> > > > >>>> > - A new PR is reviewed shortly after it's opened. Draft PRs are > > >>>> reviewed > > >>>> > once they're marked ready for review. > > >>>> > - After a review posts findings, the bot waits. It reviews again > only > > >>>> > when the author pushes new commits. > > >>>> > - It also waits while a PR has open review comments from anyone > that > > >>>> are > > >>>> > newer than the latest commit, so it doesn't pile on while the > > >>>> author is > > >>>> > still responding to reviewers. > > >>>> > - Replying in a thread without pushing doesn't trigger a > re-review. > > >>>> > > > >>>> > *When it stops* > > >>>> > > > >>>> > - When a review finds nothing new. The PR gets a fresh review if > the > > >>>> > author pushes again later. > > >>>> > - When the PR is merged or closed. > > >>>> > > > >>>> > *Early numbers* > > >>>> > > > >>>> > Over the first ~12 hours (Oct 1), it covered 10 PRs, including 5 > > >>>> > re-reviews after the author pushed. It posted 7 reviews with 14 > > >>>> findings > > >>>> > in total, mostly P3 nits. Four PRs came back clean and got no > comment. > > >>>> > Several authors pushed fixes, and the follow-up reviews found > nothing > > >>>> > new. > > >>>> > > > >>>> > *Feedback* > > >>>> > > > >>>> > Please let me know if the comments are useful, noisy, or wrong, > either > > >>>> > here or on the PR. If you'd rather it not review your PRs, tell > me and > > >>>> > I'll exclude them. I'm also happy to adjust when it triggers > based on > > >>>> > what the community prefers. > > >>>> > > > >>>> > Thanks, > > >>>> > Hyukjin > > >>>> > > > >>>> > > >>>> > --------------------------------------------------------------------- > > >>>> To unsubscribe e-mail: [email protected] > > >>>> > > >>>> > > > > --------------------------------------------------------------------- > To unsubscribe e-mail: [email protected] > >
