headsup. I will stop the bot today. Will likely introduce `ai-review-requested` label (only committers can set) to trigger the review most likely today for testing it out, see https://github.com/apache/spark/pull/59502
On Mon, 12 Oct 2026 at 08:14, Hyukjin Kwon <[email protected]> wrote: > Another quick heads up. > > - I am basically stopping (or rather not posting) the reviews when > there are no findings. > - If there are updates from the author, the review will be triggered. > To minimize the cost, I am not setting the effort as max but rather trigger > multiple times whenever the author pushes a newer commit. > - I am looking especially through P3 review comments or not. Please > let me know if you guys are getting too many P3 nits, and if that bugs you. > > > > On Thu, 8 Oct 2026 at 10:54, Hyukjin Kwon <[email protected]> wrote: > >> Just quick headup, I am having some trouble with automation right now. I >> would need to test/run manually so it would be triggered like everyday for >> now. >> >> On Sat, 3 Oct 2026 at 01:53, Dongjoon Hyun <[email protected]> wrote: >> >>> +1 >>> >>> Thanks, Hyukjin. >>> >>> Could we introduce a few safeguards for the new review bot? >>> >>> Specifically: >>> - A GitHub label to opt out or disable the bot per PR. >>> - Protections against excessive ASF Jira API calls. >>> >>> This would prevent the bot from being blocked/banned by ASF Jira >>> and allow reviews to proceed even if Jira is down. It also helps >>> manage worst-case scenarios where the bot leaves repetitive or >>> unhelpful nitpicks. >>> >>> Best, >>> Dongjoon >>> >>> On 2026-10-01 21:58 UTC Hyukjin Kwon wrote: >>> > Hi all, >>> > >>> > I've been running a trial of automated AI code review on new >>> apache/spark >>> > PRs, and I'd like to share how it works and get your feedback. >>> > >>> > For now the reviews are posted from my own GitHub account >>> (@HyukjinKwon) >>> > while I check that it behaves well. Once it's proven, I plan to move >>> it to >>> > the https://github.com/spark-test account, so the reviews are clearly >>> > separate from my personal ones. I plan to run this for the next week. >>> > >>> > *What it does* >>> > >>> > Each review is a full, multi-pass review of the PR diff, run by Claude >>> > (via Claude Code). Findings are checked against the code before they're >>> > posted. They're posted as a normal GitHub review: a short summary plus >>> > inline comments, each labeled with a severity, e.g. "Non-blocking (P2)" >>> > or "Nit (P3)". Example: >>> > >>> https://github.com/apache/spark/pull/59184#pullrequestreview-5379759787 >>> > >>> > *No LGTM, no approvals* >>> > >>> > - It never approves a PR and never posts "LGTM". Every review is a >>> plain >>> > "Comment" review, so it doesn't count as an approval or a change >>> > request, and it has no effect on merging. >>> > - If a review finds nothing new, it posts nothing at all: no "no >>> > findings" or "all comments addressed" message. >>> > - It doesn't push commits or edit anyone's PR. >>> > - Human review and committer judgement are unchanged. Please treat its >>> > comments like any other reviewer's: feel free to push back or ignore. >>> > >>> > *When a review is triggered* >>> > >>> > - A new PR is reviewed shortly after it's opened. Draft PRs are >>> reviewed >>> > once they're marked ready for review. >>> > - After a review posts findings, the bot waits. It reviews again only >>> > when the author pushes new commits. >>> > - It also waits while a PR has open review comments from anyone that >>> are >>> > newer than the latest commit, so it doesn't pile on while the author >>> is >>> > still responding to reviewers. >>> > - Replying in a thread without pushing doesn't trigger a re-review. >>> > >>> > *When it stops* >>> > >>> > - When a review finds nothing new. The PR gets a fresh review if the >>> > author pushes again later. >>> > - When the PR is merged or closed. >>> > >>> > *Early numbers* >>> > >>> > Over the first ~12 hours (Oct 1), it covered 10 PRs, including 5 >>> > re-reviews after the author pushed. It posted 7 reviews with 14 >>> findings >>> > in total, mostly P3 nits. Four PRs came back clean and got no comment. >>> > Several authors pushed fixes, and the follow-up reviews found nothing >>> > new. >>> > >>> > *Feedback* >>> > >>> > Please let me know if the comments are useful, noisy, or wrong, either >>> > here or on the PR. If you'd rather it not review your PRs, tell me and >>> > I'll exclude them. I'm also happy to adjust when it triggers based on >>> > what the community prefers. >>> > >>> > Thanks, >>> > Hyukjin >>> > >>> >>> --------------------------------------------------------------------- >>> To unsubscribe e-mail: [email protected] >>> >>>
