Seems like this is working quite fine although sometimes it fails (and says to try again). Let me increase the parallelism to 8. So it reviews 8 at a time.
On Wed, 4 Nov 2026 at 08:47, Holden Karau <[email protected]> wrote: > Very cool :) > > On Tue, Nov 3, 2026 at 2:51 PM Hyukjin Kwon via dev <[email protected]> > wrote: > >> Just FYI. Now, the bot is reviewing 4 at a time. The review takes around >> 1 hour. So sometimes it takes time. I am still monitoring. >> >> On Fri, 23 Oct 2026 at 07:09, Hyukjin Kwon <[email protected]> wrote: >> >>> FYI, I am monitoring and controlling details now. Effort, timeout, >>> security stuff. >>> >>> On Thu, 22 Oct 2026 at 22:51, Dongjoon Hyun <[email protected]> wrote: >>> >>>> Got it. Thank you! >>>> >>>> Dongjoon. >>>> >>>> On 2026/10/22 11:44:04 Hyukjin Kwon wrote: >>>> > Just to reduce any surprise, I set another bot to monitor the bot >>>> itself. >>>> > If it does anything wrong, it halts immediately, and removes/reverts >>>> > whatever was wrong. >>>> > >>>> > For committers, please directly remove the `ai-review-requested` >>>> label if >>>> > there seems anything wrong. That would make the bot stop too. >>>> > >>>> > On Thu, 22 Oct 2026 at 20:21, Hyukjin Kwon <[email protected]> >>>> wrote: >>>> > >>>> > > Ok .. I managed to enable the bots. Please try >>>> `ai-review-requested` label >>>> > > out (only commiters can do). and please let me know if there are >>>> anything >>>> > > suspicious or not working well 👍 >>>> > > >>>> > > On Tue, 20 Oct 2026 at 08:31, Dongjoon Hyun <[email protected]> >>>> wrote: >>>> > > >>>> > >> Thank you so much for keeping working and sharing the status, >>>> Hyukjin. >>>> > >> >>>> > >> Dongjoon. >>>> > >> >>>> > >> On 2026/10/19 21:50:36 Hyukjin Kwon wrote: >>>> > >> > update: I am still struggling / testing. `ai-review-requested` >>>> label >>>> > >> does >>>> > >> > not work for now. >>>> > >> > >>>> > >> > On Fri, 16 Oct 2026 at 07:05, Hyukjin Kwon <[email protected] >>>> > >>>> > >> wrote: >>>> > >> > >>>> > >> > > headsup. I will stop the bot today. Will likely introduce >>>> > >> > > `ai-review-requested` label (only committers can set) to >>>> trigger the >>>> > >> review >>>> > >> > > most likely today for testing it out, see >>>> > >> > > https://github.com/apache/spark/pull/59502 >>>> > >> > > >>>> > >> > > On Mon, 12 Oct 2026 at 08:14, Hyukjin Kwon < >>>> [email protected]> >>>> > >> wrote: >>>> > >> > > >>>> > >> > >> Another quick heads up. >>>> > >> > >> >>>> > >> > >> - I am basically stopping (or rather not posting) the >>>> reviews when >>>> > >> > >> there are no findings. >>>> > >> > >> - If there are updates from the author, the review will be >>>> > >> triggered. >>>> > >> > >> To minimize the cost, I am not setting the effort as max >>>> but >>>> > >> rather trigger >>>> > >> > >> multiple times whenever the author pushes a newer commit. >>>> > >> > >> - I am looking especially through P3 review comments or >>>> not. >>>> > >> Please >>>> > >> > >> let me know if you guys are getting too many P3 nits, and >>>> if that >>>> > >> bugs you. >>>> > >> > >> >>>> > >> > >> >>>> > >> > >> >>>> > >> > >> On Thu, 8 Oct 2026 at 10:54, Hyukjin Kwon < >>>> [email protected]> >>>> > >> wrote: >>>> > >> > >> >>>> > >> > >>> Just quick headup, I am having some trouble with automation >>>> right >>>> > >> now. I >>>> > >> > >>> would need to test/run manually so it would be triggered like >>>> > >> everyday for >>>> > >> > >>> now. >>>> > >> > >>> >>>> > >> > >>> On Sat, 3 Oct 2026 at 01:53, Dongjoon Hyun < >>>> [email protected]> >>>> > >> wrote: >>>> > >> > >>> >>>> > >> > >>>> +1 >>>> > >> > >>>> >>>> > >> > >>>> Thanks, Hyukjin. >>>> > >> > >>>> >>>> > >> > >>>> Could we introduce a few safeguards for the new review bot? >>>> > >> > >>>> >>>> > >> > >>>> Specifically: >>>> > >> > >>>> - A GitHub label to opt out or disable the bot per PR. >>>> > >> > >>>> - Protections against excessive ASF Jira API calls. >>>> > >> > >>>> >>>> > >> > >>>> This would prevent the bot from being blocked/banned by ASF >>>> Jira >>>> > >> > >>>> and allow reviews to proceed even if Jira is down. It also >>>> helps >>>> > >> > >>>> manage worst-case scenarios where the bot leaves repetitive >>>> or >>>> > >> > >>>> unhelpful nitpicks. >>>> > >> > >>>> >>>> > >> > >>>> Best, >>>> > >> > >>>> Dongjoon >>>> > >> > >>>> >>>> > >> > >>>> On 2026-10-01 21:58 UTC Hyukjin Kwon wrote: >>>> > >> > >>>> > Hi all, >>>> > >> > >>>> > >>>> > >> > >>>> > I've been running a trial of automated AI code review on >>>> new >>>> > >> > >>>> apache/spark >>>> > >> > >>>> > PRs, and I'd like to share how it works and get your >>>> feedback. >>>> > >> > >>>> > >>>> > >> > >>>> > For now the reviews are posted from my own GitHub account >>>> > >> > >>>> (@HyukjinKwon) >>>> > >> > >>>> > while I check that it behaves well. Once it's proven, I >>>> plan to >>>> > >> move >>>> > >> > >>>> it to >>>> > >> > >>>> > the https://github.com/spark-test account, so the >>>> reviews are >>>> > >> clearly >>>> > >> > >>>> > separate from my personal ones. I plan to run this for >>>> the next >>>> > >> week. >>>> > >> > >>>> > >>>> > >> > >>>> > *What it does* >>>> > >> > >>>> > >>>> > >> > >>>> > Each review is a full, multi-pass review of the PR diff, >>>> run by >>>> > >> Claude >>>> > >> > >>>> > (via Claude Code). Findings are checked against the code >>>> before >>>> > >> > >>>> they're >>>> > >> > >>>> > posted. They're posted as a normal GitHub review: a short >>>> > >> summary plus >>>> > >> > >>>> > inline comments, each labeled with a severity, e.g. >>>> "Non-blocking >>>> > >> > >>>> (P2)" >>>> > >> > >>>> > or "Nit (P3)". Example: >>>> > >> > >>>> > >>>> > >> > >>>> >>>> > >> >>>> https://github.com/apache/spark/pull/59184#pullrequestreview-5379759787 >>>> > >> > >>>> > >>>> > >> > >>>> > *No LGTM, no approvals* >>>> > >> > >>>> > >>>> > >> > >>>> > - It never approves a PR and never posts "LGTM". Every >>>> review is >>>> > >> a >>>> > >> > >>>> plain >>>> > >> > >>>> > "Comment" review, so it doesn't count as an approval or >>>> a >>>> > >> change >>>> > >> > >>>> > request, and it has no effect on merging. >>>> > >> > >>>> > - If a review finds nothing new, it posts nothing at all: >>>> no "no >>>> > >> > >>>> > findings" or "all comments addressed" message. >>>> > >> > >>>> > - It doesn't push commits or edit anyone's PR. >>>> > >> > >>>> > - Human review and committer judgement are unchanged. >>>> Please >>>> > >> treat its >>>> > >> > >>>> > comments like any other reviewer's: feel free to push >>>> back or >>>> > >> > >>>> ignore. >>>> > >> > >>>> > >>>> > >> > >>>> > *When a review is triggered* >>>> > >> > >>>> > >>>> > >> > >>>> > - A new PR is reviewed shortly after it's opened. Draft >>>> PRs are >>>> > >> > >>>> reviewed >>>> > >> > >>>> > once they're marked ready for review. >>>> > >> > >>>> > - After a review posts findings, the bot waits. It >>>> reviews again >>>> > >> only >>>> > >> > >>>> > when the author pushes new commits. >>>> > >> > >>>> > - It also waits while a PR has open review comments from >>>> anyone >>>> > >> that >>>> > >> > >>>> are >>>> > >> > >>>> > newer than the latest commit, so it doesn't pile on >>>> while the >>>> > >> > >>>> author is >>>> > >> > >>>> > still responding to reviewers. >>>> > >> > >>>> > - Replying in a thread without pushing doesn't trigger a >>>> > >> re-review. >>>> > >> > >>>> > >>>> > >> > >>>> > *When it stops* >>>> > >> > >>>> > >>>> > >> > >>>> > - When a review finds nothing new. The PR gets a fresh >>>> review if >>>> > >> the >>>> > >> > >>>> > author pushes again later. >>>> > >> > >>>> > - When the PR is merged or closed. >>>> > >> > >>>> > >>>> > >> > >>>> > *Early numbers* >>>> > >> > >>>> > >>>> > >> > >>>> > Over the first ~12 hours (Oct 1), it covered 10 PRs, >>>> including 5 >>>> > >> > >>>> > re-reviews after the author pushed. It posted 7 reviews >>>> with 14 >>>> > >> > >>>> findings >>>> > >> > >>>> > in total, mostly P3 nits. Four PRs came back clean and >>>> got no >>>> > >> comment. >>>> > >> > >>>> > Several authors pushed fixes, and the follow-up reviews >>>> found >>>> > >> nothing >>>> > >> > >>>> > new. >>>> > >> > >>>> > >>>> > >> > >>>> > *Feedback* >>>> > >> > >>>> > >>>> > >> > >>>> > Please let me know if the comments are useful, noisy, or >>>> wrong, >>>> > >> either >>>> > >> > >>>> > here or on the PR. If you'd rather it not review your >>>> PRs, tell >>>> > >> me and >>>> > >> > >>>> > I'll exclude them. I'm also happy to adjust when it >>>> triggers >>>> > >> based on >>>> > >> > >>>> > what the community prefers. >>>> > >> > >>>> > >>>> > >> > >>>> > Thanks, >>>> > >> > >>>> > Hyukjin >>>> > >> > >>>> > >>>> > >> > >>>> >>>> > >> > >>>> >>>> > >> >>>> --------------------------------------------------------------------- >>>> > >> > >>>> To unsubscribe e-mail: [email protected] >>>> > >> > >>>> >>>> > >> > >>>> >>>> > >> > >>>> > >> >>>> > >> >>>> --------------------------------------------------------------------- >>>> > >> To unsubscribe e-mail: [email protected] >>>> > >> >>>> > >> >>>> > >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe e-mail: [email protected] >>>> >>>> > > -- > Twitter: https://twitter.com/holdenkarau > Fight Health Insurance: https://www.fighthealthinsurance.com/ > <https://www.fighthealthinsurance.com/?q=hk_email> > Books (Learning Spark, High Performance Spark, etc.): > https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9> > YouTube Live Streams: https://www.youtube.com/user/holdenkarau > Pronouns: she/her >
