For 1, I thought about it and I don’t think it’s doable though. GitHub requires an account to be either in the committer list, or have interacted with the PR (e.g., has commented the pr) before, to be eligible to be requested though the reviewer UI. So this may not work for Spark-test account.
Best, Yicong Huang [email protected] On Nov 14, 2026 at 8:01 AM -0800, Hyukjin Kwon via dev <[email protected]>, wrote: > Ah for 1. you mean to request review to spark-test account at "Reviewers" UI? > I think that's a good idea. Let me take a look. > > 2. .. :-). I wonder if longer details are better or short ones are better. > The main problem about this is security stuff .. this contains some > credentials when it runs in fact, and I would like to avoid others to > manipulate it for now. > But I do agree that we would need some ways to configure this though. I think > this one I would have to think about more ;-(. > > > On Fri, 13 Nov 2026 at 00:39, Nicholas Chammas <[email protected]> > wrote: > > Thanks for working on this, Hyukjin. The reviews are often helpful and > > improve the quality of contributions. > > > > Two questions: > > > > 1. Is it possible to summon the bot using the usual reviewer UI on GitHub > > vs. needing to add a label? That would be a bit cleaner. > > > > 2. Can we configure the bot to be more brief? Many of its review comments > > are extremely long, and this encourages the human receiving the feedback to > > not read it and instead ask their own bot to summarize and/or respond. So > > then we have two bots talking to one another via a human proxy. 😄 > > > > I know that sometimes the added verbiage is useful as context or proof of > > work. Could we have the bot structure each comment as a brief, executive > > summary and tuck any additional detail away behind a collapsed `<details>` > > element? GitHub supports those. > > > > Nick > > > > > > > On Nov 9, 2026, at 5:24 PM, Hyukjin Kwon via dev <[email protected]> > > > wrote: > > > > > > And just for a bit of context: > > > > > > The failure rate is 6-7% under 500 changed lines, 19% for 500–2,000, and > > > 56% over 2,000; 18% overall. A failure takes a median 3.4 h. > > > So we should probably avoid doing this for large PRs for now - or make > > > the PR smaller ;-). > > > > > > > > > > On Tue, 10 Nov 2026 at 07:17, Hyukjin Kwon <[email protected]> wrote: > > > > > BTW, just for a bit of more context, I am slowly increasing the > > > > > parallelism. For now, I am monitoring the review results for all - it > > > > > gives me notification to my phone for every review :-). > > > > > > > > > > > On Tue, 10 Nov 2026 at 06:47, Hyukjin Kwon <[email protected]> > > > > > > wrote: > > > > > > > Seems like this is working quite fine although sometimes it fails > > > > > > > (and says to try again). Let me increase the parallelism to 8. So > > > > > > > it reviews 8 at a time. > > > > > > > > > > > > > > > On Wed, 4 Nov 2026 at 08:47, Holden Karau > > > > > > > > <[email protected]> wrote: > > > > > > > > > Very cool :) > > > > > > > > > > > > > > > > > > > On Tue, Nov 3, 2026 at 2:51 PM Hyukjin Kwon via dev > > > > > > > > > > <[email protected]> wrote: > > > > > > > > > > > Just FYI. Now, the bot is reviewing 4 at a time. The > > > > > > > > > > > review takes around 1 hour. So sometimes it takes time. I > > > > > > > > > > > am still monitoring. > > > > > > > > > > > > > > > > > > > > > > > On Fri, 23 Oct 2026 at 07:09, Hyukjin Kwon > > > > > > > > > > > > <[email protected]> wrote: > > > > > > > > > > > > > FYI, I am monitoring and controlling details now. > > > > > > > > > > > > > Effort, timeout, security stuff. > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, 22 Oct 2026 at 22:51, Dongjoon Hyun > > > > > > > > > > > > > > <[email protected]> wrote: > > > > > > > > > > > > > > > Got it. Thank you! > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Dongjoon. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On 2026/10/22 11:44:04 Hyukjin Kwon wrote: > > > > > > > > > > > > > > > > Just to reduce any surprise, I set another bot > > > > > > > > > > > > > > > > to monitor the bot itself. > > > > > > > > > > > > > > > > If it does anything wrong, it halts > > > > > > > > > > > > > > > > immediately, and removes/reverts > > > > > > > > > > > > > > > > whatever was wrong. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > For committers, please directly remove the > > > > > > > > > > > > > > > > `ai-review-requested` label if > > > > > > > > > > > > > > > > there seems anything wrong. That would make the > > > > > > > > > > > > > > > > bot stop too. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, 22 Oct 2026 at 20:21, Hyukjin Kwon > > > > > > > > > > > > > > > > <[email protected]> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Ok .. I managed to enable the bots. Please > > > > > > > > > > > > > > > > > try `ai-review-requested` label > > > > > > > > > > > > > > > > > out (only commiters can do). and please let > > > > > > > > > > > > > > > > > me know if there are anything > > > > > > > > > > > > > > > > > suspicious or not working well 👍 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, 20 Oct 2026 at 08:31, Dongjoon Hyun > > > > > > > > > > > > > > > > > <[email protected]> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >> Thank you so much for keeping working and > > > > > > > > > > > > > > > > >> sharing the status, Hyukjin. > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > >> Dongjoon. > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > >> On 2026/10/19 21:50:36 Hyukjin Kwon wrote: > > > > > > > > > > > > > > > > >> > update: I am still struggling / testing. > > > > > > > > > > > > > > > > >> > `ai-review-requested` label > > > > > > > > > > > > > > > > >> does > > > > > > > > > > > > > > > > >> > not work for now. > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> > On Fri, 16 Oct 2026 at 07:05, Hyukjin Kwon > > > > > > > > > > > > > > > > >> > <[email protected]> > > > > > > > > > > > > > > > > >> wrote: > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > headsup. I will stop the bot today. Will > > > > > > > > > > > > > > > > >> > > likely introduce > > > > > > > > > > > > > > > > >> > > `ai-review-requested` label (only > > > > > > > > > > > > > > > > >> > > committers can set) to trigger the > > > > > > > > > > > > > > > > >> review > > > > > > > > > > > > > > > > >> > > most likely today for testing it out, see > > > > > > > > > > > > > > > > >> > > https://github.com/apache/spark/pull/59502 > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > >> > > On Mon, 12 Oct 2026 at 08:14, Hyukjin > > > > > > > > > > > > > > > > >> > > Kwon <[email protected]> > > > > > > > > > > > > > > > > >> wrote: > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > >> > >> Another quick heads up. > > > > > > > > > > > > > > > > >> > >> > > > > > > > > > > > > > > > > >> > >> - I am basically stopping (or rather > > > > > > > > > > > > > > > > >> > >>not posting) the reviews when > > > > > > > > > > > > > > > > >> > >> there are no findings. > > > > > > > > > > > > > > > > >> > >> - If there are updates from the > > > > > > > > > > > > > > > > >> > >>author, the review will be > > > > > > > > > > > > > > > > >> triggered. > > > > > > > > > > > > > > > > >> > >> To minimize the cost, I am not > > > > > > > > > > > > > > > > >> > >>setting the effort as max but > > > > > > > > > > > > > > > > >> rather trigger > > > > > > > > > > > > > > > > >> > >> multiple times whenever the author > > > > > > > > > > > > > > > > >> > >>pushes a newer commit. > > > > > > > > > > > > > > > > >> > >> - I am looking especially through P3 > > > > > > > > > > > > > > > > >> > >>review comments or not. > > > > > > > > > > > > > > > > >> Please > > > > > > > > > > > > > > > > >> > >> let me know if you guys are getting > > > > > > > > > > > > > > > > >> > >>too many P3 nits, and if that > > > > > > > > > > > > > > > > >> bugs you. > > > > > > > > > > > > > > > > >> > >> > > > > > > > > > > > > > > > > >> > >> > > > > > > > > > > > > > > > > >> > >> > > > > > > > > > > > > > > > > >> > >> On Thu, 8 Oct 2026 at 10:54, Hyukjin > > > > > > > > > > > > > > > > >> > >> Kwon <[email protected]> > > > > > > > > > > > > > > > > >> wrote: > > > > > > > > > > > > > > > > >> > >> > > > > > > > > > > > > > > > > >> > >>> Just quick headup, I am having some > > > > > > > > > > > > > > > > >> > >>> trouble with automation right > > > > > > > > > > > > > > > > >> now. I > > > > > > > > > > > > > > > > >> > >>> would need to test/run manually so it > > > > > > > > > > > > > > > > >> > >>> would be triggered like > > > > > > > > > > > > > > > > >> everyday for > > > > > > > > > > > > > > > > >> > >>> now. > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > >> > >>> On Sat, 3 Oct 2026 at 01:53, Dongjoon > > > > > > > > > > > > > > > > >> > >>> Hyun <[email protected]> > > > > > > > > > > > > > > > > >> wrote: > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > >> > >>>> +1 > > > > > > > > > > > > > > > > >> > >>>> > > > > > > > > > > > > > > > > >> > >>>> Thanks, Hyukjin. > > > > > > > > > > > > > > > > >> > >>>> > > > > > > > > > > > > > > > > >> > >>>> Could we introduce a few safeguards > > > > > > > > > > > > > > > > >> > >>>> for the new review bot? > > > > > > > > > > > > > > > > >> > >>>> > > > > > > > > > > > > > > > > >> > >>>> Specifically: > > > > > > > > > > > > > > > > >> > >>>> - A GitHub label to opt out or > > > > > > > > > > > > > > > > >> > >>>> disable the bot per PR. > > > > > > > > > > > > > > > > >> > >>>> - Protections against excessive ASF > > > > > > > > > > > > > > > > >> > >>>> Jira API calls. > > > > > > > > > > > > > > > > >> > >>>> > > > > > > > > > > > > > > > > >> > >>>> This would prevent the bot from being > > > > > > > > > > > > > > > > >> > >>>> blocked/banned by ASF Jira > > > > > > > > > > > > > > > > >> > >>>> and allow reviews to proceed even if > > > > > > > > > > > > > > > > >> > >>>> Jira is down. It also helps > > > > > > > > > > > > > > > > >> > >>>> manage worst-case scenarios where the > > > > > > > > > > > > > > > > >> > >>>> bot leaves repetitive or > > > > > > > > > > > > > > > > >> > >>>> unhelpful nitpicks. > > > > > > > > > > > > > > > > >> > >>>> > > > > > > > > > > > > > > > > >> > >>>> Best, > > > > > > > > > > > > > > > > >> > >>>> Dongjoon > > > > > > > > > > > > > > > > >> > >>>> > > > > > > > > > > > > > > > > >> > >>>> On 2026-10-01 21:58 UTC Hyukjin Kwon > > > > > > > > > > > > > > > > >> > >>>> wrote: > > > > > > > > > > > > > > > > >> > >>>> > Hi all, > > > > > > > > > > > > > > > > >> > >>>> > > > > > > > > > > > > > > > > > >> > >>>> > I've been running a trial of > > > > > > > > > > > > > > > > >> > >>>> > automated AI code review on new > > > > > > > > > > > > > > > > >> > >>>> apache/spark > > > > > > > > > > > > > > > > >> > >>>> > PRs, and I'd like to share how it > > > > > > > > > > > > > > > > >> > >>>> > works and get your feedback. > > > > > > > > > > > > > > > > >> > >>>> > > > > > > > > > > > > > > > > > >> > >>>> > For now the reviews are posted from > > > > > > > > > > > > > > > > >> > >>>> > my own GitHub account > > > > > > > > > > > > > > > > >> > >>>> (@HyukjinKwon) > > > > > > > > > > > > > > > > >> > >>>> > while I check that it behaves well. > > > > > > > > > > > > > > > > >> > >>>> > Once it's proven, I plan to > > > > > > > > > > > > > > > > >> move > > > > > > > > > > > > > > > > >> > >>>> it to > > > > > > > > > > > > > > > > >> > >>>> > the https://github.com/spark-test > > > > > > > > > > > > > > > > >> > >>>> > account, so the reviews are > > > > > > > > > > > > > > > > >> clearly > > > > > > > > > > > > > > > > >> > >>>> > separate from my personal ones. I > > > > > > > > > > > > > > > > >> > >>>> > plan to run this for the next > > > > > > > > > > > > > > > > >> week. > > > > > > > > > > > > > > > > >> > >>>> > > > > > > > > > > > > > > > > > >> > >>>> > *What it does* > > > > > > > > > > > > > > > > >> > >>>> > > > > > > > > > > > > > > > > > >> > >>>> > Each review is a full, multi-pass > > > > > > > > > > > > > > > > >> > >>>> > review of the PR diff, run by > > > > > > > > > > > > > > > > >> Claude > > > > > > > > > > > > > > > > >> > >>>> > (via Claude Code). Findings are > > > > > > > > > > > > > > > > >> > >>>> > checked against the code before > > > > > > > > > > > > > > > > >> > >>>> they're > > > > > > > > > > > > > > > > >> > >>>> > posted. They're posted as a normal > > > > > > > > > > > > > > > > >> > >>>> > GitHub review: a short > > > > > > > > > > > > > > > > >> summary plus > > > > > > > > > > > > > > > > >> > >>>> > inline comments, each labeled with > > > > > > > > > > > > > > > > >> > >>>> > a severity, e.g. "Non-blocking > > > > > > > > > > > > > > > > >> > >>>> (P2)" > > > > > > > > > > > > > > > > >> > >>>> > or "Nit (P3)". Example: > > > > > > > > > > > > > > > > >> > >>>> > > > > > > > > > > > > > > > > > >> > >>>> > > > > > > > > > > > > > > > > >> https://github.com/apache/spark/pull/59184#pullrequestreview-5379759787 > > > > > > > > > > > > > > > > >> > >>>> > > > > > > > > > > > > > > > > > >> > >>>> > *No LGTM, no approvals* > > > > > > > > > > > > > > > > >> > >>>> > > > > > > > > > > > > > > > > > >> > >>>> > - It never approves a PR and never > > > > > > > > > > > > > > > > >> > >>>> > posts "LGTM". Every review is > > > > > > > > > > > > > > > > >> a > > > > > > > > > > > > > > > > >> > >>>> plain > > > > > > > > > > > > > > > > >> > >>>> > "Comment" review, so it doesn't > > > > > > > > > > > > > > > > >> > >>>> >count as an approval or a > > > > > > > > > > > > > > > > >> change > > > > > > > > > > > > > > > > >> > >>>> > request, and it has no effect on > > > > > > > > > > > > > > > > >> > >>>> >merging. > > > > > > > > > > > > > > > > >> > >>>> > - If a review finds nothing new, it > > > > > > > > > > > > > > > > >> > >>>> > posts nothing at all: no "no > > > > > > > > > > > > > > > > >> > >>>> > findings" or "all comments > > > > > > > > > > > > > > > > >> > >>>> >addressed" message. > > > > > > > > > > > > > > > > >> > >>>> > - It doesn't push commits or edit > > > > > > > > > > > > > > > > >> > >>>> > anyone's PR. > > > > > > > > > > > > > > > > >> > >>>> > - Human review and committer > > > > > > > > > > > > > > > > >> > >>>> > judgement are unchanged. Please > > > > > > > > > > > > > > > > >> treat its > > > > > > > > > > > > > > > > >> > >>>> > comments like any other > > > > > > > > > > > > > > > > >> > >>>> >reviewer's: feel free to push back or > > > > > > > > > > > > > > > > >> > >>>> ignore. > > > > > > > > > > > > > > > > >> > >>>> > > > > > > > > > > > > > > > > > >> > >>>> > *When a review is triggered* > > > > > > > > > > > > > > > > >> > >>>> > > > > > > > > > > > > > > > > > >> > >>>> > - A new PR is reviewed shortly > > > > > > > > > > > > > > > > >> > >>>> > after it's opened. Draft PRs are > > > > > > > > > > > > > > > > >> > >>>> reviewed > > > > > > > > > > > > > > > > >> > >>>> > once they're marked ready for > > > > > > > > > > > > > > > > >> > >>>> >review. > > > > > > > > > > > > > > > > >> > >>>> > - After a review posts findings, > > > > > > > > > > > > > > > > >> > >>>> > the bot waits. It reviews again > > > > > > > > > > > > > > > > >> only > > > > > > > > > > > > > > > > >> > >>>> > when the author pushes new > > > > > > > > > > > > > > > > >> > >>>> >commits. > > > > > > > > > > > > > > > > >> > >>>> > - It also waits while a PR has open > > > > > > > > > > > > > > > > >> > >>>> > review comments from anyone > > > > > > > > > > > > > > > > >> that > > > > > > > > > > > > > > > > >> > >>>> are > > > > > > > > > > > > > > > > >> > >>>> > newer than the latest commit, so > > > > > > > > > > > > > > > > >> > >>>> >it doesn't pile on while the > > > > > > > > > > > > > > > > >> > >>>> author is > > > > > > > > > > > > > > > > >> > >>>> > still responding to reviewers. > > > > > > > > > > > > > > > > >> > >>>> > - Replying in a thread without > > > > > > > > > > > > > > > > >> > >>>> > pushing doesn't trigger a > > > > > > > > > > > > > > > > >> re-review. > > > > > > > > > > > > > > > > >> > >>>> > > > > > > > > > > > > > > > > > >> > >>>> > *When it stops* > > > > > > > > > > > > > > > > >> > >>>> > > > > > > > > > > > > > > > > > >> > >>>> > - When a review finds nothing new. > > > > > > > > > > > > > > > > >> > >>>> > The PR gets a fresh review if > > > > > > > > > > > > > > > > >> the > > > > > > > > > > > > > > > > >> > >>>> > author pushes again later. > > > > > > > > > > > > > > > > >> > >>>> > - When the PR is merged or closed. > > > > > > > > > > > > > > > > >> > >>>> > > > > > > > > > > > > > > > > > >> > >>>> > *Early numbers* > > > > > > > > > > > > > > > > >> > >>>> > > > > > > > > > > > > > > > > > >> > >>>> > Over the first ~12 hours (Oct 1), > > > > > > > > > > > > > > > > >> > >>>> > it covered 10 PRs, including 5 > > > > > > > > > > > > > > > > >> > >>>> > re-reviews after the author pushed. > > > > > > > > > > > > > > > > >> > >>>> > It posted 7 reviews with 14 > > > > > > > > > > > > > > > > >> > >>>> findings > > > > > > > > > > > > > > > > >> > >>>> > in total, mostly P3 nits. Four PRs > > > > > > > > > > > > > > > > >> > >>>> > came back clean and got no > > > > > > > > > > > > > > > > >> comment. > > > > > > > > > > > > > > > > >> > >>>> > Several authors pushed fixes, and > > > > > > > > > > > > > > > > >> > >>>> > the follow-up reviews found > > > > > > > > > > > > > > > > >> nothing > > > > > > > > > > > > > > > > >> > >>>> > new. > > > > > > > > > > > > > > > > >> > >>>> > > > > > > > > > > > > > > > > > >> > >>>> > *Feedback* > > > > > > > > > > > > > > > > >> > >>>> > > > > > > > > > > > > > > > > > >> > >>>> > Please let me know if the comments > > > > > > > > > > > > > > > > >> > >>>> > are useful, noisy, or wrong, > > > > > > > > > > > > > > > > >> either > > > > > > > > > > > > > > > > >> > >>>> > here or on the PR. If you'd rather > > > > > > > > > > > > > > > > >> > >>>> > it not review your PRs, tell > > > > > > > > > > > > > > > > >> me and > > > > > > > > > > > > > > > > >> > >>>> > I'll exclude them. I'm also happy > > > > > > > > > > > > > > > > >> > >>>> > to adjust when it triggers > > > > > > > > > > > > > > > > >> based on > > > > > > > > > > > > > > > > >> > >>>> > what the community prefers. > > > > > > > > > > > > > > > > >> > >>>> > > > > > > > > > > > > > > > > > >> > >>>> > Thanks, > > > > > > > > > > > > > > > > >> > >>>> > Hyukjin > > > > > > > > > > > > > > > > >> > >>>> > > > > > > > > > > > > > > > > > >> > >>>> > > > > > > > > > > > > > > > > >> > >>>> > > > > > > > > > > > > > > > > >> --------------------------------------------------------------------- > > > > > > > > > > > > > > > > >> > >>>> To unsubscribe e-mail: > > > > > > > > > > > > > > > > >> > >>>> [email protected] > > > > > > > > > > > > > > > > >> > >>>> > > > > > > > > > > > > > > > > >> > >>>> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > >> --------------------------------------------------------------------- > > > > > > > > > > > > > > > > >> To unsubscribe e-mail: > > > > > > > > > > > > > > > > >> [email protected] > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > > > > > > > > > > To unsubscribe e-mail: > > > > > > > > > > > > > > > [email protected] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > Twitter: https://twitter.com/holdenkarau > > > > > > > > > Fight Health Insurance: https://www.fighthealthinsurance.com/ > > > > > > > > > Books (Learning Spark, High Performance Spark, etc.): > > > > > > > > > https://amzn.to/2MaRAG9 > > > > > > > > > YouTube Live Streams: https://www.youtube.com/user/holdenkarau > > > > > > > > > Pronouns: she/her > >
