I removed the "ai-review-requested" label, and now committers can add "spark-test" account as a reviewer. Then it will start reviewing.
On Sun, 15 Nov 2026 at 06:52, Hyukjin Kwon <[email protected]> wrote: > I actually added / configured the spark-test account so it should be added > as a reviewer ( > > https://github.com/apache/spark/blob/b49942ec319628db8723496310938c91d6b95759/.asf.yaml#L23 > ). I was distracted to something else so couldn't take a closer look yet > tho :-). Increased the timeout fwiw. > > On Sun, Nov 15, 2026 at 1:55 AM Yicong Huang via dev <[email protected]> > wrote: > >> For 1, I thought about it and I don’t think it’s doable though. GitHub >> requires an account to be either in the committer list, or have interacted >> with the PR (e.g., has commented the pr) before, to be eligible to be >> requested though the reviewer UI. So this may not work for Spark-test >> account. >> >> Best, >> Yicong Huang <https://yicong-huang.github.io> >> [email protected] >> >> On Nov 14, 2026 at 8:01 AM -0800, Hyukjin Kwon via dev < >> [email protected]>, wrote: >> >> Ah for 1. you mean to request review to spark-test account at "Reviewers" >> UI? I think that's a good idea. Let me take a look. >> >> 2. .. :-). I wonder if longer details are better or short ones are >> better. The main problem about this is security stuff .. this contains some >> credentials when it runs in fact, and I would like to avoid others to >> manipulate it for now. >> But I do agree that we would need some ways to configure this though. I >> think this one I would have to think about more ;-(. >> >> >> On Fri, 13 Nov 2026 at 00:39, Nicholas Chammas < >> [email protected]> wrote: >> >>> Thanks for working on this, Hyukjin. The reviews are often helpful and >>> improve the quality of contributions. >>> >>> Two questions: >>> >>> 1. Is it possible to summon the bot using the usual reviewer UI on >>> GitHub vs. needing to add a label? That would be a bit cleaner. >>> >>> 2. Can we configure the bot to be more brief? Many of its review >>> comments are extremely long, and this encourages the human receiving the >>> feedback to not read it and instead ask their own bot to summarize and/or >>> respond. So then we have two bots talking to one another via a human proxy. >>> 😄 >>> >>> I know that sometimes the added verbiage is useful as context or proof >>> of work. Could we have the bot structure each comment as a brief, executive >>> summary and tuck any additional detail away behind a collapsed `<details>` >>> element? GitHub supports those. >>> >>> Nick >>> >>> >>> On Nov 9, 2026, at 5:24 PM, Hyukjin Kwon via dev <[email protected]> >>> wrote: >>> >>> And just for a bit of context: >>> >>> The failure rate is 6-7% under 500 changed lines, 19% for 500–2,000, and >>> 56% over 2,000; 18% overall. A failure takes a median 3.4 h. >>> So we should probably avoid doing this for large PRs for now - or make >>> the PR smaller ;-). >>> >>> >>> On Tue, 10 Nov 2026 at 07:17, Hyukjin Kwon <[email protected]> wrote: >>> >>>> BTW, just for a bit of more context, I am slowly increasing the >>>> parallelism. For now, I am monitoring the review results for all - it gives >>>> me notification to my phone for every review :-). >>>> >>>> On Tue, 10 Nov 2026 at 06:47, Hyukjin Kwon <[email protected]> >>>> wrote: >>>> >>>>> Seems like this is working quite fine although sometimes it fails (and >>>>> says to try again). Let me increase the parallelism to 8. So it reviews 8 >>>>> at a time. >>>>> >>>>> On Wed, 4 Nov 2026 at 08:47, Holden Karau <[email protected]> >>>>> wrote: >>>>> >>>>>> Very cool :) >>>>>> >>>>>> On Tue, Nov 3, 2026 at 2:51 PM Hyukjin Kwon via dev < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Just FYI. Now, the bot is reviewing 4 at a time. The review takes >>>>>>> around 1 hour. So sometimes it takes time. I am still monitoring. >>>>>>> >>>>>>> On Fri, 23 Oct 2026 at 07:09, Hyukjin Kwon <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> FYI, I am monitoring and controlling details now. Effort, timeout, >>>>>>>> security stuff. >>>>>>>> >>>>>>>> On Thu, 22 Oct 2026 at 22:51, Dongjoon Hyun <[email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Got it. Thank you! >>>>>>>>> >>>>>>>>> Dongjoon. >>>>>>>>> >>>>>>>>> On 2026/10/22 11:44:04 Hyukjin Kwon wrote: >>>>>>>>> > Just to reduce any surprise, I set another bot to monitor the >>>>>>>>> bot itself. >>>>>>>>> > If it does anything wrong, it halts immediately, and >>>>>>>>> removes/reverts >>>>>>>>> > whatever was wrong. >>>>>>>>> > >>>>>>>>> > For committers, please directly remove the `ai-review-requested` >>>>>>>>> label if >>>>>>>>> > there seems anything wrong. That would make the bot stop too. >>>>>>>>> > >>>>>>>>> > On Thu, 22 Oct 2026 at 20:21, Hyukjin Kwon <[email protected]> >>>>>>>>> wrote: >>>>>>>>> > >>>>>>>>> > > Ok .. I managed to enable the bots. Please try >>>>>>>>> `ai-review-requested` label >>>>>>>>> > > out (only commiters can do). and please let me know if there >>>>>>>>> are anything >>>>>>>>> > > suspicious or not working well 👍 >>>>>>>>> > > >>>>>>>>> > > On Tue, 20 Oct 2026 at 08:31, Dongjoon Hyun < >>>>>>>>> [email protected]> wrote: >>>>>>>>> > > >>>>>>>>> > >> Thank you so much for keeping working and sharing the status, >>>>>>>>> Hyukjin. >>>>>>>>> > >> >>>>>>>>> > >> Dongjoon. >>>>>>>>> > >> >>>>>>>>> > >> On 2026/10/19 21:50:36 Hyukjin Kwon wrote: >>>>>>>>> > >> > update: I am still struggling / testing. >>>>>>>>> `ai-review-requested` label >>>>>>>>> > >> does >>>>>>>>> > >> > not work for now. >>>>>>>>> > >> > >>>>>>>>> > >> > On Fri, 16 Oct 2026 at 07:05, Hyukjin Kwon < >>>>>>>>> [email protected]> >>>>>>>>> > >> wrote: >>>>>>>>> > >> > >>>>>>>>> > >> > > headsup. I will stop the bot today. Will likely introduce >>>>>>>>> > >> > > `ai-review-requested` label (only committers can set) to >>>>>>>>> trigger the >>>>>>>>> > >> review >>>>>>>>> > >> > > most likely today for testing it out, see >>>>>>>>> > >> > > https://github.com/apache/spark/pull/59502 >>>>>>>>> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fspark%2Fpull%2F59502&data=05%7C02%7Cyiconghuang%40umass.edu%7C74d9c558bfe24daa363708df42688978%7C7bd08b0b33954dc194bbd0b2e56a497f%7C0%7C0%7C639302688893297671%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=3uaAdvC5IWOTAu6QL19WTTdiyMASvOfjvApITn83QPI%3D&reserved=0> >>>>>>>>> > >> > > >>>>>>>>> > >> > > On Mon, 12 Oct 2026 at 08:14, Hyukjin Kwon < >>>>>>>>> [email protected]> >>>>>>>>> > >> wrote: >>>>>>>>> > >> > > >>>>>>>>> > >> > >> Another quick heads up. >>>>>>>>> > >> > >> >>>>>>>>> > >> > >> - I am basically stopping (or rather not posting) the >>>>>>>>> reviews when >>>>>>>>> > >> > >> there are no findings. >>>>>>>>> > >> > >> - If there are updates from the author, the review >>>>>>>>> will be >>>>>>>>> > >> triggered. >>>>>>>>> > >> > >> To minimize the cost, I am not setting the effort as >>>>>>>>> max but >>>>>>>>> > >> rather trigger >>>>>>>>> > >> > >> multiple times whenever the author pushes a newer >>>>>>>>> commit. >>>>>>>>> > >> > >> - I am looking especially through P3 review comments >>>>>>>>> or not. >>>>>>>>> > >> Please >>>>>>>>> > >> > >> let me know if you guys are getting too many P3 nits, >>>>>>>>> and if that >>>>>>>>> > >> bugs you. >>>>>>>>> > >> > >> >>>>>>>>> > >> > >> >>>>>>>>> > >> > >> >>>>>>>>> > >> > >> On Thu, 8 Oct 2026 at 10:54, Hyukjin Kwon < >>>>>>>>> [email protected]> >>>>>>>>> > >> wrote: >>>>>>>>> > >> > >> >>>>>>>>> > >> > >>> Just quick headup, I am having some trouble with >>>>>>>>> automation right >>>>>>>>> > >> now. I >>>>>>>>> > >> > >>> would need to test/run manually so it would be >>>>>>>>> triggered like >>>>>>>>> > >> everyday for >>>>>>>>> > >> > >>> now. >>>>>>>>> > >> > >>> >>>>>>>>> > >> > >>> On Sat, 3 Oct 2026 at 01:53, Dongjoon Hyun < >>>>>>>>> [email protected]> >>>>>>>>> > >> wrote: >>>>>>>>> > >> > >>> >>>>>>>>> > >> > >>>> +1 >>>>>>>>> > >> > >>>> >>>>>>>>> > >> > >>>> Thanks, Hyukjin. >>>>>>>>> > >> > >>>> >>>>>>>>> > >> > >>>> Could we introduce a few safeguards for the new review >>>>>>>>> bot? >>>>>>>>> > >> > >>>> >>>>>>>>> > >> > >>>> Specifically: >>>>>>>>> > >> > >>>> - A GitHub label to opt out or disable the bot per PR. >>>>>>>>> > >> > >>>> - Protections against excessive ASF Jira API calls. >>>>>>>>> > >> > >>>> >>>>>>>>> > >> > >>>> This would prevent the bot from being blocked/banned >>>>>>>>> by ASF Jira >>>>>>>>> > >> > >>>> and allow reviews to proceed even if Jira is down. It >>>>>>>>> also helps >>>>>>>>> > >> > >>>> manage worst-case scenarios where the bot leaves >>>>>>>>> repetitive or >>>>>>>>> > >> > >>>> unhelpful nitpicks. >>>>>>>>> > >> > >>>> >>>>>>>>> > >> > >>>> Best, >>>>>>>>> > >> > >>>> Dongjoon >>>>>>>>> > >> > >>>> >>>>>>>>> > >> > >>>> On 2026-10-01 21:58 UTC Hyukjin Kwon wrote: >>>>>>>>> > >> > >>>> > Hi all, >>>>>>>>> > >> > >>>> > >>>>>>>>> > >> > >>>> > I've been running a trial of automated AI code >>>>>>>>> review on new >>>>>>>>> > >> > >>>> apache/spark >>>>>>>>> > >> > >>>> > PRs, and I'd like to share how it works and get your >>>>>>>>> feedback. >>>>>>>>> > >> > >>>> > >>>>>>>>> > >> > >>>> > For now the reviews are posted from my own GitHub >>>>>>>>> account >>>>>>>>> > >> > >>>> (@HyukjinKwon) >>>>>>>>> > >> > >>>> > while I check that it behaves well. Once it's >>>>>>>>> proven, I plan to >>>>>>>>> > >> move >>>>>>>>> > >> > >>>> it to >>>>>>>>> > >> > >>>> > the https://github.com/spark-test >>>>>>>>> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fspark-test&data=05%7C02%7Cyiconghuang%40umass.edu%7C74d9c558bfe24daa363708df42688978%7C7bd08b0b33954dc194bbd0b2e56a497f%7C0%7C0%7C639302688893310149%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=vm2ZgYFs1QtpiuW5xhXCpspKMGFN6W%2FZlWo16ydPNBY%3D&reserved=0> >>>>>>>>> account, so the reviews are >>>>>>>>> > >> clearly >>>>>>>>> > >> > >>>> > separate from my personal ones. I plan to run this >>>>>>>>> for the next >>>>>>>>> > >> week. >>>>>>>>> > >> > >>>> > >>>>>>>>> > >> > >>>> > *What it does* >>>>>>>>> > >> > >>>> > >>>>>>>>> > >> > >>>> > Each review is a full, multi-pass review of the PR >>>>>>>>> diff, run by >>>>>>>>> > >> Claude >>>>>>>>> > >> > >>>> > (via Claude Code). Findings are checked against the >>>>>>>>> code before >>>>>>>>> > >> > >>>> they're >>>>>>>>> > >> > >>>> > posted. They're posted as a normal GitHub review: a >>>>>>>>> short >>>>>>>>> > >> summary plus >>>>>>>>> > >> > >>>> > inline comments, each labeled with a severity, e.g. >>>>>>>>> "Non-blocking >>>>>>>>> > >> > >>>> (P2)" >>>>>>>>> > >> > >>>> > or "Nit (P3)". Example: >>>>>>>>> > >> > >>>> > >>>>>>>>> > >> > >>>> >>>>>>>>> > >> >>>>>>>>> https://github.com/apache/spark/pull/59184#pullrequestreview-5379759787 >>>>>>>>> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fspark%2Fpull%2F59184%23pullrequestreview-5379759787&data=05%7C02%7Cyiconghuang%40umass.edu%7C74d9c558bfe24daa363708df42688978%7C7bd08b0b33954dc194bbd0b2e56a497f%7C0%7C0%7C639302688893321184%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=gBjqnYLbvi1jyxDKcgdb0ibWL549iSkpuByWv6wfe%2Bs%3D&reserved=0> >>>>>>>>> > >> > >>>> > >>>>>>>>> > >> > >>>> > *No LGTM, no approvals* >>>>>>>>> > >> > >>>> > >>>>>>>>> > >> > >>>> > - It never approves a PR and never posts "LGTM". >>>>>>>>> Every review is >>>>>>>>> > >> a >>>>>>>>> > >> > >>>> plain >>>>>>>>> > >> > >>>> > "Comment" review, so it doesn't count as an >>>>>>>>> approval or a >>>>>>>>> > >> change >>>>>>>>> > >> > >>>> > request, and it has no effect on merging. >>>>>>>>> > >> > >>>> > - If a review finds nothing new, it posts nothing at >>>>>>>>> all: no "no >>>>>>>>> > >> > >>>> > findings" or "all comments addressed" message. >>>>>>>>> > >> > >>>> > - It doesn't push commits or edit anyone's PR. >>>>>>>>> > >> > >>>> > - Human review and committer judgement are >>>>>>>>> unchanged. Please >>>>>>>>> > >> treat its >>>>>>>>> > >> > >>>> > comments like any other reviewer's: feel free to >>>>>>>>> push back or >>>>>>>>> > >> > >>>> ignore. >>>>>>>>> > >> > >>>> > >>>>>>>>> > >> > >>>> > *When a review is triggered* >>>>>>>>> > >> > >>>> > >>>>>>>>> > >> > >>>> > - A new PR is reviewed shortly after it's opened. >>>>>>>>> Draft PRs are >>>>>>>>> > >> > >>>> reviewed >>>>>>>>> > >> > >>>> > once they're marked ready for review. >>>>>>>>> > >> > >>>> > - After a review posts findings, the bot waits. It >>>>>>>>> reviews again >>>>>>>>> > >> only >>>>>>>>> > >> > >>>> > when the author pushes new commits. >>>>>>>>> > >> > >>>> > - It also waits while a PR has open review comments >>>>>>>>> from anyone >>>>>>>>> > >> that >>>>>>>>> > >> > >>>> are >>>>>>>>> > >> > >>>> > newer than the latest commit, so it doesn't pile >>>>>>>>> on while the >>>>>>>>> > >> > >>>> author is >>>>>>>>> > >> > >>>> > still responding to reviewers. >>>>>>>>> > >> > >>>> > - Replying in a thread without pushing doesn't >>>>>>>>> trigger a >>>>>>>>> > >> re-review. >>>>>>>>> > >> > >>>> > >>>>>>>>> > >> > >>>> > *When it stops* >>>>>>>>> > >> > >>>> > >>>>>>>>> > >> > >>>> > - When a review finds nothing new. The PR gets a >>>>>>>>> fresh review if >>>>>>>>> > >> the >>>>>>>>> > >> > >>>> > author pushes again later. >>>>>>>>> > >> > >>>> > - When the PR is merged or closed. >>>>>>>>> > >> > >>>> > >>>>>>>>> > >> > >>>> > *Early numbers* >>>>>>>>> > >> > >>>> > >>>>>>>>> > >> > >>>> > Over the first ~12 hours (Oct 1), it covered 10 PRs, >>>>>>>>> including 5 >>>>>>>>> > >> > >>>> > re-reviews after the author pushed. It posted 7 >>>>>>>>> reviews with 14 >>>>>>>>> > >> > >>>> findings >>>>>>>>> > >> > >>>> > in total, mostly P3 nits. Four PRs came back clean >>>>>>>>> and got no >>>>>>>>> > >> comment. >>>>>>>>> > >> > >>>> > Several authors pushed fixes, and the follow-up >>>>>>>>> reviews found >>>>>>>>> > >> nothing >>>>>>>>> > >> > >>>> > new. >>>>>>>>> > >> > >>>> > >>>>>>>>> > >> > >>>> > *Feedback* >>>>>>>>> > >> > >>>> > >>>>>>>>> > >> > >>>> > Please let me know if the comments are useful, >>>>>>>>> noisy, or wrong, >>>>>>>>> > >> either >>>>>>>>> > >> > >>>> > here or on the PR. If you'd rather it not review >>>>>>>>> your PRs, tell >>>>>>>>> > >> me and >>>>>>>>> > >> > >>>> > I'll exclude them. I'm also happy to adjust when it >>>>>>>>> triggers >>>>>>>>> > >> based on >>>>>>>>> > >> > >>>> > what the community prefers. >>>>>>>>> > >> > >>>> > >>>>>>>>> > >> > >>>> > Thanks, >>>>>>>>> > >> > >>>> > Hyukjin >>>>>>>>> > >> > >>>> > >>>>>>>>> > >> > >>>> >>>>>>>>> > >> > >>>> >>>>>>>>> > >> >>>>>>>>> --------------------------------------------------------------------- >>>>>>>>> > >> > >>>> To unsubscribe e-mail: >>>>>>>>> [email protected] >>>>>>>>> > >> > >>>> >>>>>>>>> > >> > >>>> >>>>>>>>> > >> > >>>>>>>>> > >> >>>>>>>>> > >> >>>>>>>>> --------------------------------------------------------------------- >>>>>>>>> > >> To unsubscribe e-mail: [email protected] >>>>>>>>> > >> >>>>>>>>> > >> >>>>>>>>> > >>>>>>>>> >>>>>>>>> >>>>>>>>> --------------------------------------------------------------------- >>>>>>>>> To unsubscribe e-mail: [email protected] >>>>>>>>> >>>>>>>>> >>>>>> >>>>>> -- >>>>>> Twitter: https://twitter.com/holdenkarau >>>>>> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftwitter.com%2Fholdenkarau&data=05%7C02%7Cyiconghuang%40umass.edu%7C74d9c558bfe24daa363708df42688978%7C7bd08b0b33954dc194bbd0b2e56a497f%7C0%7C0%7C639302688893331775%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=ZDxz4lCNg7RbaVYFnEsPjffq4UxeVcT2VOwIQ7dhavI%3D&reserved=0> >>>>>> Fight Health Insurance: https://www.fighthealthinsurance.com/ >>>>>> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.fighthealthinsurance.com%2F%3Fq%3Dhk_email&data=05%7C02%7Cyiconghuang%40umass.edu%7C74d9c558bfe24daa363708df42688978%7C7bd08b0b33954dc194bbd0b2e56a497f%7C0%7C0%7C639302688893342171%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=KCXHNepr67P48xO8othccAjKAGME3TMuzM3qz%2FohA7E%3D&reserved=0> >>>>>> Books (Learning Spark, High Performance Spark, etc.): >>>>>> https://amzn.to/2MaRAG9 >>>>>> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Famzn.to%2F2MaRAG9&data=05%7C02%7Cyiconghuang%40umass.edu%7C74d9c558bfe24daa363708df42688978%7C7bd08b0b33954dc194bbd0b2e56a497f%7C0%7C0%7C639302688893352588%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=vczk0xLgWEmXDFmeNo7rI0SNc%2FakZTQjXR2RUA7RLa8%3D&reserved=0> >>>>>> YouTube Live Streams: https://www.youtube.com/user/holdenkarau >>>>>> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.youtube.com%2Fuser%2Fholdenkarau&data=05%7C02%7Cyiconghuang%40umass.edu%7C74d9c558bfe24daa363708df42688978%7C7bd08b0b33954dc194bbd0b2e56a497f%7C0%7C0%7C639302688893362855%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=Miu1Xlj3fkF1W3zORncYRstv7HPvc7xQWvE9UmUhYtQ%3D&reserved=0> >>>>>> Pronouns: she/her >>>>>> >>>>> >>>
