Sorry for slightly resurrecting this thread. But something I just noticed is that it appears that copilot auto-reviews from apache/arrow are counting against my personal copilot credits. For my Github account, I have the setting turned off to automatically request copilot review, but on apache/arrow it does get automatically requested [e.g. 1]. I _believe_ our repo settings override personal ones here. But also, those reviews are (seemingly, Github does not make this easy to find!) being charged to my personal Github account — but the dates match up almost exactly.
I also noticed that a number of recent PRs don't seem to have the auto-review happening, which is odd (but if those contributors don't have usage/credits available in copilot, it's possible that the triggering doesn't happen?) [e.g. 2] I will say I appreciate the copilot review (it's not the best bot review I've seen, but it's not so bad these days!) but for these PRs being able to trigger it manually would at least reduce the number of times it churns unnecessarily. The cost so far has been minimal to me, but I haven't opened too many PRs. This might be just a problem with my configuration, or maybe some subtle change on Github's end? Does anyone else experience the same? -Jon [1] — https://github.com/apache/arrow/pull/55610 [2] — https://github.com/apache/arrow/pull/55599 On Mon, Jun 8, 2026 at 5:29 AM Alenka Frim <[email protected]> wrote: > > Hi, > > I agree on mixing 1. and 2. > > One more question that came to my mind: when is Copilot happy > with the state of the PR? I haven't seen Copilot give an approval > mark and am guessing the review that summarizes and does not > give any comments/suggestions is the "approval" review? > > Should we suggest in our docs that this "approval" review is what > new contributors are aiming for? > > Thank you for opening up the PR and all the work around this > Kou! > > Best, > Alenka > > V V ned., 7. jun. 2026 ob 10:37 je oseba Sutou Kouhei <[email protected]> > napisala: > > > Hi, > > > > Good point. > > > > How about mixing 1. and 2.? > > First-time contributors must use 2. and others can use 1. > > > > I've opened a PR that reflects feedback from Alenka, Raúl > > and Andrew: > > https://github.com/apache/arrow/pull/50117 > > > > > > Thanks, > > -- > > kou > > > > In <CAFhtnRwskcruBM9t4f9z1GsfQwJFLB3hEstTLseGKH7=oss...@mail.gmail.com> > > "Re: [DISCUSS] Auto GitHub Copilot review" on Sat, 6 Jun 2026 07:49:34 > > -0400, > > Andrew Lamb <[email protected]> wrote: > > > > > We have also found copilot reviews useful for initial pass after draft > > > reviews. > > > > > > If we enable copilot reviews, I think we should also document the > > > expectation for contributors responding to them. > > > For example > > > 1. Is it acceptable to ignore copilot reviews > > > 2. Should all copilot comments be addressed (explicitly responded to or > > > implemented)? > > > 3. Something else? > > > > > > Andrew > > > > > > On Fri, Jun 5, 2026 at 6:34 AM Raúl Cumplido <[email protected]> wrote: > > > > > >> Thanks Kou, after ~2 weeks testing it I find it useful for non draft > > >> PRs, on the other hand, I think we should remove it for draft PRs. > > >> > > >> When someone submits a draft PR usually means it's still not ready for > > >> review. Having Copilot adding comments for every commit feels like a > > >> waste of resources and in my opinion it generates clutter on the PR. > > >> > > >> El mié, 27 may 2026 a las 4:29, Gang Wu (<[email protected]>) escribió: > > >> > > > >> > I think GitHub Copilot is helpful at least for a preliminary review. > > >> > > > >> > On Wed, May 27, 2026 at 10:06 AM Sutou Kouhei <[email protected]> > > >> wrote: > > >> > > > > >> > > Hi, > > >> > > > > >> > > Thanks for sharing your product! It may help us but can we > > >> > > try GitHub Compilot review as the first step? Because we can > > >> > > try it without asking INFRA. If we find that GitHub Copilot > > >> > > review helps us or GitHub Copilot review will help us if > > >> > > it's more clever, let's try your product! > > >> > > > > >> > > Thanks, > > >> > > -- > > >> > > kou > > >> > > > > >> > > In < > > cajpuwma336ffrdkqbvi9076yvr7cmy79nuy5zwn-ec_vgz2...@mail.gmail.com > > >> > > > >> > > "Re: [DISCUSS] Auto GitHub Copilot review" on Mon, 25 May 2026 > > >> 16:43:19 -0500, > > >> > > Wes McKinney <[email protected]> wrote: > > >> > > > > >> > > > Aside: I have been building a self-hosted AI code review system > > that > > >> I > > >> > > > think does a better job of finding bugs in PRs than Copilot does: > > >> > > > https://www.roborev.io/. You can install and try it locally to > > >> compare > > >> > > > its reviews with Copilot side-by-side (I recommend using codex + > > >> > > > gpt-5.5 for reviews). > > >> > > > > > >> > > > It might be tricky to get Infra to create a GitHub application > > >> > > > (https://www.roborev.io/integrations/github/), but it could be > > >> worth a > > >> > > > try. Reviews generally cost $0.20 - $2 (depending on the size of > > the > > >> > > > change) if you are paying API rates for GPT-5.5, it might not be a > > >> bad > > >> > > > use of sponsorship funds. > > >> > > > > > >> > > > On Mon, May 25, 2026 at 4:35 PM Sutou Kouhei <[email protected]> > > >> wrote: > > >> > > >> > > >> > > >> Hi, > > >> > > >> > > >> > > >> In apache/arrow, we don't have enough PR review resources > > >> > > >> for now. GitHub Copilot review may help this > > >> > > >> situation. GitHub Copilot and a PR author may fix trivial > > >> > > >> problems before human reviews. > > >> > > >> > > >> > > >> I'm trying GitHub Copilot review recently for this > > >> > > >> propose. For example: > > >> > > >> https://github.com/apache/arrow/pull/49813 > > >> > > >> > > >> > > >> I start GitHub Copilot review manually for now. I received a > > >> > > >> feedback that the following workflow bothers a PR author: > > >> > > >> > > >> > > >> 1. PR author pushes > > >> > > >> 2. I trigger GitHub Copilot review half a day later because > > >> > > >> of timezone difference > > >> > > >> 3. PR author addresses review comments > > >> > > >> 4. 2. > > >> > > >> 5. 3. > > >> > > >> 6. ... > > >> > > >> > > >> > > >> > > https://github.com/apache/arrow/pull/49813#discussion_r3289753445 > > >> > > >> > > >> > > >> I thought that I may be able to improve this workflow by > > >> > > >> automating the 2. step. > > >> > > >> > > >> > > >> > > >> > > >> Recently, .asf.yaml added support for GitHub Copilot > > >> > > >> configuration: > > >> > > >> > > >> > > >> * https://github.com/apache/infrastructure-asfyaml/pull/89 > > >> > > >> * > > >> https://github.com/apache/infrastructure-asfyaml#copilot_code_review > > >> > > >> > > >> > > >> > > >> > > >> So I've enabled auto GitHub Copilot review: > > >> > > >> > > >> > > >> https://github.com/apache/arrow/pull/50023 > > >> > > >> > > >> > > >> It may help us or bother us. Could you share your insights > > >> > > >> with GitHub Copilot review? I want to reduce maintenance > > >> > > >> cost. > > >> > > >> > > >> > > >> FYI: Here are some feedback from Alenka: > > >> > > >> > > https://github.com/apache/arrow/pull/50023#issuecomment-4535261566 > > >> > > >> > > >> > > >> We may improve GitHub Copilot auto review by adding custom > > >> > > >> instructions: > > >> > > >> > > >> > > https://docs.github.com/en/copilot/how-tos/copilot-on-github/customize-copilot/add-custom-instructions/add-repository-instructions > > >> > > >> > > >> > > >> > > >> > > >> Thanks, > > >> > > >> -- > > >> > > >> kou > > >> > >
