+1 for "review_on_push: false". Does that mean review happens only when a draft PR is moved out of draft? I'd disable auto-reviewing draft PRs too if possible.
Rok On Tue, Nov 17, 2026 at 9:25 PM Nic Crane <[email protected]> wrote: > Hi Jon, > > I think this is the default setting of how it works (i.e. bill the user). > It's set in our `.asf.yaml`[1] file in the `copilot_code_review` block > which is a setting which overrides individual user settings with the repo > setting and bills it to the user[2]. > > I think you're right in that the reviews don't appear on the account of > people without Copilot enabled[3]. > > Perhaps we could update the setting so it only reviews on the initial PR > instead of every push by updating the setting `review_on_push: false` in > our `.asf.yaml` file? If we did this, then any subsequent reviews are > billed to the requester. > > Nic > > > > [1] https://github.com/apache/arrow/blob/main/.asf.yaml#L43-L46 > [2] > > https://docs.github.com/en/copilot/concepts/agent-capabilities/agent-powered-workflows-and-operations/code-review#code-review-usage-and-billing > [3] > > https://docs.github.com/en/copilot/concepts/agent-capabilities/agent-powered-workflows-and-operations/code-review#automatic-pull-request-reviews > > On Sat, 14 Nov 2026 at 08:49, Jonathan Keane <[email protected]> wrote: > > > Sorry for slightly resurrecting this thread. But something I just > > noticed is that it appears that copilot auto-reviews from apache/arrow > > are counting against my personal copilot credits. For my Github > > account, I have the setting turned off to automatically request > > copilot review, but on apache/arrow it does get automatically > > requested [e.g. 1]. I _believe_ our repo settings override personal > > ones here. But also, those reviews are (seemingly, Github does not > > make this easy to find!) being charged to my personal Github account — > > but the dates match up almost exactly. > > > > I also noticed that a number of recent PRs don't seem to have the > > auto-review happening, which is odd (but if those contributors don't > > have usage/credits available in copilot, it's possible that the > > triggering doesn't happen?) [e.g. 2] > > > > I will say I appreciate the copilot review (it's not the best bot > > review I've seen, but it's not so bad these days!) but for these PRs > > being able to trigger it manually would at least reduce the number of > > times it churns unnecessarily. The cost so far has been minimal to me, > > but I haven't opened too many PRs. > > > > This might be just a problem with my configuration, or maybe some > > subtle change on Github's end? Does anyone else experience the same? > > > > -Jon > > > > > > [1] — https://github.com/apache/arrow/pull/55610 > > [2] — https://github.com/apache/arrow/pull/55599 > > > > On Mon, Jun 8, 2026 at 5:29 AM Alenka Frim <[email protected]> > wrote: > > > > > > Hi, > > > > > > I agree on mixing 1. and 2. > > > > > > One more question that came to my mind: when is Copilot happy > > > with the state of the PR? I haven't seen Copilot give an approval > > > mark and am guessing the review that summarizes and does not > > > give any comments/suggestions is the "approval" review? > > > > > > Should we suggest in our docs that this "approval" review is what > > > new contributors are aiming for? > > > > > > Thank you for opening up the PR and all the work around this > > > Kou! > > > > > > Best, > > > Alenka > > > > > > V V ned., 7. jun. 2026 ob 10:37 je oseba Sutou Kouhei < > > [email protected]> > > > napisala: > > > > > > > Hi, > > > > > > > > Good point. > > > > > > > > How about mixing 1. and 2.? > > > > First-time contributors must use 2. and others can use 1. > > > > > > > > I've opened a PR that reflects feedback from Alenka, Raúl > > > > and Andrew: > > > > https://github.com/apache/arrow/pull/50117 > > > > > > > > > > > > Thanks, > > > > -- > > > > kou > > > > > > > > In <CAFhtnRwskcruBM9t4f9z1GsfQwJFLB3hEstTLseGKH7= > [email protected] > > > > > > > "Re: [DISCUSS] Auto GitHub Copilot review" on Sat, 6 Jun 2026 > > 07:49:34 > > > > -0400, > > > > Andrew Lamb <[email protected]> wrote: > > > > > > > > > We have also found copilot reviews useful for initial pass after > > draft > > > > > reviews. > > > > > > > > > > If we enable copilot reviews, I think we should also document the > > > > > expectation for contributors responding to them. > > > > > For example > > > > > 1. Is it acceptable to ignore copilot reviews > > > > > 2. Should all copilot comments be addressed (explicitly responded > to > > or > > > > > implemented)? > > > > > 3. Something else? > > > > > > > > > > Andrew > > > > > > > > > > On Fri, Jun 5, 2026 at 6:34 AM Raúl Cumplido <[email protected]> > > wrote: > > > > > > > > > >> Thanks Kou, after ~2 weeks testing it I find it useful for non > draft > > > > >> PRs, on the other hand, I think we should remove it for draft PRs. > > > > >> > > > > >> When someone submits a draft PR usually means it's still not ready > > for > > > > >> review. Having Copilot adding comments for every commit feels > like a > > > > >> waste of resources and in my opinion it generates clutter on the > PR. > > > > >> > > > > >> El mié, 27 may 2026 a las 4:29, Gang Wu (<[email protected]>) > > escribió: > > > > >> > > > > > >> > I think GitHub Copilot is helpful at least for a preliminary > > review. > > > > >> > > > > > >> > On Wed, May 27, 2026 at 10:06 AM Sutou Kouhei < > [email protected] > > > > > > > >> wrote: > > > > >> > > > > > > >> > > Hi, > > > > >> > > > > > > >> > > Thanks for sharing your product! It may help us but can we > > > > >> > > try GitHub Compilot review as the first step? Because we can > > > > >> > > try it without asking INFRA. If we find that GitHub Copilot > > > > >> > > review helps us or GitHub Copilot review will help us if > > > > >> > > it's more clever, let's try your product! > > > > >> > > > > > > >> > > Thanks, > > > > >> > > -- > > > > >> > > kou > > > > >> > > > > > > >> > > In < > > > > cajpuwma336ffrdkqbvi9076yvr7cmy79nuy5zwn-ec_vgz2...@mail.gmail.com > > > > >> > > > > > >> > > "Re: [DISCUSS] Auto GitHub Copilot review" on Mon, 25 May > 2026 > > > > >> 16:43:19 -0500, > > > > >> > > Wes McKinney <[email protected]> wrote: > > > > >> > > > > > > >> > > > Aside: I have been building a self-hosted AI code review > > system > > > > that > > > > >> I > > > > >> > > > think does a better job of finding bugs in PRs than Copilot > > does: > > > > >> > > > https://www.roborev.io/. You can install and try it locally > > to > > > > >> compare > > > > >> > > > its reviews with Copilot side-by-side (I recommend using > > codex + > > > > >> > > > gpt-5.5 for reviews). > > > > >> > > > > > > > >> > > > It might be tricky to get Infra to create a GitHub > application > > > > >> > > > (https://www.roborev.io/integrations/github/), but it could > > be > > > > >> worth a > > > > >> > > > try. Reviews generally cost $0.20 - $2 (depending on the > size > > of > > > > the > > > > >> > > > change) if you are paying API rates for GPT-5.5, it might > not > > be a > > > > >> bad > > > > >> > > > use of sponsorship funds. > > > > >> > > > > > > > >> > > > On Mon, May 25, 2026 at 4:35 PM Sutou Kouhei < > > [email protected]> > > > > >> wrote: > > > > >> > > >> > > > > >> > > >> Hi, > > > > >> > > >> > > > > >> > > >> In apache/arrow, we don't have enough PR review resources > > > > >> > > >> for now. GitHub Copilot review may help this > > > > >> > > >> situation. GitHub Copilot and a PR author may fix trivial > > > > >> > > >> problems before human reviews. > > > > >> > > >> > > > > >> > > >> I'm trying GitHub Copilot review recently for this > > > > >> > > >> propose. For example: > > > > >> > > >> https://github.com/apache/arrow/pull/49813 > > > > >> > > >> > > > > >> > > >> I start GitHub Copilot review manually for now. I received > a > > > > >> > > >> feedback that the following workflow bothers a PR author: > > > > >> > > >> > > > > >> > > >> 1. PR author pushes > > > > >> > > >> 2. I trigger GitHub Copilot review half a day later because > > > > >> > > >> of timezone difference > > > > >> > > >> 3. PR author addresses review comments > > > > >> > > >> 4. 2. > > > > >> > > >> 5. 3. > > > > >> > > >> 6. ... > > > > >> > > >> > > > > >> > > >> > > > > https://github.com/apache/arrow/pull/49813#discussion_r3289753445 > > > > >> > > >> > > > > >> > > >> I thought that I may be able to improve this workflow by > > > > >> > > >> automating the 2. step. > > > > >> > > >> > > > > >> > > >> > > > > >> > > >> Recently, .asf.yaml added support for GitHub Copilot > > > > >> > > >> configuration: > > > > >> > > >> > > > > >> > > >> * https://github.com/apache/infrastructure-asfyaml/pull/89 > > > > >> > > >> * > > > > >> > > https://github.com/apache/infrastructure-asfyaml#copilot_code_review > > > > >> > > >> > > > > >> > > >> > > > > >> > > >> So I've enabled auto GitHub Copilot review: > > > > >> > > >> > > > > >> > > >> https://github.com/apache/arrow/pull/50023 > > > > >> > > >> > > > > >> > > >> It may help us or bother us. Could you share your insights > > > > >> > > >> with GitHub Copilot review? I want to reduce maintenance > > > > >> > > >> cost. > > > > >> > > >> > > > > >> > > >> FYI: Here are some feedback from Alenka: > > > > >> > > >> > > > > https://github.com/apache/arrow/pull/50023#issuecomment-4535261566 > > > > >> > > >> > > > > >> > > >> We may improve GitHub Copilot auto review by adding custom > > > > >> > > >> instructions: > > > > >> > > >> > > > > >> > > > > > > > https://docs.github.com/en/copilot/how-tos/copilot-on-github/customize-copilot/add-custom-instructions/add-repository-instructions > > > > >> > > >> > > > > >> > > >> > > > > >> > > >> Thanks, > > > > >> > > >> -- > > > > >> > > >> kou > > > > >> > > > > > > >
