+1 for "review_on_push: false".
Does that mean review happens only when a draft PR is moved out of draft?
I'd disable auto-reviewing draft PRs too if possible.

Rok


On Tue, Nov 17, 2026 at 9:25 PM Nic Crane <[email protected]> wrote:

> Hi Jon,
>
> I think this is the default setting of how it works (i.e. bill the user).
> It's set in our `.asf.yaml`[1] file in the `copilot_code_review` block
> which is a setting which overrides individual user settings with the repo
> setting and bills it to the user[2].
>
> I think you're right in that the reviews don't appear on the account of
> people without Copilot enabled[3].
>
> Perhaps we could update the setting so it only reviews on the initial PR
> instead of every push by updating the setting `review_on_push: false` in
> our `.asf.yaml` file?  If we did this, then any subsequent reviews are
> billed to the requester.
>
> Nic
>
>
>
> [1] https://github.com/apache/arrow/blob/main/.asf.yaml#L43-L46
> [2]
>
> https://docs.github.com/en/copilot/concepts/agent-capabilities/agent-powered-workflows-and-operations/code-review#code-review-usage-and-billing
> [3]
>
> https://docs.github.com/en/copilot/concepts/agent-capabilities/agent-powered-workflows-and-operations/code-review#automatic-pull-request-reviews
>
> On Sat, 14 Nov 2026 at 08:49, Jonathan Keane <[email protected]> wrote:
>
> > Sorry for slightly resurrecting this thread. But something I just
> > noticed is that it appears that copilot auto-reviews from apache/arrow
> > are counting against my personal copilot credits. For my Github
> > account, I have the setting turned off to automatically request
> > copilot review, but on apache/arrow it does get automatically
> > requested [e.g. 1]. I _believe_ our repo settings override personal
> > ones here. But also, those reviews are (seemingly, Github does not
> > make this easy to find!) being charged to my personal Github account —
> > but the dates match up almost exactly.
> >
> > I also noticed that a number of recent PRs don't seem to have the
> > auto-review happening, which is odd (but if those contributors don't
> > have usage/credits available in copilot, it's possible that the
> > triggering doesn't happen?) [e.g. 2]
> >
> > I will say I appreciate the copilot review (it's not the best bot
> > review I've seen, but it's not so bad these days!) but for these PRs
> > being able to trigger it manually would at least reduce the number of
> > times it churns unnecessarily. The cost so far has been minimal to me,
> > but I haven't opened too many PRs.
> >
> > This might be just a problem with my configuration, or maybe some
> > subtle change on Github's end? Does anyone else experience the same?
> >
> > -Jon
> >
> >
> > [1] — https://github.com/apache/arrow/pull/55610
> > [2] — https://github.com/apache/arrow/pull/55599
> >
> > On Mon, Jun 8, 2026 at 5:29 AM Alenka Frim <[email protected]>
> wrote:
> > >
> > > Hi,
> > >
> > > I agree on mixing 1. and 2.
> > >
> > > One more question that came to my mind: when is Copilot happy
> > > with the state of the PR? I haven't seen Copilot give an approval
> > > mark and am guessing the review that summarizes and does not
> > > give any comments/suggestions is the "approval" review?
> > >
> > > Should we suggest in our docs that this "approval" review is what
> > > new contributors are aiming for?
> > >
> > > Thank you for opening up the PR and all the work around this
> > > Kou!
> > >
> > > Best,
> > > Alenka
> > >
> > > V V ned., 7. jun. 2026 ob 10:37 je oseba Sutou Kouhei <
> > [email protected]>
> > > napisala:
> > >
> > > > Hi,
> > > >
> > > > Good point.
> > > >
> > > > How about mixing 1. and 2.?
> > > > First-time contributors must use 2. and others can use 1.
> > > >
> > > > I've opened a PR that reflects feedback from Alenka, Raúl
> > > > and Andrew:
> > > > https://github.com/apache/arrow/pull/50117
> > > >
> > > >
> > > > Thanks,
> > > > --
> > > > kou
> > > >
> > > > In <CAFhtnRwskcruBM9t4f9z1GsfQwJFLB3hEstTLseGKH7=
> [email protected]
> > >
> > > >   "Re: [DISCUSS] Auto GitHub Copilot review" on Sat, 6 Jun 2026
> > 07:49:34
> > > > -0400,
> > > >   Andrew Lamb <[email protected]> wrote:
> > > >
> > > > > We have also found copilot reviews useful for initial pass after
> > draft
> > > > > reviews.
> > > > >
> > > > > If we enable copilot reviews, I think we should also document the
> > > > > expectation for contributors responding to them.
> > > > > For example
> > > > > 1. Is it acceptable to ignore copilot reviews
> > > > > 2. Should all copilot comments be addressed (explicitly responded
> to
> > or
> > > > > implemented)?
> > > > > 3. Something else?
> > > > >
> > > > > Andrew
> > > > >
> > > > > On Fri, Jun 5, 2026 at 6:34 AM Raúl Cumplido <[email protected]>
> > wrote:
> > > > >
> > > > >> Thanks Kou, after ~2 weeks testing it I find it useful for non
> draft
> > > > >> PRs, on the other hand, I think we should remove it for draft PRs.
> > > > >>
> > > > >> When someone submits a draft PR usually means it's still not ready
> > for
> > > > >> review. Having Copilot adding comments for every commit feels
> like a
> > > > >> waste of resources and in my opinion it generates clutter on the
> PR.
> > > > >>
> > > > >> El mié, 27 may 2026 a las 4:29, Gang Wu (<[email protected]>)
> > escribió:
> > > > >> >
> > > > >> > I think GitHub Copilot is helpful at least for a preliminary
> > review.
> > > > >> >
> > > > >> > On Wed, May 27, 2026 at 10:06 AM Sutou Kouhei <
> [email protected]
> > >
> > > > >> wrote:
> > > > >> > >
> > > > >> > > Hi,
> > > > >> > >
> > > > >> > > Thanks for sharing your product! It may help us but can we
> > > > >> > > try GitHub Compilot review as the first step? Because we can
> > > > >> > > try it without asking INFRA. If we find that GitHub Copilot
> > > > >> > > review helps us or GitHub Copilot review will help us if
> > > > >> > > it's more clever, let's try your product!
> > > > >> > >
> > > > >> > > Thanks,
> > > > >> > > --
> > > > >> > > kou
> > > > >> > >
> > > > >> > > In <
> > > > cajpuwma336ffrdkqbvi9076yvr7cmy79nuy5zwn-ec_vgz2...@mail.gmail.com
> > > > >> >
> > > > >> > >   "Re: [DISCUSS] Auto GitHub Copilot review" on Mon, 25 May
> 2026
> > > > >> 16:43:19 -0500,
> > > > >> > >   Wes McKinney <[email protected]> wrote:
> > > > >> > >
> > > > >> > > > Aside: I have been building a self-hosted AI code review
> > system
> > > > that
> > > > >> I
> > > > >> > > > think does a better job of finding bugs in PRs than Copilot
> > does:
> > > > >> > > > https://www.roborev.io/. You can install and try it locally
> > to
> > > > >> compare
> > > > >> > > > its reviews with Copilot side-by-side (I recommend using
> > codex +
> > > > >> > > > gpt-5.5 for reviews).
> > > > >> > > >
> > > > >> > > > It might be tricky to get Infra to create a GitHub
> application
> > > > >> > > > (https://www.roborev.io/integrations/github/), but it could
> > be
> > > > >> worth a
> > > > >> > > > try. Reviews generally cost $0.20 - $2 (depending on the
> size
> > of
> > > > the
> > > > >> > > > change) if you are paying API rates for GPT-5.5, it might
> not
> > be a
> > > > >> bad
> > > > >> > > > use of sponsorship funds.
> > > > >> > > >
> > > > >> > > > On Mon, May 25, 2026 at 4:35 PM Sutou Kouhei <
> > [email protected]>
> > > > >> wrote:
> > > > >> > > >>
> > > > >> > > >> Hi,
> > > > >> > > >>
> > > > >> > > >> In apache/arrow, we don't have enough PR review resources
> > > > >> > > >> for now. GitHub Copilot review may help this
> > > > >> > > >> situation. GitHub Copilot and a PR author may fix trivial
> > > > >> > > >> problems before human reviews.
> > > > >> > > >>
> > > > >> > > >> I'm trying GitHub Copilot review recently for this
> > > > >> > > >> propose. For example:
> > > > >> > > >> https://github.com/apache/arrow/pull/49813
> > > > >> > > >>
> > > > >> > > >> I start GitHub Copilot review manually for now. I received
> a
> > > > >> > > >> feedback that the following workflow bothers a PR author:
> > > > >> > > >>
> > > > >> > > >> 1. PR author pushes
> > > > >> > > >> 2. I trigger GitHub Copilot review half a day later because
> > > > >> > > >>    of timezone difference
> > > > >> > > >> 3. PR author addresses review comments
> > > > >> > > >> 4. 2.
> > > > >> > > >> 5. 3.
> > > > >> > > >> 6. ...
> > > > >> > > >>
> > > > >> > > >>
> > > > https://github.com/apache/arrow/pull/49813#discussion_r3289753445
> > > > >> > > >>
> > > > >> > > >> I thought that I may be able to improve this workflow by
> > > > >> > > >> automating the 2. step.
> > > > >> > > >>
> > > > >> > > >>
> > > > >> > > >> Recently, .asf.yaml added support for GitHub Copilot
> > > > >> > > >> configuration:
> > > > >> > > >>
> > > > >> > > >> * https://github.com/apache/infrastructure-asfyaml/pull/89
> > > > >> > > >> *
> > > > >>
> > https://github.com/apache/infrastructure-asfyaml#copilot_code_review
> > > > >> > > >>
> > > > >> > > >>
> > > > >> > > >> So I've enabled auto GitHub Copilot review:
> > > > >> > > >>
> > > > >> > > >> https://github.com/apache/arrow/pull/50023
> > > > >> > > >>
> > > > >> > > >> It may help us or bother us. Could you share your insights
> > > > >> > > >> with GitHub Copilot review? I want to reduce maintenance
> > > > >> > > >> cost.
> > > > >> > > >>
> > > > >> > > >> FYI: Here are some feedback from Alenka:
> > > > >> > > >>
> > > > https://github.com/apache/arrow/pull/50023#issuecomment-4535261566
> > > > >> > > >>
> > > > >> > > >> We may improve GitHub Copilot auto review by adding custom
> > > > >> > > >> instructions:
> > > > >> > > >>
> > > > >>
> > > >
> >
> https://docs.github.com/en/copilot/how-tos/copilot-on-github/customize-copilot/add-custom-instructions/add-repository-instructions
> > > > >> > > >>
> > > > >> > > >>
> > > > >> > > >> Thanks,
> > > > >> > > >> --
> > > > >> > > >> kou
> > > > >>
> > > >
> >
>

Reply via email to