Re: [VOTE] Code Review Response-time SLO

[Jean-Baptiste Onofré]

Yeah, that sounds reasonable to me. We just have to provide an update
about the review status, not necessary the review itself ;)

Regards
JB

On 07/06/2018 16:25, Kenneth Knowles wrote:
> Yea, this was what I mean by my question "What is the action when an SLO
> is missed?"
> 
> For example, the way we implement could just be a line in the
> contribution guide like this:
> 
> "We aspire to respond quickly to all pull requests, even if it can take
> some time to review them. If you have not heard a response in 
> then please @mention someone again, try someone else, or reach out on
> Slack or dev@."
> 
> It does sound a bit overly technical to say we have an SLO of 
> with an SLA that you should ask again.
> 
> Kenn
> 
> On Thu, Jun 7, 2018 at 7:16 AM Jean-Baptiste Onofré  > wrote:
> 
> Agree.
> 
> Having a best effort is fine but not a strong commitment, because, like
> in any Apache project, lot of people works on the project in their spare
> time, or even during business time,  with lower priority.
> 
> Regards
> JB
> 
> On 07/06/2018 16:11, Thomas Weise wrote:
> > I like the idea of speeding up code review and promptly responding to
> > PRs, since this is more motivating to the contributors, especially
> those
> > that are not committers.
> >
> > However, the notion of SLO or target response time is something that
> > works within a company with dedicated resources. It may not be the
> right
> > approach for a diverse community with contributors of various
> > backgrounds and time commitments. Establishing such policy as
> > expectation at the project level seems problematic.
> >
> > Perhaps this can be a guideline or recommendation, but I don't see how
> > it can be measured or enforced at the project level.
> >
> > Thanks,
> > Thomas
> >
> >
> > On Thu, Jun 7, 2018 at 6:09 AM, Etienne Chauchot
> mailto:echauc...@apache.org>
> > >> wrote:
> >
> >     Yes, I agree with that, I thought it was the delay to start
> the review.
> >     +1 then on the delay to answer
> >
> >     Etienne
> >
> >     Le jeudi 07 juin 2018 à 11:02 +0100, Reuven Lax a écrit :
> >>     However I think it's reasonable to expect a response within 3
> >>     days, even if that response is not an actual review. For
> instance,
> >>     the response could be "I'm extremely busy right now, and it will
> >>     take another week for me to get to this."
> >>
> >>     On Thu, Jun 7, 2018, 10:40 AM Etienne Chauchot
> >>     mailto:echauc...@apache.org>
> >> wrote:
> >>>     +1 on the purpose, but I think 3 business days for proposal
> 1 are
> >>>     over-optimistic.
> >>>     As an example I saw PRs taking weeks before the review started.
> >>>     IMHO, I think you should consider raising up the 3 days bar
> a bit.
> >>>
> >>>     Best
> >>>     Etienne
> >>>
> >>>     Le mardi 05 juin 2018 à 15:04 -0700, Andrew Pilloud a écrit :
>      The one thing that seems a little odd to me is using business
>      days. The beam community spans across company and country
>      boundaries, which makes business days a slightly ambiguous
>      concept. Not everyone's holidays and weekends line up. I also
>      agree with Alan, we need to build in time to disconnect. If
> this
>      is opt-in, it might make sense to collect working hours and
> make
>      it easy to opt out if you aren't going to be available for an
>      extended period. Until recently we had a page listing committer
>      companies and time zones. Bringing that back would be a
> good option.
> 
>      Andrew
> 
>      On Tue, Jun 5, 2018 at 2:29 PM Huygaa Batsaikhan
>      mailto:bat...@google.com>
> >> wrote:
> >     Thanks Ken for pointing out that vote is a two step process. I
> >     will write an extended written doc about pros and cons of the
> >     SLO and any related topics. In the meantime, feel free to use
> >     this thread to express any suggestions and concerns.
> >
> >     Thanks, Huygaa
> >
> >
> 
> -- 
> Jean-Baptiste Onofré
> jbono...@apache.org 
> http://blog.nanthrax.net
> Talend - http://www.talend.com
> 

-- 
Jean-Baptiste Onofré
jbono...@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com

Re: [VOTE] Code Review Response-time SLO

[Kenneth Knowles]

Yea, this was what I mean by my question "What is the action when an SLO is
missed?"

For example, the way we implement could just be a line in the contribution
guide like this:

"We aspire to respond quickly to all pull requests, even if it can take
some time to review them. If you have not heard a response in 
then please @mention someone again, try someone else, or reach out on Slack
or dev@."

It does sound a bit overly technical to say we have an SLO of 
with an SLA that you should ask again.

Kenn

On Thu, Jun 7, 2018 at 7:16 AM Jean-Baptiste Onofré  wrote:

> Agree.
>
> Having a best effort is fine but not a strong commitment, because, like
> in any Apache project, lot of people works on the project in their spare
> time, or even during business time,  with lower priority.
>
> Regards
> JB
>
> On 07/06/2018 16:11, Thomas Weise wrote:
> > I like the idea of speeding up code review and promptly responding to
> > PRs, since this is more motivating to the contributors, especially those
> > that are not committers.
> >
> > However, the notion of SLO or target response time is something that
> > works within a company with dedicated resources. It may not be the right
> > approach for a diverse community with contributors of various
> > backgrounds and time commitments. Establishing such policy as
> > expectation at the project level seems problematic.
> >
> > Perhaps this can be a guideline or recommendation, but I don't see how
> > it can be measured or enforced at the project level.
> >
> > Thanks,
> > Thomas
> >
> >
> > On Thu, Jun 7, 2018 at 6:09 AM, Etienne Chauchot  > > wrote:
> >
> > Yes, I agree with that, I thought it was the delay to start the
> review.
> > +1 then on the delay to answer
> >
> > Etienne
> >
> > Le jeudi 07 juin 2018 à 11:02 +0100, Reuven Lax a écrit :
> >> However I think it's reasonable to expect a response within 3
> >> days, even if that response is not an actual review. For instance,
> >> the response could be "I'm extremely busy right now, and it will
> >> take another week for me to get to this."
> >>
> >> On Thu, Jun 7, 2018, 10:40 AM Etienne Chauchot
> >> mailto:echauc...@apache.org>> wrote:
> >>> +1 on the purpose, but I think 3 business days for proposal 1 are
> >>> over-optimistic.
> >>> As an example I saw PRs taking weeks before the review started.
> >>> IMHO, I think you should consider raising up the 3 days bar a bit.
> >>>
> >>> Best
> >>> Etienne
> >>>
> >>> Le mardi 05 juin 2018 à 15:04 -0700, Andrew Pilloud a écrit :
>  The one thing that seems a little odd to me is using business
>  days. The beam community spans across company and country
>  boundaries, which makes business days a slightly ambiguous
>  concept. Not everyone's holidays and weekends line up. I also
>  agree with Alan, we need to build in time to disconnect. If this
>  is opt-in, it might make sense to collect working hours and make
>  it easy to opt out if you aren't going to be available for an
>  extended period. Until recently we had a page listing committer
>  companies and time zones. Bringing that back would be a good
> option.
> 
>  Andrew
> 
>  On Tue, Jun 5, 2018 at 2:29 PM Huygaa Batsaikhan
>  mailto:bat...@google.com>> wrote:
> > Thanks Ken for pointing out that vote is a two step process. I
> > will write an extended written doc about pros and cons of the
> > SLO and any related topics. In the meantime, feel free to use
> > this thread to express any suggestions and concerns.
> >
> > Thanks, Huygaa
> >
> >
>
> --
> Jean-Baptiste Onofré
> jbono...@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>

Re: [VOTE] Code Review Response-time SLO

[Jean-Baptiste Onofré]

Agree.

Having a best effort is fine but not a strong commitment, because, like
in any Apache project, lot of people works on the project in their spare
time, or even during business time,  with lower priority.

Regards
JB

On 07/06/2018 16:11, Thomas Weise wrote:
> I like the idea of speeding up code review and promptly responding to
> PRs, since this is more motivating to the contributors, especially those
> that are not committers.
> 
> However, the notion of SLO or target response time is something that
> works within a company with dedicated resources. It may not be the right
> approach for a diverse community with contributors of various
> backgrounds and time commitments. Establishing such policy as
> expectation at the project level seems problematic.
> 
> Perhaps this can be a guideline or recommendation, but I don't see how
> it can be measured or enforced at the project level.
> 
> Thanks,
> Thomas
> 
> 
> On Thu, Jun 7, 2018 at 6:09 AM, Etienne Chauchot  > wrote:
> 
> Yes, I agree with that, I thought it was the delay to start the review.
> +1 then on the delay to answer
> 
> Etienne
> 
> Le jeudi 07 juin 2018 à 11:02 +0100, Reuven Lax a écrit :
>> However I think it's reasonable to expect a response within 3
>> days, even if that response is not an actual review. For instance,
>> the response could be "I'm extremely busy right now, and it will
>> take another week for me to get to this."
>>
>> On Thu, Jun 7, 2018, 10:40 AM Etienne Chauchot
>> mailto:echauc...@apache.org>> wrote:
>>> +1 on the purpose, but I think 3 business days for proposal 1 are
>>> over-optimistic.
>>> As an example I saw PRs taking weeks before the review started.
>>> IMHO, I think you should consider raising up the 3 days bar a bit.
>>>
>>> Best
>>> Etienne
>>>
>>> Le mardi 05 juin 2018 à 15:04 -0700, Andrew Pilloud a écrit :
 The one thing that seems a little odd to me is using business
 days. The beam community spans across company and country
 boundaries, which makes business days a slightly ambiguous
 concept. Not everyone's holidays and weekends line up. I also
 agree with Alan, we need to build in time to disconnect. If this
 is opt-in, it might make sense to collect working hours and make
 it easy to opt out if you aren't going to be available for an
 extended period. Until recently we had a page listing committer
 companies and time zones. Bringing that back would be a good option.

 Andrew

 On Tue, Jun 5, 2018 at 2:29 PM Huygaa Batsaikhan
 mailto:bat...@google.com>> wrote:
> Thanks Ken for pointing out that vote is a two step process. I
> will write an extended written doc about pros and cons of the
> SLO and any related topics. In the meantime, feel free to use
> this thread to express any suggestions and concerns.
>
> Thanks, Huygaa
> 
> 

-- 
Jean-Baptiste Onofré
jbono...@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com

Re: [VOTE] Code Review Response-time SLO

[Thomas Weise]

I like the idea of speeding up code review and promptly responding to PRs,
since this is more motivating to the contributors, especially those that
are not committers.

However, the notion of SLO or target response time is something that works
within a company with dedicated resources. It may not be the right approach
for a diverse community with contributors of various backgrounds and time
commitments. Establishing such policy as expectation at the project level
seems problematic.

Perhaps this can be a guideline or recommendation, but I don't see how it
can be measured or enforced at the project level.

Thanks,
Thomas


On Thu, Jun 7, 2018 at 6:09 AM, Etienne Chauchot 
wrote:

> Yes, I agree with that, I thought it was the delay to start the review.
> +1 then on the delay to answer
>
> Etienne
>
> Le jeudi 07 juin 2018 à 11:02 +0100, Reuven Lax a écrit :
>
> However I think it's reasonable to expect a response within 3 days, even
> if that response is not an actual review. For instance, the response could
> be "I'm extremely busy right now, and it will take another week for me to
> get to this."
>
> On Thu, Jun 7, 2018, 10:40 AM Etienne Chauchot 
> wrote:
>
> +1 on the purpose, but I think 3 business days for proposal 1 are
> over-optimistic.
> As an example I saw PRs taking weeks before the review started.
> IMHO, I think you should consider raising up the 3 days bar a bit.
>
> Best
> Etienne
>
> Le mardi 05 juin 2018 à 15:04 -0700, Andrew Pilloud a écrit :
>
> The one thing that seems a little odd to me is using business days. The
> beam community spans across company and country boundaries, which makes
> business days a slightly ambiguous concept. Not everyone's holidays and
> weekends line up. I also agree with Alan, we need to build in time to
> disconnect. If this is opt-in, it might make sense to collect working hours
> and make it easy to opt out if you aren't going to be available for an
> extended period. Until recently we had a page listing committer companies
> and time zones. Bringing that back would be a good option.
>
> Andrew
>
> On Tue, Jun 5, 2018 at 2:29 PM Huygaa Batsaikhan 
> wrote:
>
> Thanks Ken for pointing out that vote is a two step process. I will write
> an extended written doc about pros and cons of the SLO and any related
> topics. In the meantime, feel free to use this thread to express any
> suggestions and concerns.
>
> Thanks, Huygaa
>
>

Re: [VOTE] Code Review Response-time SLO

[Etienne Chauchot]

Yes, I agree with that, I thought it was the delay to start the review. +1 then 
on the delay to answer
Etienne
Le jeudi 07 juin 2018 à 11:02 +0100, Reuven Lax a écrit :
> However I think it's reasonable to expect a response within 3 days, even if 
> that response is not an actual review. For
> instance, the response could be "I'm extremely busy right now, and it will 
> take another week for me to get to this."
> On Thu, Jun 7, 2018, 10:40 AM Etienne Chauchot  wrote:
> > +1 on the purpose, but I think 3 business days for proposal 1 are 
> > over-optimistic. As an example I saw PRs taking
> > weeks before the review started. IMHO, I think you should consider raising 
> > up the 3 days bar a bit.
> > BestEtienne
> > Le mardi 05 juin 2018 à 15:04 -0700, Andrew Pilloud a écrit :
> > > The one thing that seems a little odd to me is using business days. The 
> > > beam community spans across company and
> > > country boundaries, which makes business days a slightly ambiguous 
> > > concept. Not everyone's holidays and weekends
> > > line up. I also agree with Alan, we need to build in time to disconnect. 
> > > If this is opt-in, it might make sense to
> > > collect working hours and make it easy to opt out if you aren't going to 
> > > be available for an extended period.
> > > Until recently we had a page listing committer companies and time zones. 
> > > Bringing that back would be a good
> > > option.
> > > Andrew
> > > On Tue, Jun 5, 2018 at 2:29 PM Huygaa Batsaikhan  
> > > wrote:
> > > > Thanks Ken for pointing out that vote is a two step process. I will 
> > > > write an extended written doc about pros and
> > > > cons of the SLO and any related topics. In the meantime, feel free to 
> > > > use this thread to express any suggestions
> > > > and concerns.
> > > > Thanks, Huygaa

Re: [VOTE] Code Review Response-time SLO

[Reuven Lax]

However I think it's reasonable to expect a response within 3 days, even if
that response is not an actual review. For instance, the response could be
"I'm extremely busy right now, and it will take another week for me to get
to this."

On Thu, Jun 7, 2018, 10:40 AM Etienne Chauchot  wrote:

> +1 on the purpose, but I think 3 business days for proposal 1 are
> over-optimistic.
> As an example I saw PRs taking weeks before the review started.
> IMHO, I think you should consider raising up the 3 days bar a bit.
>
> Best
> Etienne
>
> Le mardi 05 juin 2018 à 15:04 -0700, Andrew Pilloud a écrit :
>
> The one thing that seems a little odd to me is using business days. The
> beam community spans across company and country boundaries, which makes
> business days a slightly ambiguous concept. Not everyone's holidays and
> weekends line up. I also agree with Alan, we need to build in time to
> disconnect. If this is opt-in, it might make sense to collect working hours
> and make it easy to opt out if you aren't going to be available for an
> extended period. Until recently we had a page listing committer companies
> and time zones. Bringing that back would be a good option.
>
> Andrew
>
> On Tue, Jun 5, 2018 at 2:29 PM Huygaa Batsaikhan 
> wrote:
>
> Thanks Ken for pointing out that vote is a two step process. I will write
> an extended written doc about pros and cons of the SLO and any related
> topics. In the meantime, feel free to use this thread to express any
> suggestions and concerns.
>
> Thanks, Huygaa
>
>

Re: [VOTE] Code Review Response-time SLO

[Etienne Chauchot]

+1 on the purpose, but I think 3 business days for proposal 1 are 
over-optimistic. As an example I saw PRs taking weeks
before the review started. IMHO, I think you should consider raising up the 3 
days bar a bit.
BestEtienne
Le mardi 05 juin 2018 à 15:04 -0700, Andrew Pilloud a écrit :
> The one thing that seems a little odd to me is using business days. The beam 
> community spans across company and
> country boundaries, which makes business days a slightly ambiguous concept. 
> Not everyone's holidays and weekends line
> up. I also agree with Alan, we need to build in time to disconnect. If this 
> is opt-in, it might make sense to collect
> working hours and make it easy to opt out if you aren't going to be available 
> for an extended period. Until recently
> we had a page listing committer companies and time zones. Bringing that back 
> would be a good option.
> Andrew
> On Tue, Jun 5, 2018 at 2:29 PM Huygaa Batsaikhan  wrote:
> > Thanks Ken for pointing out that vote is a two step process. I will write 
> > an extended written doc about pros and
> > cons of the SLO and any related topics. In the meantime, feel free to use 
> > this thread to express any suggestions and
> > concerns.
> > Thanks, Huygaa

Re: [VOTE] Code Review Response-time SLO

[Andrew Pilloud]

The one thing that seems a little odd to me is using business days. The
beam community spans across company and country boundaries, which makes
business days a slightly ambiguous concept. Not everyone's holidays and
weekends line up. I also agree with Alan, we need to build in time to
disconnect. If this is opt-in, it might make sense to collect working hours
and make it easy to opt out if you aren't going to be available for an
extended period. Until recently we had a page listing committer companies
and time zones. Bringing that back would be a good option.

Andrew

On Tue, Jun 5, 2018 at 2:29 PM Huygaa Batsaikhan  wrote:

> Thanks Ken for pointing out that vote is a two step process. I will write
> an extended written doc about pros and cons of the SLO and any related
> topics. In the meantime, feel free to use this thread to express any
> suggestions and concerns.
>
> Thanks, Huygaa
>

Re: [VOTE] Code Review Response-time SLO

[Huygaa Batsaikhan]

Thanks Ken for pointing out that vote is a two step process. I will write
an extended written doc about pros and cons of the SLO and any related
topics. In the meantime, feel free to use this thread to express any
suggestions and concerns.

Thanks, Huygaa

Re: [VOTE] Code Review Response-time SLO

[Kenneth Knowles]

I love that you are pushing this forward. Just a note on mailing list
practice - I think we might want a round of discussion before declaring a
vote. A vote is a formal thing and this seems a bit more open-ended still.

Couple of thoughts:

- What is the action when an SLO is missed?
- Contributors are definitely encouraged to volunteer for initial reviews.
Helping the community like that is great, and something I'd look at when
considering someone as a potential committer.

Kenn

On Tue, Jun 5, 2018 at 1:20 PM Alan Myrvold  wrote:

> Proposal 1: +1
> Proposal 2: 0
>
> Additional comments:
> Does this apply to committers only, or are contributors encourage to
> volunteer as an additional reviewer?
> I like the idea of documenting who is keen to review which area, plus
> reviewer availability.
> From a contributor standpoint, it is worth avoiding long delays; perhaps
> having a dashboard or email of reviews that are awaiting feedback?
> Will there be a document to propose tooling?
> The 24 hours appears to be 7 days a week, 365/6 days a year? It would be
> good to build in time for everyone to disconnect.
>
> On Tue, Jun 5, 2018 at 1:07 PM Scott Wegner  wrote:
>
>> Proposal 1: +0
>> Proposal 2: +1
>>
>> Additional Comments:
>> I like the idea of providing a clear expectation to contributors on how
>> promptly they can expect code review feedback. From your proposal, my main
>> feedback would to prefer a single goal instead of two. Having a single goal
>> seems simpler when choosing a code reviewer ("PersonA might have more
>> context on my change, but PersonB will give quicker feedback.."). Multiple
>> tiers of commitment also feels like it implies level of commitment to the
>> community ("PersonB must be more committed to Beam than PersonA because
>> they agreed to faster code reviews").
>>
>> There is a balance to strike: we should provide prompt feedback and clear
>> expectations for code contributors, but we shouldn't set unreasonable
>> targets for code reviewers. I'd be interested in hearing from others
>> whether 24-hours seems unreasonable.
>>
>> I think that we should also make it clear with any policy that opting-in
>> is optional and not required to be part of the Beam community. There are
>> many ways to contribute to Beam, and opting-in to reviewing code
>> contributions is just one of them.
>>
>> On Mon, Jun 4, 2018 at 3:20 PM Huygaa Batsaikhan 
>> wrote:
>>
>>> Proposal 1: +1
>>> Proposal 2: +1
>>> Additional Comments: This is an example vote
>>>
>>> On Mon, Jun 4, 2018 at 3:15 PM Huygaa Batsaikhan 
>>> wrote:
>>>
 A few months ago, Reuven sent out an email
 
 about improvements to Beam's code review process. Because the email covered
 multiple issues, we did not really dig deep into each of them. One of the
 suggestions was to agree on a code review response turnaround time (SLO
 ). Here is the
 direct quote:

 It would be great if we could agree on a response-time SLA for Beam code
 reviews. The response might be “I am unable to do the review until next
 week,” however even that is better than getting no response.


 All the comments on the original thread supported having an agreed upon
 SLO. Therefore, I would like to discuss possible response-time SLO and
 finalize it within this thread. For the purpose of this discussion, let's
 put aside related topics such as the need of tooling support like PR
 dashboard or reviewer availability for future discussions.

 *My proposals*

 *Proposal 1*
 I propose having a *Default* review response time as *3 business days*.
 This aligns with the frequency we consider most developers are checking the
 dev list. My reasoning is, if one is checking the dev list, they could also
 check their PR review queue.

 *Proposal 2*
 I propose having an *Opt-in* review response time as *24 hours*.
 Contributors are happy when reviewers respond swiftly to their PRs.
 Specially, when we are making multiple small changes to Beam, waiting for
 even a few days is frustrating. I understand that not all the reviewers can
 review PRs daily. However, if some of us can incorporate half an hour of
 beam review to our schedule, it could improve contributors' experience
 drastically. Therefore, I suggest us having opt-in response time of 24
 hours. We can discuss how we can communicate this SLO to contributors and
 reviewers in a separate thread.

 Please vote on these 2 proposals and propose any other solutions using
 within this template:

 Template:
 Proposal 1: <+-1> 
 Proposal 2: <+-1> 
 Additional Comments: 

 Example answer:
 Proposal 1: +1 Great idea
 Proposal 2: +1
 Additional Comments: I have this 

Re: [VOTE] Code Review Response-time SLO

[Alan Myrvold]

Proposal 1: +1
Proposal 2: 0

Additional comments:
Does this apply to committers only, or are contributors encourage to
volunteer as an additional reviewer?
I like the idea of documenting who is keen to review which area, plus
reviewer availability.
>From a contributor standpoint, it is worth avoiding long delays; perhaps
having a dashboard or email of reviews that are awaiting feedback?
Will there be a document to propose tooling?
The 24 hours appears to be 7 days a week, 365/6 days a year? It would be
good to build in time for everyone to disconnect.

On Tue, Jun 5, 2018 at 1:07 PM Scott Wegner  wrote:

> Proposal 1: +0
> Proposal 2: +1
>
> Additional Comments:
> I like the idea of providing a clear expectation to contributors on how
> promptly they can expect code review feedback. From your proposal, my main
> feedback would to prefer a single goal instead of two. Having a single goal
> seems simpler when choosing a code reviewer ("PersonA might have more
> context on my change, but PersonB will give quicker feedback.."). Multiple
> tiers of commitment also feels like it implies level of commitment to the
> community ("PersonB must be more committed to Beam than PersonA because
> they agreed to faster code reviews").
>
> There is a balance to strike: we should provide prompt feedback and clear
> expectations for code contributors, but we shouldn't set unreasonable
> targets for code reviewers. I'd be interested in hearing from others
> whether 24-hours seems unreasonable.
>
> I think that we should also make it clear with any policy that opting-in
> is optional and not required to be part of the Beam community. There are
> many ways to contribute to Beam, and opting-in to reviewing code
> contributions is just one of them.
>
> On Mon, Jun 4, 2018 at 3:20 PM Huygaa Batsaikhan 
> wrote:
>
>> Proposal 1: +1
>> Proposal 2: +1
>> Additional Comments: This is an example vote
>>
>> On Mon, Jun 4, 2018 at 3:15 PM Huygaa Batsaikhan 
>> wrote:
>>
>>> A few months ago, Reuven sent out an email
>>> 
>>> about improvements to Beam's code review process. Because the email covered
>>> multiple issues, we did not really dig deep into each of them. One of the
>>> suggestions was to agree on a code review response turnaround time (SLO
>>> ). Here is the
>>> direct quote:
>>>
>>> It would be great if we could agree on a response-time SLA for Beam code
>>> reviews. The response might be “I am unable to do the review until next
>>> week,” however even that is better than getting no response.
>>>
>>>
>>> All the comments on the original thread supported having an agreed upon
>>> SLO. Therefore, I would like to discuss possible response-time SLO and
>>> finalize it within this thread. For the purpose of this discussion, let's
>>> put aside related topics such as the need of tooling support like PR
>>> dashboard or reviewer availability for future discussions.
>>>
>>> *My proposals*
>>>
>>> *Proposal 1*
>>> I propose having a *Default* review response time as *3 business days*.
>>> This aligns with the frequency we consider most developers are checking the
>>> dev list. My reasoning is, if one is checking the dev list, they could also
>>> check their PR review queue.
>>>
>>> *Proposal 2*
>>> I propose having an *Opt-in* review response time as *24 hours*.
>>> Contributors are happy when reviewers respond swiftly to their PRs.
>>> Specially, when we are making multiple small changes to Beam, waiting for
>>> even a few days is frustrating. I understand that not all the reviewers can
>>> review PRs daily. However, if some of us can incorporate half an hour of
>>> beam review to our schedule, it could improve contributors' experience
>>> drastically. Therefore, I suggest us having opt-in response time of 24
>>> hours. We can discuss how we can communicate this SLO to contributors and
>>> reviewers in a separate thread.
>>>
>>> Please vote on these 2 proposals and propose any other solutions using
>>> within this template:
>>>
>>> Template:
>>> Proposal 1: <+-1> 
>>> Proposal 2: <+-1> 
>>> Additional Comments: 
>>>
>>> Example answer:
>>> Proposal 1: +1 Great idea
>>> Proposal 2: +1
>>> Additional Comments: I have this idea foobar 
>>>
>>> Thank you,
>>> Huygaa
>>>
>>>

Re: [VOTE] Code Review Response-time SLO

[Scott Wegner]

Proposal 1: +0
Proposal 2: +1

Additional Comments:
I like the idea of providing a clear expectation to contributors on how
promptly they can expect code review feedback. From your proposal, my main
feedback would to prefer a single goal instead of two. Having a single goal
seems simpler when choosing a code reviewer ("PersonA might have more
context on my change, but PersonB will give quicker feedback.."). Multiple
tiers of commitment also feels like it implies level of commitment to the
community ("PersonB must be more committed to Beam than PersonA because
they agreed to faster code reviews").

There is a balance to strike: we should provide prompt feedback and clear
expectations for code contributors, but we shouldn't set unreasonable
targets for code reviewers. I'd be interested in hearing from others
whether 24-hours seems unreasonable.

I think that we should also make it clear with any policy that opting-in is
optional and not required to be part of the Beam community. There are many
ways to contribute to Beam, and opting-in to reviewing code contributions
is just one of them.

On Mon, Jun 4, 2018 at 3:20 PM Huygaa Batsaikhan  wrote:

> Proposal 1: +1
> Proposal 2: +1
> Additional Comments: This is an example vote
>
> On Mon, Jun 4, 2018 at 3:15 PM Huygaa Batsaikhan 
> wrote:
>
>> A few months ago, Reuven sent out an email
>> 
>> about improvements to Beam's code review process. Because the email covered
>> multiple issues, we did not really dig deep into each of them. One of the
>> suggestions was to agree on a code review response turnaround time (SLO
>> ). Here is the
>> direct quote:
>>
>> It would be great if we could agree on a response-time SLA for Beam code
>> reviews. The response might be “I am unable to do the review until next
>> week,” however even that is better than getting no response.
>>
>>
>> All the comments on the original thread supported having an agreed upon
>> SLO. Therefore, I would like to discuss possible response-time SLO and
>> finalize it within this thread. For the purpose of this discussion, let's
>> put aside related topics such as the need of tooling support like PR
>> dashboard or reviewer availability for future discussions.
>>
>> *My proposals*
>>
>> *Proposal 1*
>> I propose having a *Default* review response time as *3 business days*.
>> This aligns with the frequency we consider most developers are checking the
>> dev list. My reasoning is, if one is checking the dev list, they could also
>> check their PR review queue.
>>
>> *Proposal 2*
>> I propose having an *Opt-in* review response time as *24 hours*.
>> Contributors are happy when reviewers respond swiftly to their PRs.
>> Specially, when we are making multiple small changes to Beam, waiting for
>> even a few days is frustrating. I understand that not all the reviewers can
>> review PRs daily. However, if some of us can incorporate half an hour of
>> beam review to our schedule, it could improve contributors' experience
>> drastically. Therefore, I suggest us having opt-in response time of 24
>> hours. We can discuss how we can communicate this SLO to contributors and
>> reviewers in a separate thread.
>>
>> Please vote on these 2 proposals and propose any other solutions using
>> within this template:
>>
>> Template:
>> Proposal 1: <+-1> 
>> Proposal 2: <+-1> 
>> Additional Comments: 
>>
>> Example answer:
>> Proposal 1: +1 Great idea
>> Proposal 2: +1
>> Additional Comments: I have this idea foobar 
>>
>> Thank you,
>> Huygaa
>>
>>

Re: [VOTE] Code Review Response-time SLO

[Huygaa Batsaikhan]

Proposal 1: +1
Proposal 2: +1
Additional Comments: This is an example vote

On Mon, Jun 4, 2018 at 3:15 PM Huygaa Batsaikhan  wrote:

> A few months ago, Reuven sent out an email
> 
> about improvements to Beam's code review process. Because the email covered
> multiple issues, we did not really dig deep into each of them. One of the
> suggestions was to agree on a code review response turnaround time (SLO
> ). Here is the
> direct quote:
>
> It would be great if we could agree on a response-time SLA for Beam code
> reviews. The response might be “I am unable to do the review until next
> week,” however even that is better than getting no response.
>
>
> All the comments on the original thread supported having an agreed upon
> SLO. Therefore, I would like to discuss possible response-time SLO and
> finalize it within this thread. For the purpose of this discussion, let's
> put aside related topics such as the need of tooling support like PR
> dashboard or reviewer availability for future discussions.
>
> *My proposals*
>
> *Proposal 1*
> I propose having a *Default* review response time as *3 business days*.
> This aligns with the frequency we consider most developers are checking the
> dev list. My reasoning is, if one is checking the dev list, they could also
> check their PR review queue.
>
> *Proposal 2*
> I propose having an *Opt-in* review response time as *24 hours*.
> Contributors are happy when reviewers respond swiftly to their PRs.
> Specially, when we are making multiple small changes to Beam, waiting for
> even a few days is frustrating. I understand that not all the reviewers can
> review PRs daily. However, if some of us can incorporate half an hour of
> beam review to our schedule, it could improve contributors' experience
> drastically. Therefore, I suggest us having opt-in response time of 24
> hours. We can discuss how we can communicate this SLO to contributors and
> reviewers in a separate thread.
>
> Please vote on these 2 proposals and propose any other solutions using
> within this template:
>
> Template:
> Proposal 1: <+-1> 
> Proposal 2: <+-1> 
> Additional Comments: 
>
> Example answer:
> Proposal 1: +1 Great idea
> Proposal 2: +1
> Additional Comments: I have this idea foobar 
>
> Thank you,
> Huygaa
>
>

[VOTE] Code Review Response-time SLO

[Huygaa Batsaikhan]

A few months ago, Reuven sent out an email

about improvements to Beam's code review process. Because the email covered
multiple issues, we did not really dig deep into each of them. One of the
suggestions was to agree on a code review response turnaround time (SLO
). Here is the
direct quote:

It would be great if we could agree on a response-time SLA for Beam code
reviews. The response might be “I am unable to do the review until next
week,” however even that is better than getting no response.


All the comments on the original thread supported having an agreed upon
SLO. Therefore, I would like to discuss possible response-time SLO and
finalize it within this thread. For the purpose of this discussion, let's
put aside related topics such as the need of tooling support like PR
dashboard or reviewer availability for future discussions.

*My proposals*

*Proposal 1*
I propose having a *Default* review response time as *3 business days*.
This aligns with the frequency we consider most developers are checking the
dev list. My reasoning is, if one is checking the dev list, they could also
check their PR review queue.

*Proposal 2*
I propose having an *Opt-in* review response time as *24 hours*.
Contributors are happy when reviewers respond swiftly to their PRs.
Specially, when we are making multiple small changes to Beam, waiting for
even a few days is frustrating. I understand that not all the reviewers can
review PRs daily. However, if some of us can incorporate half an hour of
beam review to our schedule, it could improve contributors' experience
drastically. Therefore, I suggest us having opt-in response time of 24
hours. We can discuss how we can communicate this SLO to contributors and
reviewers in a separate thread.

Please vote on these 2 proposals and propose any other solutions using
within this template:

Template:
Proposal 1: <+-1> 
Proposal 2: <+-1> 
Additional Comments: 

Example answer:
Proposal 1: +1 Great idea
Proposal 2: +1
Additional Comments: I have this idea foobar 

Thank you,
Huygaa