Re: [Xen-devel] [PATCH v2 4/6] Add Code Review Guide

[Lars Kurth]

> On 9 Dec 2019, at 11:02, Lars Kurth  wrote:
> 
> 
> 
> On 06/12/2019, 09:51, "Jan Beulich"  > wrote:
> 
>On 06.12.2019 00:41, Lars Kurth wrote:
>> I propose to add the following section to code-review-guide.md
>> 
>> 
>> ## Problematic Patch Reviews
>> 
>> A typical waterfall software development process is sequential with the 
>> following 
>> steps: define requirements, analyse, design, code, test and deploy. Problems 
>> uncovered by code review or testing at such a late stage can cause costly 
>> redesign 
>> and delays. The principle of **[Shift 
>> Left](https://devopedia.org/shift-left)** is to take a 
>> task that is traditionally performed at a late stage in the process and 
>> perform that task 
>> at earlier stages. The goal is to save time by avoiding refactoring.
>> 
>> Typically, problematic patch reviews uncover issues such as wrong or missed 
>> assumptions, a problematic architecture or design, or other bugs that 
>> require 
>> significant re-implementation of a patch series to fix the issue.
>> 
>> The principle of **Shift Left** also applies in code reviews. Let's assume a 
>> series has
>> a major flaw: ideally, this flaw would be picked up in the **first or second 
>> iteration** of 
>> the code review. As significant parts of the code may have to be re-written, 
>> it does not 
>> make sense for reviewers to highlight minor issues (such as style issues) 
>> until major 
>> flaws have been addressed. By providing feedback on minor issues reviewers 
>> cause 
>> the code author and themselves extra work by asking for changes to code, 
>> which 
>> ultimately may be changed later.
>> 
>> The question then becomes, how do code reviewers identify major issues 
>> early? 
>> 
>> This is where I really need help. Are there any tips and recommendations 
>> that we could give?
>> I can clearly highlight that we have RFC series, but in practice that does 
>> not solve the problem as RFCs don’t get prioritized
>> How do reviewers normally approach a series: do you a) take a big picture 
>> view first, or b) do most of you work through a series sequentially
> 
>Afaic - depends heavily on the patch / series. I wouldn't typically
>peek ahead in a series, but it has happened. But as you say
>(elsewhere) the cover letter should put in place the "big picture".
>A series should generally be reviewable going from patch to patch,
>having the cover letter in mind.
> 
> I am wondering what others do. 
> 
> I think explaining the basic work-flow from the viewpoint of a reviewer and 
> code author maybe in a separate section, which is not tied to the problem 
> case would make sense. More input from other maintainers would be valuable. 
> My gut-feel is that most reviewers "read and review" series sequentially, 
> which has implications for the author. E.g.
> - docs/design docs should be at the beginning of a series
> - key header files or changes to them should be at the beginning of a series
> - Etc
> 
>> I then propose to change the following section in communication-practice.md
>> 
>> ### Prioritize significant flaws
>> If a patch or patch series has significant flaws, such as
>> * It is built on wrong assumptions
>> * There are issues with the architecture or the design
> 
>In such a case a full review of course doesn't make much sense. But
>this is far from the typical situation. Way more often you have some
>_part_ of a patch or series which has a bigger issue, but other
>parts are in need of no or just minor changes.
> 
> I know that this is an unusual situation. But it has happened in clusters 
> frequently in the past.
> 
> I am wondering whether we should introduce some informal convention to mark 
> _part_ of a series as problematic. A simple example of how to do this in the 
> cover letter would do
> 
>> it does not make sense to do a detailed code review. In such cases, it is 
>> best to
>> focus on the major issues first and deal with style and minor issues in a 
>> subsequent
>> review. Not all series have significant flaws, but most series have 
>> different classes of 
>> changes that are required for acceptance: covering a range of major code 
>> modifications to minor code style fixes. To avoid misunderstandings between 
>> reviewers and contributors, it is important to establish and agree whether a 
>> series or 
>> part of a series has a significant flaw and agree a course of action. 
>> 
>> A pragmatic approach would be to
>> * Highlight problematic portions of a series in the cover letter 
>> * For the patch author and reviewer(s) to agree that for problematic to omit 
>> style and
>> minor issues in the review, until the significant flaw is addressed
>> 
>> This saves both the patch author and reviewer(s) time. Note that some 
>> background
>> is covered in detail in [Problematic Patch 
>> Reviews](resolving-disagreement.md#problems).
> 
>I have no issues with the suggested text in general, but I 

Re: [Xen-devel] [PATCH v2 4/6] Add Code Review Guide

[Lars Kurth]

On 06/12/2019, 09:51, "Jan Beulich"  wrote:

On 06.12.2019 00:41, Lars Kurth wrote:
> I propose to add the following section to code-review-guide.md
> 
> 
> ## Problematic Patch Reviews
> 
> A typical waterfall software development process is sequential with the 
following 
> steps: define requirements, analyse, design, code, test and deploy. 
Problems 
> uncovered by code review or testing at such a late stage can cause costly 
redesign 
> and delays. The principle of **[Shift 
Left](https://devopedia.org/shift-left)** is to take a 
> task that is traditionally performed at a late stage in the process and 
perform that task 
> at earlier stages. The goal is to save time by avoiding refactoring.
> 
> Typically, problematic patch reviews uncover issues such as wrong or 
missed 
> assumptions, a problematic architecture or design, or other bugs that 
require 
> significant re-implementation of a patch series to fix the issue.
> 
> The principle of **Shift Left** also applies in code reviews. Let's 
assume a series has
> a major flaw: ideally, this flaw would be picked up in the **first or 
second iteration** of 
> the code review. As significant parts of the code may have to be 
re-written, it does not 
> make sense for reviewers to highlight minor issues (such as style issues) 
until major 
> flaws have been addressed. By providing feedback on minor issues 
reviewers cause 
> the code author and themselves extra work by asking for changes to code, 
which 
> ultimately may be changed later.
> 
> The question then becomes, how do code reviewers identify major issues 
early? 
> 
> This is where I really need help. Are there any tips and recommendations 
that we could give?
> I can clearly highlight that we have RFC series, but in practice that 
does not solve the problem as RFCs don’t get prioritized
> How do reviewers normally approach a series: do you a) take a big picture 
view first, or b) do most of you work through a series sequentially

Afaic - depends heavily on the patch / series. I wouldn't typically
peek ahead in a series, but it has happened. But as you say
(elsewhere) the cover letter should put in place the "big picture".
A series should generally be reviewable going from patch to patch,
having the cover letter in mind.

I am wondering what others do. 

I think explaining the basic work-flow from the viewpoint of a reviewer and 
code author maybe in a separate section, which is not tied to the problem case 
would make sense. More input from other maintainers would be valuable. My 
gut-feel is that most reviewers "read and review" series sequentially, which 
has implications for the author. E.g.
- docs/design docs should be at the beginning of a series
- key header files or changes to them should be at the beginning of a series
- Etc
   
> I then propose to change the following section in 
communication-practice.md
> 
> ### Prioritize significant flaws
> If a patch or patch series has significant flaws, such as
> * It is built on wrong assumptions
> * There are issues with the architecture or the design

In such a case a full review of course doesn't make much sense. But
this is far from the typical situation. Way more often you have some
_part_ of a patch or series which has a bigger issue, but other
parts are in need of no or just minor changes.

I know that this is an unusual situation. But it has happened in clusters 
frequently in the past.

I am wondering whether we should introduce some informal convention to mark 
_part_ of a series as problematic. A simple example of how to do this in the 
cover letter would do

> it does not make sense to do a detailed code review. In such cases, it is 
best to
> focus on the major issues first and deal with style and minor issues in a 
subsequent
> review. Not all series have significant flaws, but most series have 
different classes of 
> changes that are required for acceptance: covering a range of major code 
> modifications to minor code style fixes. To avoid misunderstandings 
between 
> reviewers and contributors, it is important to establish and agree 
whether a series or 
> part of a series has a significant flaw and agree a course of action. 
> 
> A pragmatic approach would be to
> * Highlight problematic portions of a series in the cover letter 
> * For the patch author and reviewer(s) to agree that for problematic to 
omit style and
> minor issues in the review, until the significant flaw is addressed
> 
> This saves both the patch author and reviewer(s) time. Note that some 
background
> is covered in detail in [Problematic Patch 
Reviews](resolving-disagreement.md#problems).

I have no issues with the suggested text in general, but I also don't
think it makes much 

Re: [Xen-devel] [PATCH v2 4/6] Add Code Review Guide

[Jan Beulich]

On 06.12.2019 00:41, Lars Kurth wrote:
> I propose to add the following section to code-review-guide.md
> 
> 
> ## Problematic Patch Reviews
> 
> A typical waterfall software development process is sequential with the 
> following 
> steps: define requirements, analyse, design, code, test and deploy. Problems 
> uncovered by code review or testing at such a late stage can cause costly 
> redesign 
> and delays. The principle of **[Shift 
> Left](https://devopedia.org/shift-left)** is to take a 
> task that is traditionally performed at a late stage in the process and 
> perform that task 
> at earlier stages. The goal is to save time by avoiding refactoring.
> 
> Typically, problematic patch reviews uncover issues such as wrong or missed 
> assumptions, a problematic architecture or design, or other bugs that require 
> significant re-implementation of a patch series to fix the issue.
> 
> The principle of **Shift Left** also applies in code reviews. Let's assume a 
> series has
> a major flaw: ideally, this flaw would be picked up in the **first or second 
> iteration** of 
> the code review. As significant parts of the code may have to be re-written, 
> it does not 
> make sense for reviewers to highlight minor issues (such as style issues) 
> until major 
> flaws have been addressed. By providing feedback on minor issues reviewers 
> cause 
> the code author and themselves extra work by asking for changes to code, 
> which 
> ultimately may be changed later.
> 
> The question then becomes, how do code reviewers identify major issues early? 
> 
> This is where I really need help. Are there any tips and recommendations that 
> we could give?
> I can clearly highlight that we have RFC series, but in practice that does 
> not solve the problem as RFCs don’t get prioritized
> How do reviewers normally approach a series: do you a) take a big picture 
> view first, or b) do most of you work through a series sequentially

Afaic - depends heavily on the patch / series. I wouldn't typically
peek ahead in a series, but it has happened. But as you say
(elsewhere) the cover letter should put in place the "big picture".
A series should generally be reviewable going from patch to patch,
having the cover letter in mind.

> I then propose to change the following section in communication-practice.md
> 
> ### Prioritize significant flaws
> If a patch or patch series has significant flaws, such as
> * It is built on wrong assumptions
> * There are issues with the architecture or the design

In such a case a full review of course doesn't make much sense. But
this is far from the typical situation. Way more often you have some
_part_ of a patch or series which has a bigger issue, but other
parts are in need of no or just minor changes.

> it does not make sense to do a detailed code review. In such cases, it is 
> best to
> focus on the major issues first and deal with style and minor issues in a 
> subsequent
> review. Not all series have significant flaws, but most series have different 
> classes of 
> changes that are required for acceptance: covering a range of major code 
> modifications to minor code style fixes. To avoid misunderstandings between 
> reviewers and contributors, it is important to establish and agree whether a 
> series or 
> part of a series has a significant flaw and agree a course of action. 
> 
> A pragmatic approach would be to
> * Highlight problematic portions of a series in the cover letter 
> * For the patch author and reviewer(s) to agree that for problematic to omit 
> style and
> minor issues in the review, until the significant flaw is addressed
> 
> This saves both the patch author and reviewer(s) time. Note that some 
> background
> is covered in detail in [Problematic Patch 
> Reviews](resolving-disagreement.md#problems).

I have no issues with the suggested text in general, but I also don't
think it makes much of a difference wrt what I had mentioned before.
I guess part of the problem here is that there are things which imo
you can't really give recipes for how to approach, if the expectation
is that it would fit at least the vast majority of cases. For code
reviews this means that I don't think there should be any wording
suggesting they should be done in a certain form; there may be wording
suggesting they _could_ be done in a certain form (e.g. to help
people not knowing at all how to get started).

Jan

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2 4/6] Add Code Review Guide

[Lars Kurth]

On 28/11/2019, 12:12, "Rich Persaud"  wrote:

On Nov 28, 2019, at 05:12, Jan Beulich  wrote:
> 
> On 28.11.2019 01:54, Stefano Stabellini wrote:
>>> On Thu, 26 Sep 2019, Lars Kurth wrote:
>>> From: Lars Kurth 
>>> 
>>> This document highlights what reviewers such as maintainers and 
committers look
>>> for when reviewing code. It sets expectations for code authors and 
provides
>>> a framework for code reviewers.
>> 
>> I think the document is missing a couple of things:
>> 
>> - a simple one line statement that possibly the most important thing in
>>  a code review is to indentify any bugs in the code
>> 
>> - an explanation that requests for major changes to the series should be
>>  made early on (i.e. let's not change the architecture of a feature at
>>  v9 if possible) I also made this comment in reply to patch #5. I'll
>>  let you decide where is the best place for it.
> 
> This needs balancing. People crucial to the evaluation of a new
> feature and its implementation simply may not have the time to
> reply prior to v9. We've had situations where people posted new
> revisions every other day, sometimes even more than one per day.
> 
> As indicated in several other contexts before - imo people not
> helping to shoulder the review load should also not have the
> expectation that their (large) contributions will be looked at
> in due course. 

To make this actionable, we could have:

- reviewer demand index:  automated index of open patches still in need of 
review, sorted by decreasing age

- review flow control:  each new patch submission cites one recent review 
by the patch submitter, for a patch of comparable size

- reviewer supply growth:  a bootstrapping guide for new reviewers and 
submitters, with patterns, anti-patterns, and examples to be emulated

That is a great idea. However, I would not want to hold up the publication of 
these documents on these suggestions. Some of them would require implementing 
tools. I was hoping there would be more progress on lore and others 
tooling/workflow related stuff by now. So I think for now, I think it is 
sufficient to set expectations better.

Regards
Lars

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2 4/6] Add Code Review Guide

[Stefano Stabellini]

On Thu, 28 Nov 2019, Jan Beulich wrote:
> On 28.11.2019 01:54, Stefano Stabellini wrote:
> > On Thu, 26 Sep 2019, Lars Kurth wrote:
> >> From: Lars Kurth 
> >>
> >> This document highlights what reviewers such as maintainers and committers 
> >> look
> >> for when reviewing code. It sets expectations for code authors and provides
> >> a framework for code reviewers.
> > 
> > I think the document is missing a couple of things:
> > 
> > - a simple one line statement that possibly the most important thing in
> >   a code review is to indentify any bugs in the code
> > 
> > - an explanation that requests for major changes to the series should be
> >   made early on (i.e. let's not change the architecture of a feature at
> >   v9 if possible) I also made this comment in reply to patch #5. I'll
> >   let you decide where is the best place for it.
> 
> This needs balancing. People crucial to the evaluation of a new
> feature and its implementation simply may not have the time to
> reply prior to v9. We've had situations where people posted new
> revisions every other day, sometimes even more than one per day.

Yes, you are right, it needs balancing. This is not meant to encourage
contributors to send 9 versions of a series within a week or two :-)

We could say that "contributors should make sure to give enough time to
all the key stakeholders to review the series".



> As indicated in several other contexts before - imo people not
> helping to shoulder the review load should also not have the
> expectation that their (large) contributions will be looked at
> in due course. 

I think you are right on this point, and maybe we could add something to
that effect

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2 4/6] Add Code Review Guide

[Rich Persaud]

On Nov 28, 2019, at 05:12, Jan Beulich  wrote:
> 
> On 28.11.2019 01:54, Stefano Stabellini wrote:
>>> On Thu, 26 Sep 2019, Lars Kurth wrote:
>>> From: Lars Kurth 
>>> 
>>> This document highlights what reviewers such as maintainers and committers 
>>> look
>>> for when reviewing code. It sets expectations for code authors and provides
>>> a framework for code reviewers.
>> 
>> I think the document is missing a couple of things:
>> 
>> - a simple one line statement that possibly the most important thing in
>>  a code review is to indentify any bugs in the code
>> 
>> - an explanation that requests for major changes to the series should be
>>  made early on (i.e. let's not change the architecture of a feature at
>>  v9 if possible) I also made this comment in reply to patch #5. I'll
>>  let you decide where is the best place for it.
> 
> This needs balancing. People crucial to the evaluation of a new
> feature and its implementation simply may not have the time to
> reply prior to v9. We've had situations where people posted new
> revisions every other day, sometimes even more than one per day.
> 
> As indicated in several other contexts before - imo people not
> helping to shoulder the review load should also not have the
> expectation that their (large) contributions will be looked at
> in due course. 

To make this actionable, we could have:

- reviewer demand index:  automated index of open patches still in need of 
review, sorted by decreasing age

- review flow control:  each new patch submission cites one recent review by 
the patch submitter, for a patch of comparable size

- reviewer supply growth:  a bootstrapping guide for new reviewers and 
submitters, with patterns, anti-patterns, and examples to be emulated

Rich
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2 4/6] Add Code Review Guide

[Lars Kurth]

On 28/11/2019, 07:37, "Jan Beulich"  wrote:

On 28.11.2019 14:06, Lars Kurth wrote:
> I can certainly add something on the timing , along the lines of
> * For complex series, consider the time it takes to do reviews (maybe 
with a guide of LOC per hour) and give reviewers enough time to
> * For series with design issues or large questions, try and highlight the 
key open issues in cover letters clearly and solicit feedback from key 
maintainers who can comment on the open issue. The idea is to save both the 
contributor and the reviewers time by focussing on what needs to be resolved 
> * Don’t repost a series, unless all review comments are addressed
> or the reviewers asked you to do so. The problem with this is that
> this is somewhat in conflict with the "let's focus on the core
> issues and not get distracted by details early on in a review cycle".
> In other words, this can only work, if reviewers focus on major
> issues in early reviews only and do not focus on style, coding
> standards, etc.

But this doesn't make much sense either, because then full re-reviews
need to happen anyway on later versions, to also deal with the minor
issues. For RFC kind of series omitting style and alike feedback
certainly makes sense, but as soon as a patch is non-RFC, it should
be considered good to go in by the submitter.

OK, I think we have a disconnect between ideal and reality. 

I see two issues today
* Key maintainers don't always review RFC series [they end up at the bottom of 
the priority list, even though spending time on RFCs will save time elsewhere 
later]. So the effect is that then the contributor assumes there are no major 
issues and ends it as a proper series
* In practice what has happened often in the past is that design, architecture, 
assumption flaws are found in early versions of a series.
   - This usually happens because of an oversight or because there was no 
design discussion prior to the series being posted and agreed
   - Common sense would dictate that the biggest benefit for both the reviewer, 
the contributor and the community as a whole would be to try and focus on such 
flaws and leave everything aside
   - Of course there may be value in doing a detailed reviews of such a series 
as there may be bits that are unaffected by such a flaw
   - But there will likely be parts which are not: doing a detailed review of 
such portions wastes everyone's time

So coming back to your point. Ideally, it would be nice if we had the 
capability to call out parts of a series as "problematic" and treating such 
parts differently

Lars

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2 4/6] Add Code Review Guide

[Jan Beulich]

On 28.11.2019 14:06, Lars Kurth wrote:
> I can certainly add something on the timing , along the lines of
> * For complex series, consider the time it takes to do reviews (maybe with a 
> guide of LOC per hour) and give reviewers enough time to
> * For series with design issues or large questions, try and highlight the key 
> open issues in cover letters clearly and solicit feedback from key 
> maintainers who can comment on the open issue. The idea is to save both the 
> contributor and the reviewers time by focussing on what needs to be resolved 
> * Don’t repost a series, unless all review comments are addressed
> or the reviewers asked you to do so. The problem with this is that
> this is somewhat in conflict with the "let's focus on the core
> issues and not get distracted by details early on in a review cycle".
> In other words, this can only work, if reviewers focus on major
> issues in early reviews only and do not focus on style, coding
> standards, etc.

But this doesn't make much sense either, because then full re-reviews
need to happen anyway on later versions, to also deal with the minor
issues. For RFC kind of series omitting style and alike feedback
certainly makes sense, but as soon as a patch is non-RFC, it should
be considered good to go in by the submitter.

Jan

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2 4/6] Add Code Review Guide

[Lars Kurth]

On 28/11/2019, 04:09, "Jan Beulich"  wrote:

On 28.11.2019 01:54, Stefano Stabellini wrote:
> On Thu, 26 Sep 2019, Lars Kurth wrote:
>> From: Lars Kurth 
>>
>> This document highlights what reviewers such as maintainers and 
committers look
>> for when reviewing code. It sets expectations for code authors and 
provides
>> a framework for code reviewers.
> 
> I think the document is missing a couple of things:
> 
> - a simple one line statement that possibly the most important thing in
>   a code review is to indentify any bugs in the code
> 
> - an explanation that requests for major changes to the series should be
>   made early on (i.e. let's not change the architecture of a feature at
>   v9 if possible) I also made this comment in reply to patch #5. I'll
>   let you decide where is the best place for it.

This needs balancing. People crucial to the evaluation of a new
feature and its implementation simply may not have the time to
reply prior to v9. We've had situations where people posted new
revisions every other day, sometimes even more than one per day.

I can certainly add something on the timing , along the lines of
* For complex series, consider the time it takes to do reviews (maybe with a 
guide of LOC per hour) and give reviewers enough time to
* For series with design issues or large questions, try and highlight the key 
open issues in cover letters clearly and solicit feedback from key maintainers 
who can comment on the open issue. The idea is to save both the contributor and 
the reviewers time by focussing on what needs to be resolved 
* Don’t repost a series, unless all review comments are addressed or the 
reviewers asked you to do so. The problem with this is that this is somewhat in 
conflict with the "let's focus on the core issues and not get distracted by 
details early on in a review cycle". In other words, this can only work, if 
reviewers focus on major issues in early reviews only and do not focus on 
style, coding standards, etc. As soon as a reviewer comes back with detailed 
feedback, the contributor will feel obliged to fix these. This creates a 
motivation to want to please the reviewer send out new versions of series 
fixing cosmetic issues without addressing the substantial issues, leading to 
what Jan describes. I am looking for opinions here.  

As indicated in several other contexts before - imo people not
helping to shoulder the review load should also not have the
expectation that their (large) contributions will be looked at
in due course. 

I can add something to this effect.  

Lars


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2 4/6] Add Code Review Guide

[Jan Beulich]

On 28.11.2019 01:54, Stefano Stabellini wrote:
> On Thu, 26 Sep 2019, Lars Kurth wrote:
>> From: Lars Kurth 
>>
>> This document highlights what reviewers such as maintainers and committers 
>> look
>> for when reviewing code. It sets expectations for code authors and provides
>> a framework for code reviewers.
> 
> I think the document is missing a couple of things:
> 
> - a simple one line statement that possibly the most important thing in
>   a code review is to indentify any bugs in the code
> 
> - an explanation that requests for major changes to the series should be
>   made early on (i.e. let's not change the architecture of a feature at
>   v9 if possible) I also made this comment in reply to patch #5. I'll
>   let you decide where is the best place for it.

This needs balancing. People crucial to the evaluation of a new
feature and its implementation simply may not have the time to
reply prior to v9. We've had situations where people posted new
revisions every other day, sometimes even more than one per day.

As indicated in several other contexts before - imo people not
helping to shoulder the review load should also not have the
expectation that their (large) contributions will be looked at
in due course. 

Jan

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2 4/6] Add Code Review Guide

[Stefano Stabellini]

On Thu, 26 Sep 2019, Lars Kurth wrote:
> From: Lars Kurth 
> 
> This document highlights what reviewers such as maintainers and committers 
> look
> for when reviewing code. It sets expectations for code authors and provides
> a framework for code reviewers.

I think the document is missing a couple of things:

- a simple one line statement that possibly the most important thing in
  a code review is to indentify any bugs in the code

- an explanation that requests for major changes to the series should be
  made early on (i.e. let's not change the architecture of a feature at
  v9 if possible) I also made this comment in reply to patch #5. I'll
  let you decide where is the best place for it.


> Signed-off-by: Lars Kurth 
> ---
> Cc: minios-de...@lists.xenproject.org
> Cc: xen-...@lists.xenproject.org
> Cc: win-pv-de...@lists.xenproject.org
> Cc: mirageos-de...@lists.xenproject.org
> Cc: committ...@xenproject.org
> ---
>  code-review-guide.md | 125 
> +++
>  1 file changed, 125 insertions(+)
>  create mode 100644 code-review-guide.md
> 
> diff --git a/code-review-guide.md b/code-review-guide.md
> new file mode 100644
> index 000..8639431
> --- /dev/null
> +++ b/code-review-guide.md
> @@ -0,0 +1,125 @@
> +# Code Review Guide
> +
> +This document highlights what reviewers such as maintainers and committers 
> look
> +for when reviewing your code. It sets expectations for code authors and 
> provides
> +a framework for code reviewers.
> +
> +This document does **not cover** the following topics:
> +* [Communication Best Practice](communication-practice.md)
> +* [Resolving Disagreement](resolving-disagreement.md)
> +* [Patch Submission 
> Workflow](https://wiki.xenproject.org/wiki/Submitting_Xen_Project_Patches)
> +* [Managing Patch Submission with 
> Git](https://wiki.xenproject.org/wiki/Managing_Xen_Patches_with_Git)
> +
> +## What we look for in Code Reviews
> +When performing a code review, reviewers typically look for the following 
> things
> +
> +### Is the change necessary to accomplish the goals?
> +* Is it clear what the goals are?
> +* Do we need to make a change, or can the goals be met with existing
> +  functionality?
> +
> +### Architecture / Interface
> +* Is this the best way to solve the problem?
> +* Is this the right part of the code to modify?
> +* Is this the right level of abstraction?
> +* Is the interface general enough? Too general? Forward compatible?
> +
> +### Functionality
> +* Does it do what it’s trying to do?
> +* Is it doing it in the most efficient way?
> +* Does it handle all the corner / error cases correctly?
> +
> +### Maintainability / Robustness
> +* Is the code clear? Appropriately commented?
> +* Does it duplicate another piece of code?
> +* Does the code make hidden assumptions?
> +* Does it introduce sections which need to be kept **in sync** with other 
> sections?
> +* Are there other **traps** someone modifying this code might fall into?
> +
> +**Note:** Sometimes you will work in areas which have identified 
> maintainability
> +and/or robustness issues. In such cases, maintainers may ask you to make 
> additional
> +changes, such that your submitted code does not make things worse or point 
> you
> +to other patches are already being worked on.
> +
> +### System properties
> +In some areas of the code, system properties such as
> +* Code size
> +* Performance
> +* Scalability
> +* Latency
> +* Complexity
> +* 
> +are also important during code reviews.
> +
> +### Style
> +* Comments, carriage returns, **snuggly braces**, 
> +* See 
> [CODING_STYLE](https://xenbits.xenproject.org/gitweb/?p=xen.git;a=blob;f=CODING_STYLE)
> +  and 
> [tools/libxl/CODING_STYLE](https://xenbits.xenproject.org/gitweb/?p=xen.git;a=blob;f=tools/libxl/CODING_STYLE)
> +* No extraneous whitespace changes
> +
> +### Documentation and testing
> +* If there is pre-existing documentation in the tree, such as man pages, 
> design
> +  documents, etc. a contributor may be asked to update the documentation 
> alongside
> +  the change. Documentation is typically present in the
> +  [docs](https://xenbits.xen.org/gitweb/?p=xen.git;a=tree;f=docs) folder.
> +* When adding new features that have an impact on the end-user,
> +  a contributor should include an update to the
> +  [SUPPORT.md](https://xenbits.xen.org/gitweb/?p=xen.git;a=tree;f=docs) file.
> +  Typically, more complex features require several patch series before it is 
> ready to be
> +  advertised in SUPPORT.md
> +* When adding new features, a contributor may be asked to provide tests or
> +  ensure that existing tests pass
> +
> + Testing for the Xen Project Hypervisor
> +Tests are typically located in one of the following directories
> +* **Unit tests**: 
> [tools/tests](https://xenbits.xenproject.org/gitweb/?p=xen.git;a=tree;f=tools/tests)
> +or 
> [xen/test](https://xenbits.xenproject.org/gitweb/?p=xen.git;a=tree;f=xen/test)
> +  Unit testing is hard for a system like Xen and 

[Xen-devel] [PATCH v2 4/6] Add Code Review Guide

[Lars Kurth]

From: Lars Kurth 

This document highlights what reviewers such as maintainers and committers look
for when reviewing code. It sets expectations for code authors and provides
a framework for code reviewers.

Signed-off-by: Lars Kurth 
---
Cc: minios-de...@lists.xenproject.org
Cc: xen-...@lists.xenproject.org
Cc: win-pv-de...@lists.xenproject.org
Cc: mirageos-de...@lists.xenproject.org
Cc: committ...@xenproject.org
---
 code-review-guide.md | 125 +++
 1 file changed, 125 insertions(+)
 create mode 100644 code-review-guide.md

diff --git a/code-review-guide.md b/code-review-guide.md
new file mode 100644
index 000..8639431
--- /dev/null
+++ b/code-review-guide.md
@@ -0,0 +1,125 @@
+# Code Review Guide
+
+This document highlights what reviewers such as maintainers and committers look
+for when reviewing your code. It sets expectations for code authors and 
provides
+a framework for code reviewers.
+
+This document does **not cover** the following topics:
+* [Communication Best Practice](communication-practice.md)
+* [Resolving Disagreement](resolving-disagreement.md)
+* [Patch Submission 
Workflow](https://wiki.xenproject.org/wiki/Submitting_Xen_Project_Patches)
+* [Managing Patch Submission with 
Git](https://wiki.xenproject.org/wiki/Managing_Xen_Patches_with_Git)
+
+## What we look for in Code Reviews
+When performing a code review, reviewers typically look for the following 
things
+
+### Is the change necessary to accomplish the goals?
+* Is it clear what the goals are?
+* Do we need to make a change, or can the goals be met with existing
+  functionality?
+
+### Architecture / Interface
+* Is this the best way to solve the problem?
+* Is this the right part of the code to modify?
+* Is this the right level of abstraction?
+* Is the interface general enough? Too general? Forward compatible?
+
+### Functionality
+* Does it do what it’s trying to do?
+* Is it doing it in the most efficient way?
+* Does it handle all the corner / error cases correctly?
+
+### Maintainability / Robustness
+* Is the code clear? Appropriately commented?
+* Does it duplicate another piece of code?
+* Does the code make hidden assumptions?
+* Does it introduce sections which need to be kept **in sync** with other 
sections?
+* Are there other **traps** someone modifying this code might fall into?
+
+**Note:** Sometimes you will work in areas which have identified 
maintainability
+and/or robustness issues. In such cases, maintainers may ask you to make 
additional
+changes, such that your submitted code does not make things worse or point you
+to other patches are already being worked on.
+
+### System properties
+In some areas of the code, system properties such as
+* Code size
+* Performance
+* Scalability
+* Latency
+* Complexity
+* 
+are also important during code reviews.
+
+### Style
+* Comments, carriage returns, **snuggly braces**, 
+* See 
[CODING_STYLE](https://xenbits.xenproject.org/gitweb/?p=xen.git;a=blob;f=CODING_STYLE)
+  and 
[tools/libxl/CODING_STYLE](https://xenbits.xenproject.org/gitweb/?p=xen.git;a=blob;f=tools/libxl/CODING_STYLE)
+* No extraneous whitespace changes
+
+### Documentation and testing
+* If there is pre-existing documentation in the tree, such as man pages, design
+  documents, etc. a contributor may be asked to update the documentation 
alongside
+  the change. Documentation is typically present in the
+  [docs](https://xenbits.xen.org/gitweb/?p=xen.git;a=tree;f=docs) folder.
+* When adding new features that have an impact on the end-user,
+  a contributor should include an update to the
+  [SUPPORT.md](https://xenbits.xen.org/gitweb/?p=xen.git;a=tree;f=docs) file.
+  Typically, more complex features require several patch series before it is 
ready to be
+  advertised in SUPPORT.md
+* When adding new features, a contributor may be asked to provide tests or
+  ensure that existing tests pass
+
+ Testing for the Xen Project Hypervisor
+Tests are typically located in one of the following directories
+* **Unit tests**: 
[tools/tests](https://xenbits.xenproject.org/gitweb/?p=xen.git;a=tree;f=tools/tests)
+or 
[xen/test](https://xenbits.xenproject.org/gitweb/?p=xen.git;a=tree;f=xen/test)
+  Unit testing is hard for a system like Xen and typically requires building a 
subsystem of
+  your tree. If your change can be easily unit tested, you should consider 
submitting tests
+  with your patch.
+* **Build and smoke test**: see [Xen GitLab 
CI](https://gitlab.com/xen-project/xen/pipelines)
+  Runs build tests for a combination of various distros and compilers against 
changes
+  committed to staging. Developers can join as members and test their 
development
+  branches **before** submitting a patch.
+* **XTF tests** (microkernel-based tests): see 
[XTF](https://xenbits.xenproject.org/docs/xtf/)
+  XTF has been designed to test interactions between your software and 
hardware.
+  It is a very useful tool for testing low level functionality and is