Re: [openstack-dev] [nova] 2 weeks in the bug tracker

2014-09-23 Thread Dan Prince
On Fri, 2014-09-19 at 09:13 -0400, Sean Dague wrote:
 I've spent the better part of the last 2 weeks in the Nova bug tracker
 to try to turn it into something that doesn't cause people to run away
 screaming. I don't remember exactly where we started at open bug count 2
 weeks ago (it was north of 1400, with  200 bugs in new, but it might
 have been north of 1600), but as of this email we're at  1000 open bugs
 (I'm counting Fix Committed as closed, even though LP does not), and ~0
 new bugs (depending on the time of the day).
 
 == Philosophy in Triaging ==
 
 I'm going to lay out the philosophy of triaging I've had, because this
 may also set the tone going forward.
 
 A bug tracker is a tool to help us make a better release. It does not
 exist for it's own good, it exists to help. Which means when evaluating
 what stays in and what leaves we need to evaluate if any particular
 artifact will help us make a better release. But also more importantly
 realize that there is a cost for carrying every artifact in the tracker.
 Resolving duplicates gets non linearly harder as the number of artifacts
 go up. Triaging gets non-linearly hard as the number of artifacts go up.
 
 With this I was being somewhat pragmatic about closing bugs. An old bug
 that is just a stacktrace is typically not useful. An old bug that is a
 vague sentence that we should refactor a particular module (with no
 specifics on the details) is not useful. A bug reported against a very
 old version of OpenStack where the code has changed a lot in the
 relevant area, and there aren't responses from the author, is not
 useful. Not useful bugs just add debt, and we should get rid of them.
 That makes the chance of pulling a random bug off the tracker something
 that you could actually look at fixing, instead of mostly just stalling out.
 
 So I closed a lot of stuff as Invalid / Opinion that fell into those camps.
 
 == Keeping New Bugs at close to 0 ==
 
 After driving the bugs in the New state down to zero last week, I found
 it's actually pretty easy to keep it at 0.
 
 We get 10 - 20 new bugs a day in Nova (during a weekday). Of those ~20%
 aren't actually a bug, and can be closed immediately. ~30% look like a
 bug, but don't have anywhere near enough information in them, and
 flipping them to incomplete with questions quickly means we have a real
 chance of getting the right info. ~10% are fixable in  30 minutes worth
 of work. And the rest are real bugs, that seem to have enough to dive
 into it, and can be triaged into Confirmed, set a priority, and add the
 appropriate tags for the area.
 
 But, more importantly, this means we can filter bug quality on the way
 in. And we can also encourage bug reporters that are giving us good
 stuff, or even easy stuff, as we respond quickly.
 
 Recommendation #1: we adopt a 0 new bugs policy to keep this from
 getting away from us in the future.
 
 == Our worse bug reporters are often core reviewers ==
 
 I'm going to pick on Dan Prince here, mostly because I have a recent
 concrete example, however in triaging the bug queue much of the core
 team is to blame (including myself).
 
 https://bugs.launchpad.net/nova/+bug/1368773 is a terrible bug. Also, it
 was set incomplete and no response. I'm almost 100% sure it's a dupe of
 the multiprocess bug we've been tracking down but it's so terse that you
 can't get to the bottom of it.


This bug was filed as a result of a cryptic (to me at the time) gate
unit test failure that occurred in this review:

https://review.openstack.org/#/c/120099/

I mistakenly grabbed the last timeout error instead of looking at the
original timeout. Within 30 minutes or so of my post Matt Riedemann had
correctly classified it as https://bugs.launchpad.net/nova/+bug/1357578

I've added some extra data and marked it as a dup.

Dan


 
 There were a ton of 2012 nova bugs that were basically post it notes.
 Oh, we should refactor this function. Full stop. While those are fine
 for personal tracking, their value goes to zero probably 3 months after
 they are files, especially if the reporter stops working on the issue at
 hand. Nova has plenty of wouldn't it be great if we...  ideas. I'm not
 convinced using bugs for those is useful unless we go and close them out
 aggressively if they stall.
 
 Also, if Nova core can't file a good bug, it's hard to set the example
 for others in our community.
 
 Recommendation #2: hey, Nova core, lets be better about filing the kinds
 of bugs we want to see! mkay!
 
 Recommendation #3: Let's create a tag for personal work items or
 something for these class of TODOs people are leaving themselves that
 make them a ton easier to cull later when they stall and no one else has
 enough context to pick them up.
 
 == Tags ==
 
 The aggressive tagging that Tracy brought into the project has been
 awesome. It definitely helps slice out into better functional areas.
 Here is the top of our current official tag list (and bug count):
 
 95 compute
 83 

Re: [openstack-dev] [nova] 2 weeks in the bug tracker

2014-09-23 Thread Mark McClain

On Sep 19, 2014, at 9:13 AM, Sean Dague s...@dague.net wrote:

 Cross interaction with Neutron and Cinder remains racey. We are pretty
 optimistic on when resources will be available. Even the event interface
 with Neutron hasn't fully addressed this. I think a really great Design
 Summit session would be Nova + Neutron + Cinder to figure out a shared
 architecture to address this. I'd expect this to be at least a double
 session.

+1000

mark___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] 2 weeks in the bug tracker

2014-09-23 Thread Doug Hellmann

On Sep 23, 2014, at 1:24 PM, Mark McClain m...@mcclain.xyz wrote:

 
 On Sep 19, 2014, at 9:13 AM, Sean Dague s...@dague.net wrote:
 
 Cross interaction with Neutron and Cinder remains racey. We are pretty
 optimistic on when resources will be available. Even the event interface
 with Neutron hasn't fully addressed this. I think a really great Design
 Summit session would be Nova + Neutron + Cinder to figure out a shared
 architecture to address this. I'd expect this to be at least a double
 session.
 
 +1000
 
 mark

It’s possible we could add some features to tooz (a new Oslo synchronization 
library) to help with this. Julien is the lead on tooz.

Doug


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] 2 weeks in the bug tracker

2014-09-23 Thread Devananda van der Veen
On Mon, Sep 22, 2014 at 7:51 AM, Jay S. Bryant
jsbry...@electronicjungle.net wrote:

 On 09/21/2014 07:37 PM, Matt Riedemann wrote:

 When I'm essentially +2 on a change but for a small issue like typos in
 the commit message, the need for a note in the code or a test (or change to
 a test), I've been doing those myself lately and then will give the +2.  If
 the change already has a +2 and I'd be +W but for said things, I'm more
 inclined lately to approve and then push a dependent patch on top of it with
 the changes to keep things from stalling.

 This might be a change in my workflow just because we're late in the
 release and want good bug fixes getting into the release candidates, it
 could be because of the weekly tirade of how the project is going down the
 toilet and we don't get enough things reviewed/approved, I'm not sure, but
 my point is I agree with making it socially acceptable to rewrite the commit
 message as part of the review.

 Matt,

 This is consistent with what I have been doing for Cinder as well. I know
 there are some people who prefer I not touch the commit messages and I
 respect those requests, but otherwise I make changes to keep the process
 moving.

 Jay

This is also consistent with how I've been doing things in Ironic and
I have been encouraging the core team to use their judgement when
doing this as well -- especially when it's a patch from someone we
know won't get back to it for a while (eg, because they're on
vacation) or someone that has already OK'd this workflow (eg, other
members of the core team, and regular developers we know).

-Devananda

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] 2 weeks in the bug tracker

2014-09-22 Thread Jay S. Bryant


On 09/21/2014 07:37 PM, Matt Riedemann wrote:



On 9/19/2014 8:13 AM, Sean Dague wrote:

I've spent the better part of the last 2 weeks in the Nova bug tracker
to try to turn it into something that doesn't cause people to run away
screaming. I don't remember exactly where we started at open bug count 2
weeks ago (it was north of 1400, with  200 bugs in new, but it might
have been north of 1600), but as of this email we're at  1000 open bugs
(I'm counting Fix Committed as closed, even though LP does not), and ~0
new bugs (depending on the time of the day).

== Philosophy in Triaging ==

I'm going to lay out the philosophy of triaging I've had, because this
may also set the tone going forward.

A bug tracker is a tool to help us make a better release. It does not
exist for it's own good, it exists to help. Which means when evaluating
what stays in and what leaves we need to evaluate if any particular
artifact will help us make a better release. But also more importantly
realize that there is a cost for carrying every artifact in the tracker.
Resolving duplicates gets non linearly harder as the number of artifacts
go up. Triaging gets non-linearly hard as the number of artifacts go up.

With this I was being somewhat pragmatic about closing bugs. An old bug
that is just a stacktrace is typically not useful. An old bug that is a
vague sentence that we should refactor a particular module (with no
specifics on the details) is not useful. A bug reported against a very
old version of OpenStack where the code has changed a lot in the
relevant area, and there aren't responses from the author, is not
useful. Not useful bugs just add debt, and we should get rid of them.
That makes the chance of pulling a random bug off the tracker something
that you could actually look at fixing, instead of mostly just 
stalling out.


So I closed a lot of stuff as Invalid / Opinion that fell into those 
camps.


== Keeping New Bugs at close to 0 ==

After driving the bugs in the New state down to zero last week, I found
it's actually pretty easy to keep it at 0.

We get 10 - 20 new bugs a day in Nova (during a weekday). Of those ~20%
aren't actually a bug, and can be closed immediately. ~30% look like a
bug, but don't have anywhere near enough information in them, and
flipping them to incomplete with questions quickly means we have a real
chance of getting the right info. ~10% are fixable in  30 minutes worth
of work. And the rest are real bugs, that seem to have enough to dive
into it, and can be triaged into Confirmed, set a priority, and add the
appropriate tags for the area.

But, more importantly, this means we can filter bug quality on the way
in. And we can also encourage bug reporters that are giving us good
stuff, or even easy stuff, as we respond quickly.

Recommendation #1: we adopt a 0 new bugs policy to keep this from
getting away from us in the future.

== Our worse bug reporters are often core reviewers ==

I'm going to pick on Dan Prince here, mostly because I have a recent
concrete example, however in triaging the bug queue much of the core
team is to blame (including myself).

https://bugs.launchpad.net/nova/+bug/1368773 is a terrible bug. Also, it
was set incomplete and no response. I'm almost 100% sure it's a dupe of
the multiprocess bug we've been tracking down but it's so terse that you
can't get to the bottom of it.

There were a ton of 2012 nova bugs that were basically post it notes.
Oh, we should refactor this function. Full stop. While those are fine
for personal tracking, their value goes to zero probably 3 months after
they are files, especially if the reporter stops working on the issue at
hand. Nova has plenty of wouldn't it be great if we...  ideas. I'm not
convinced using bugs for those is useful unless we go and close them out
aggressively if they stall.

Also, if Nova core can't file a good bug, it's hard to set the example
for others in our community.

Recommendation #2: hey, Nova core, lets be better about filing the kinds
of bugs we want to see! mkay!

Recommendation #3: Let's create a tag for personal work items or
something for these class of TODOs people are leaving themselves that
make them a ton easier to cull later when they stall and no one else has
enough context to pick them up.

== Tags ==

The aggressive tagging that Tracy brought into the project has been
awesome. It definitely helps slice out into better functional areas.
Here is the top of our current official tag list (and bug count):

95 compute
83 libvirt
74 api
68 vmware
67 network
41 db
40 testing
40 volumes
36 ec2
35 icehouse-backport-potential
32 low-hanging-fruit
31 xenserver
25 ironic
23 hyper-v
16 cells
14 scheduler
12 baremetal
9 ceph
9 security
8 oslo
...

So, good stuff. However I think we probably want to take a further step
and attempt to get champions for tags. So that tag owners would ensure
their bug list looks sane, and actually spend some time fixing them.
It's pretty clear, for instance, that the ec2 bugs 

Re: [openstack-dev] [nova] 2 weeks in the bug tracker

2014-09-21 Thread Kevin Benton
I think we are limited by the statuses available in launchpad, which
doesn't have a stale status. [1]

1. https://help.launchpad.net/Bugs/Statuses

On Sat, Sep 20, 2014 at 7:25 PM, Preston L. Bannister
pres...@bannister.us wrote:
 This is great. On the point of:

 If an Incomplete bug has no response after 30 days it's fair game to
 close (Invalid, Opinion, Won't Fix).


 How about Stale ... since that is where it is. (How hard to add a state?)




 On Fri, Sep 19, 2014 at 6:13 AM, Sean Dague s...@dague.net wrote:

 I've spent the better part of the last 2 weeks in the Nova bug tracker
 to try to turn it into something that doesn't cause people to run away
 screaming. I don't remember exactly where we started at open bug count 2
 weeks ago (it was north of 1400, with  200 bugs in new, but it might
 have been north of 1600), but as of this email we're at  1000 open bugs
 (I'm counting Fix Committed as closed, even though LP does not), and ~0
 new bugs (depending on the time of the day).

 == Philosophy in Triaging ==

 I'm going to lay out the philosophy of triaging I've had, because this
 may also set the tone going forward.

 A bug tracker is a tool to help us make a better release. It does not
 exist for it's own good, it exists to help. Which means when evaluating
 what stays in and what leaves we need to evaluate if any particular
 artifact will help us make a better release. But also more importantly
 realize that there is a cost for carrying every artifact in the tracker.
 Resolving duplicates gets non linearly harder as the number of artifacts
 go up. Triaging gets non-linearly hard as the number of artifacts go up.

 With this I was being somewhat pragmatic about closing bugs. An old bug
 that is just a stacktrace is typically not useful. An old bug that is a
 vague sentence that we should refactor a particular module (with no
 specifics on the details) is not useful. A bug reported against a very
 old version of OpenStack where the code has changed a lot in the
 relevant area, and there aren't responses from the author, is not
 useful. Not useful bugs just add debt, and we should get rid of them.
 That makes the chance of pulling a random bug off the tracker something
 that you could actually look at fixing, instead of mostly just stalling
 out.

 So I closed a lot of stuff as Invalid / Opinion that fell into those
 camps.

 == Keeping New Bugs at close to 0 ==

 After driving the bugs in the New state down to zero last week, I found
 it's actually pretty easy to keep it at 0.

 We get 10 - 20 new bugs a day in Nova (during a weekday). Of those ~20%
 aren't actually a bug, and can be closed immediately. ~30% look like a
 bug, but don't have anywhere near enough information in them, and
 flipping them to incomplete with questions quickly means we have a real
 chance of getting the right info. ~10% are fixable in  30 minutes worth
 of work. And the rest are real bugs, that seem to have enough to dive
 into it, and can be triaged into Confirmed, set a priority, and add the
 appropriate tags for the area.

 But, more importantly, this means we can filter bug quality on the way
 in. And we can also encourage bug reporters that are giving us good
 stuff, or even easy stuff, as we respond quickly.

 Recommendation #1: we adopt a 0 new bugs policy to keep this from
 getting away from us in the future.

 == Our worse bug reporters are often core reviewers ==

 I'm going to pick on Dan Prince here, mostly because I have a recent
 concrete example, however in triaging the bug queue much of the core
 team is to blame (including myself).

 https://bugs.launchpad.net/nova/+bug/1368773 is a terrible bug. Also, it
 was set incomplete and no response. I'm almost 100% sure it's a dupe of
 the multiprocess bug we've been tracking down but it's so terse that you
 can't get to the bottom of it.

 There were a ton of 2012 nova bugs that were basically post it notes.
 Oh, we should refactor this function. Full stop. While those are fine
 for personal tracking, their value goes to zero probably 3 months after
 they are files, especially if the reporter stops working on the issue at
 hand. Nova has plenty of wouldn't it be great if we...  ideas. I'm not
 convinced using bugs for those is useful unless we go and close them out
 aggressively if they stall.

 Also, if Nova core can't file a good bug, it's hard to set the example
 for others in our community.

 Recommendation #2: hey, Nova core, lets be better about filing the kinds
 of bugs we want to see! mkay!

 Recommendation #3: Let's create a tag for personal work items or
 something for these class of TODOs people are leaving themselves that
 make them a ton easier to cull later when they stall and no one else has
 enough context to pick them up.

 == Tags ==

 The aggressive tagging that Tracy brought into the project has been
 awesome. It definitely helps slice out into better functional areas.
 Here is the top of our current official tag list (and bug count):

 95 

Re: [openstack-dev] [nova] 2 weeks in the bug tracker

2014-09-21 Thread Matt Riedemann



On 9/19/2014 8:13 AM, Sean Dague wrote:

I've spent the better part of the last 2 weeks in the Nova bug tracker
to try to turn it into something that doesn't cause people to run away
screaming. I don't remember exactly where we started at open bug count 2
weeks ago (it was north of 1400, with  200 bugs in new, but it might
have been north of 1600), but as of this email we're at  1000 open bugs
(I'm counting Fix Committed as closed, even though LP does not), and ~0
new bugs (depending on the time of the day).

== Philosophy in Triaging ==

I'm going to lay out the philosophy of triaging I've had, because this
may also set the tone going forward.

A bug tracker is a tool to help us make a better release. It does not
exist for it's own good, it exists to help. Which means when evaluating
what stays in and what leaves we need to evaluate if any particular
artifact will help us make a better release. But also more importantly
realize that there is a cost for carrying every artifact in the tracker.
Resolving duplicates gets non linearly harder as the number of artifacts
go up. Triaging gets non-linearly hard as the number of artifacts go up.

With this I was being somewhat pragmatic about closing bugs. An old bug
that is just a stacktrace is typically not useful. An old bug that is a
vague sentence that we should refactor a particular module (with no
specifics on the details) is not useful. A bug reported against a very
old version of OpenStack where the code has changed a lot in the
relevant area, and there aren't responses from the author, is not
useful. Not useful bugs just add debt, and we should get rid of them.
That makes the chance of pulling a random bug off the tracker something
that you could actually look at fixing, instead of mostly just stalling out.

So I closed a lot of stuff as Invalid / Opinion that fell into those camps.

== Keeping New Bugs at close to 0 ==

After driving the bugs in the New state down to zero last week, I found
it's actually pretty easy to keep it at 0.

We get 10 - 20 new bugs a day in Nova (during a weekday). Of those ~20%
aren't actually a bug, and can be closed immediately. ~30% look like a
bug, but don't have anywhere near enough information in them, and
flipping them to incomplete with questions quickly means we have a real
chance of getting the right info. ~10% are fixable in  30 minutes worth
of work. And the rest are real bugs, that seem to have enough to dive
into it, and can be triaged into Confirmed, set a priority, and add the
appropriate tags for the area.

But, more importantly, this means we can filter bug quality on the way
in. And we can also encourage bug reporters that are giving us good
stuff, or even easy stuff, as we respond quickly.

Recommendation #1: we adopt a 0 new bugs policy to keep this from
getting away from us in the future.

== Our worse bug reporters are often core reviewers ==

I'm going to pick on Dan Prince here, mostly because I have a recent
concrete example, however in triaging the bug queue much of the core
team is to blame (including myself).

https://bugs.launchpad.net/nova/+bug/1368773 is a terrible bug. Also, it
was set incomplete and no response. I'm almost 100% sure it's a dupe of
the multiprocess bug we've been tracking down but it's so terse that you
can't get to the bottom of it.

There were a ton of 2012 nova bugs that were basically post it notes.
Oh, we should refactor this function. Full stop. While those are fine
for personal tracking, their value goes to zero probably 3 months after
they are files, especially if the reporter stops working on the issue at
hand. Nova has plenty of wouldn't it be great if we...  ideas. I'm not
convinced using bugs for those is useful unless we go and close them out
aggressively if they stall.

Also, if Nova core can't file a good bug, it's hard to set the example
for others in our community.

Recommendation #2: hey, Nova core, lets be better about filing the kinds
of bugs we want to see! mkay!

Recommendation #3: Let's create a tag for personal work items or
something for these class of TODOs people are leaving themselves that
make them a ton easier to cull later when they stall and no one else has
enough context to pick them up.

== Tags ==

The aggressive tagging that Tracy brought into the project has been
awesome. It definitely helps slice out into better functional areas.
Here is the top of our current official tag list (and bug count):

95 compute
83 libvirt
74 api
68 vmware
67 network
41 db
40 testing
40 volumes
36 ec2
35 icehouse-backport-potential
32 low-hanging-fruit
31 xenserver
25 ironic
23 hyper-v
16 cells
14 scheduler
12 baremetal
9 ceph
9 security
8 oslo
...

So, good stuff. However I think we probably want to take a further step
and attempt to get champions for tags. So that tag owners would ensure
their bug list looks sane, and actually spend some time fixing them.
It's pretty clear, for instance, that the ec2 bugs are just piling up,
and very few fixes coming in. 

Re: [openstack-dev] [nova] 2 weeks in the bug tracker

2014-09-20 Thread Preston L. Bannister
This is great. On the point of:

If an Incomplete bug has no response after 30 days it's fair game to
 close (Invalid, Opinion, Won't Fix).


How about Stale ... since that is where it is. (How hard to add a state?)




On Fri, Sep 19, 2014 at 6:13 AM, Sean Dague s...@dague.net wrote:

 I've spent the better part of the last 2 weeks in the Nova bug tracker
 to try to turn it into something that doesn't cause people to run away
 screaming. I don't remember exactly where we started at open bug count 2
 weeks ago (it was north of 1400, with  200 bugs in new, but it might
 have been north of 1600), but as of this email we're at  1000 open bugs
 (I'm counting Fix Committed as closed, even though LP does not), and ~0
 new bugs (depending on the time of the day).

 == Philosophy in Triaging ==

 I'm going to lay out the philosophy of triaging I've had, because this
 may also set the tone going forward.

 A bug tracker is a tool to help us make a better release. It does not
 exist for it's own good, it exists to help. Which means when evaluating
 what stays in and what leaves we need to evaluate if any particular
 artifact will help us make a better release. But also more importantly
 realize that there is a cost for carrying every artifact in the tracker.
 Resolving duplicates gets non linearly harder as the number of artifacts
 go up. Triaging gets non-linearly hard as the number of artifacts go up.

 With this I was being somewhat pragmatic about closing bugs. An old bug
 that is just a stacktrace is typically not useful. An old bug that is a
 vague sentence that we should refactor a particular module (with no
 specifics on the details) is not useful. A bug reported against a very
 old version of OpenStack where the code has changed a lot in the
 relevant area, and there aren't responses from the author, is not
 useful. Not useful bugs just add debt, and we should get rid of them.
 That makes the chance of pulling a random bug off the tracker something
 that you could actually look at fixing, instead of mostly just stalling
 out.

 So I closed a lot of stuff as Invalid / Opinion that fell into those camps.

 == Keeping New Bugs at close to 0 ==

 After driving the bugs in the New state down to zero last week, I found
 it's actually pretty easy to keep it at 0.

 We get 10 - 20 new bugs a day in Nova (during a weekday). Of those ~20%
 aren't actually a bug, and can be closed immediately. ~30% look like a
 bug, but don't have anywhere near enough information in them, and
 flipping them to incomplete with questions quickly means we have a real
 chance of getting the right info. ~10% are fixable in  30 minutes worth
 of work. And the rest are real bugs, that seem to have enough to dive
 into it, and can be triaged into Confirmed, set a priority, and add the
 appropriate tags for the area.

 But, more importantly, this means we can filter bug quality on the way
 in. And we can also encourage bug reporters that are giving us good
 stuff, or even easy stuff, as we respond quickly.

 Recommendation #1: we adopt a 0 new bugs policy to keep this from
 getting away from us in the future.

 == Our worse bug reporters are often core reviewers ==

 I'm going to pick on Dan Prince here, mostly because I have a recent
 concrete example, however in triaging the bug queue much of the core
 team is to blame (including myself).

 https://bugs.launchpad.net/nova/+bug/1368773 is a terrible bug. Also, it
 was set incomplete and no response. I'm almost 100% sure it's a dupe of
 the multiprocess bug we've been tracking down but it's so terse that you
 can't get to the bottom of it.

 There were a ton of 2012 nova bugs that were basically post it notes.
 Oh, we should refactor this function. Full stop. While those are fine
 for personal tracking, their value goes to zero probably 3 months after
 they are files, especially if the reporter stops working on the issue at
 hand. Nova has plenty of wouldn't it be great if we...  ideas. I'm not
 convinced using bugs for those is useful unless we go and close them out
 aggressively if they stall.

 Also, if Nova core can't file a good bug, it's hard to set the example
 for others in our community.

 Recommendation #2: hey, Nova core, lets be better about filing the kinds
 of bugs we want to see! mkay!

 Recommendation #3: Let's create a tag for personal work items or
 something for these class of TODOs people are leaving themselves that
 make them a ton easier to cull later when they stall and no one else has
 enough context to pick them up.

 == Tags ==

 The aggressive tagging that Tracy brought into the project has been
 awesome. It definitely helps slice out into better functional areas.
 Here is the top of our current official tag list (and bug count):

 95 compute
 83 libvirt
 74 api
 68 vmware
 67 network
 41 db
 40 testing
 40 volumes
 36 ec2
 35 icehouse-backport-potential
 32 low-hanging-fruit
 31 xenserver
 25 ironic
 23 hyper-v
 16 cells
 14 scheduler
 12 baremetal
 9 ceph
 9 

[openstack-dev] [nova] 2 weeks in the bug tracker

2014-09-19 Thread Sean Dague
I've spent the better part of the last 2 weeks in the Nova bug tracker
to try to turn it into something that doesn't cause people to run away
screaming. I don't remember exactly where we started at open bug count 2
weeks ago (it was north of 1400, with  200 bugs in new, but it might
have been north of 1600), but as of this email we're at  1000 open bugs
(I'm counting Fix Committed as closed, even though LP does not), and ~0
new bugs (depending on the time of the day).

== Philosophy in Triaging ==

I'm going to lay out the philosophy of triaging I've had, because this
may also set the tone going forward.

A bug tracker is a tool to help us make a better release. It does not
exist for it's own good, it exists to help. Which means when evaluating
what stays in and what leaves we need to evaluate if any particular
artifact will help us make a better release. But also more importantly
realize that there is a cost for carrying every artifact in the tracker.
Resolving duplicates gets non linearly harder as the number of artifacts
go up. Triaging gets non-linearly hard as the number of artifacts go up.

With this I was being somewhat pragmatic about closing bugs. An old bug
that is just a stacktrace is typically not useful. An old bug that is a
vague sentence that we should refactor a particular module (with no
specifics on the details) is not useful. A bug reported against a very
old version of OpenStack where the code has changed a lot in the
relevant area, and there aren't responses from the author, is not
useful. Not useful bugs just add debt, and we should get rid of them.
That makes the chance of pulling a random bug off the tracker something
that you could actually look at fixing, instead of mostly just stalling out.

So I closed a lot of stuff as Invalid / Opinion that fell into those camps.

== Keeping New Bugs at close to 0 ==

After driving the bugs in the New state down to zero last week, I found
it's actually pretty easy to keep it at 0.

We get 10 - 20 new bugs a day in Nova (during a weekday). Of those ~20%
aren't actually a bug, and can be closed immediately. ~30% look like a
bug, but don't have anywhere near enough information in them, and
flipping them to incomplete with questions quickly means we have a real
chance of getting the right info. ~10% are fixable in  30 minutes worth
of work. And the rest are real bugs, that seem to have enough to dive
into it, and can be triaged into Confirmed, set a priority, and add the
appropriate tags for the area.

But, more importantly, this means we can filter bug quality on the way
in. And we can also encourage bug reporters that are giving us good
stuff, or even easy stuff, as we respond quickly.

Recommendation #1: we adopt a 0 new bugs policy to keep this from
getting away from us in the future.

== Our worse bug reporters are often core reviewers ==

I'm going to pick on Dan Prince here, mostly because I have a recent
concrete example, however in triaging the bug queue much of the core
team is to blame (including myself).

https://bugs.launchpad.net/nova/+bug/1368773 is a terrible bug. Also, it
was set incomplete and no response. I'm almost 100% sure it's a dupe of
the multiprocess bug we've been tracking down but it's so terse that you
can't get to the bottom of it.

There were a ton of 2012 nova bugs that were basically post it notes.
Oh, we should refactor this function. Full stop. While those are fine
for personal tracking, their value goes to zero probably 3 months after
they are files, especially if the reporter stops working on the issue at
hand. Nova has plenty of wouldn't it be great if we...  ideas. I'm not
convinced using bugs for those is useful unless we go and close them out
aggressively if they stall.

Also, if Nova core can't file a good bug, it's hard to set the example
for others in our community.

Recommendation #2: hey, Nova core, lets be better about filing the kinds
of bugs we want to see! mkay!

Recommendation #3: Let's create a tag for personal work items or
something for these class of TODOs people are leaving themselves that
make them a ton easier to cull later when they stall and no one else has
enough context to pick them up.

== Tags ==

The aggressive tagging that Tracy brought into the project has been
awesome. It definitely helps slice out into better functional areas.
Here is the top of our current official tag list (and bug count):

95 compute
83 libvirt
74 api
68 vmware
67 network
41 db
40 testing
40 volumes
36 ec2
35 icehouse-backport-potential
32 low-hanging-fruit
31 xenserver
25 ironic
23 hyper-v
16 cells
14 scheduler
12 baremetal
9 ceph
9 security
8 oslo
...

So, good stuff. However I think we probably want to take a further step
and attempt to get champions for tags. So that tag owners would ensure
their bug list looks sane, and actually spend some time fixing them.
It's pretty clear, for instance, that the ec2 bugs are just piling up,
and very few fixes coming in. Cells seems like it's in the same camp (a
bunch 

Re: [openstack-dev] [nova] 2 weeks in the bug tracker

2014-09-19 Thread Sean Dague
On 09/19/2014 09:58 AM, Sylvain Bauza wrote:
snip
 == Keeping New Bugs at close to 0 ==

 After driving the bugs in the New state down to zero last week, I found
 it's actually pretty easy to keep it at 0.

 We get 10 - 20 new bugs a day in Nova (during a weekday). Of those ~20%
 aren't actually a bug, and can be closed immediately. ~30% look like a
 bug, but don't have anywhere near enough information in them, and
 flipping them to incomplete with questions quickly means we have a real
 chance of getting the right info. ~10% are fixable in  30 minutes worth
 of work. And the rest are real bugs, that seem to have enough to dive
 into it, and can be triaged into Confirmed, set a priority, and add the
 appropriate tags for the area.

 But, more importantly, this means we can filter bug quality on the way
 in. And we can also encourage bug reporters that are giving us good
 stuff, or even easy stuff, as we respond quickly.

 Recommendation #1: we adopt a 0 new bugs policy to keep this from
 getting away from us in the future.
 
 Agreed, provided we can review all the new bugs each week.

So I actually don't think this works if it's a weekly thing. Keeping new
bugs at 0 really has to be daily because the response to bug reports
sets up the expected cadence with the reporter. If you flip back new
bugs in  6 or 8 hrs, there is a decent chance they are still on their
same work shift, and the context is still in their head (or even the
situation is still existing).

Once you pass 24hrs the chance of that goes way down. And,
realistically, I've found that when I open the bug tracker in the
morning and there are 5 bugs, that's totally doable over the first cup
of coffee. Poking the bug tracker a couple more times during the day is
all that's needed to keep it there.

 == Our worse bug reporters are often core reviewers ==

 I'm going to pick on Dan Prince here, mostly because I have a recent
 concrete example, however in triaging the bug queue much of the core
 team is to blame (including myself).

 https://bugs.launchpad.net/nova/+bug/1368773 is a terrible bug. Also, it
 was set incomplete and no response. I'm almost 100% sure it's a dupe of
 the multiprocess bug we've been tracking down but it's so terse that you
 can't get to the bottom of it.

 There were a ton of 2012 nova bugs that were basically post it notes.
 Oh, we should refactor this function. Full stop. While those are fine
 for personal tracking, their value goes to zero probably 3 months after
 they are files, especially if the reporter stops working on the issue at
 hand. Nova has plenty of wouldn't it be great if we...  ideas. I'm not
 convinced using bugs for those is useful unless we go and close them out
 aggressively if they stall.

 Also, if Nova core can't file a good bug, it's hard to set the example
 for others in our community.

 Recommendation #2: hey, Nova core, lets be better about filing the kinds
 of bugs we want to see! mkay!

 Recommendation #3: Let's create a tag for personal work items or
 something for these class of TODOs people are leaving themselves that
 make them a ton easier to cull later when they stall and no one else has
 enough context to pick them up.
 
 I would propose to set their importance as Wishlist then. I would
 leave the tags for setting which components are impacted.

Maybe. I honestly don't think core team members should file wishlist
bugs at all. That really means feature and means a spec. Or it means
just do it (for refactoring).

 == Tags ==

 The aggressive tagging that Tracy brought into the project has been
 awesome. It definitely helps slice out into better functional areas.
 Here is the top of our current official tag list (and bug count):

 95 compute
 83 libvirt
 74 api
 68 vmware
 67 network
 41 db
 40 testing
 40 volumes
 36 ec2
 35 icehouse-backport-potential
 32 low-hanging-fruit
 31 xenserver
 25 ironic
 23 hyper-v
 16 cells
 14 scheduler
 12 baremetal
 9 ceph
 9 security
 8 oslo
 ...

 So, good stuff. However I think we probably want to take a further step
 and attempt to get champions for tags. So that tag owners would ensure
 their bug list looks sane, and actually spend some time fixing them.
 It's pretty clear, for instance, that the ec2 bugs are just piling up,
 and very few fixes coming in. Cells seems like it's in the same camp (a
 bunch of recent bugs have been cells related, it looks like a lot more
 deployments are trying it).

 Probably the most important thing in tag owners would be cleaning up the
 bugs in the tag. Realizing that 2 bugs were actually the same bug.
 Cleaning up descriptions / titles / etc so that people can move forward
 on them.

 Recommendation #4: create tag champions
 
 +1. That said, some bugs can be having more than 1 tag (for example,
 compute/conductor/scheduler), so it would mean the champions would have
 to discuss between them.
 I can volunteer for looking at the scheduler tag.

Sure. But more communication on issues is only a good thing. :)

 
 == Soft 

Re: [openstack-dev] [nova] 2 weeks in the bug tracker

2014-09-19 Thread Jay Pipes
Sorry for top-posting. Just want to say thanks for writing this up and I 
agree with all the points and recommendations you made.


-jay

On 09/19/2014 09:13 AM, Sean Dague wrote:

I've spent the better part of the last 2 weeks in the Nova bug tracker
to try to turn it into something that doesn't cause people to run away
screaming. I don't remember exactly where we started at open bug count 2
weeks ago (it was north of 1400, with  200 bugs in new, but it might
have been north of 1600), but as of this email we're at  1000 open bugs
(I'm counting Fix Committed as closed, even though LP does not), and ~0
new bugs (depending on the time of the day).

== Philosophy in Triaging ==

I'm going to lay out the philosophy of triaging I've had, because this
may also set the tone going forward.

A bug tracker is a tool to help us make a better release. It does not
exist for it's own good, it exists to help. Which means when evaluating
what stays in and what leaves we need to evaluate if any particular
artifact will help us make a better release. But also more importantly
realize that there is a cost for carrying every artifact in the tracker.
Resolving duplicates gets non linearly harder as the number of artifacts
go up. Triaging gets non-linearly hard as the number of artifacts go up.

With this I was being somewhat pragmatic about closing bugs. An old bug
that is just a stacktrace is typically not useful. An old bug that is a
vague sentence that we should refactor a particular module (with no
specifics on the details) is not useful. A bug reported against a very
old version of OpenStack where the code has changed a lot in the
relevant area, and there aren't responses from the author, is not
useful. Not useful bugs just add debt, and we should get rid of them.
That makes the chance of pulling a random bug off the tracker something
that you could actually look at fixing, instead of mostly just stalling out.

So I closed a lot of stuff as Invalid / Opinion that fell into those camps.

== Keeping New Bugs at close to 0 ==

After driving the bugs in the New state down to zero last week, I found
it's actually pretty easy to keep it at 0.

We get 10 - 20 new bugs a day in Nova (during a weekday). Of those ~20%
aren't actually a bug, and can be closed immediately. ~30% look like a
bug, but don't have anywhere near enough information in them, and
flipping them to incomplete with questions quickly means we have a real
chance of getting the right info. ~10% are fixable in  30 minutes worth
of work. And the rest are real bugs, that seem to have enough to dive
into it, and can be triaged into Confirmed, set a priority, and add the
appropriate tags for the area.

But, more importantly, this means we can filter bug quality on the way
in. And we can also encourage bug reporters that are giving us good
stuff, or even easy stuff, as we respond quickly.

Recommendation #1: we adopt a 0 new bugs policy to keep this from
getting away from us in the future.

== Our worse bug reporters are often core reviewers ==

I'm going to pick on Dan Prince here, mostly because I have a recent
concrete example, however in triaging the bug queue much of the core
team is to blame (including myself).

https://bugs.launchpad.net/nova/+bug/1368773 is a terrible bug. Also, it
was set incomplete and no response. I'm almost 100% sure it's a dupe of
the multiprocess bug we've been tracking down but it's so terse that you
can't get to the bottom of it.

There were a ton of 2012 nova bugs that were basically post it notes.
Oh, we should refactor this function. Full stop. While those are fine
for personal tracking, their value goes to zero probably 3 months after
they are files, especially if the reporter stops working on the issue at
hand. Nova has plenty of wouldn't it be great if we...  ideas. I'm not
convinced using bugs for those is useful unless we go and close them out
aggressively if they stall.

Also, if Nova core can't file a good bug, it's hard to set the example
for others in our community.

Recommendation #2: hey, Nova core, lets be better about filing the kinds
of bugs we want to see! mkay!

Recommendation #3: Let's create a tag for personal work items or
something for these class of TODOs people are leaving themselves that
make them a ton easier to cull later when they stall and no one else has
enough context to pick them up.

== Tags ==

The aggressive tagging that Tracy brought into the project has been
awesome. It definitely helps slice out into better functional areas.
Here is the top of our current official tag list (and bug count):

95 compute
83 libvirt
74 api
68 vmware
67 network
41 db
40 testing
40 volumes
36 ec2
35 icehouse-backport-potential
32 low-hanging-fruit
31 xenserver
25 ironic
23 hyper-v
16 cells
14 scheduler
12 baremetal
9 ceph
9 security
8 oslo
...

So, good stuff. However I think we probably want to take a further step
and attempt to get champions for tags. So that tag owners would ensure
their bug list looks sane, and 

Re: [openstack-dev] [nova] 2 weeks in the bug tracker

2014-09-19 Thread Michael Still
On Fri, Sep 19, 2014 at 11:13 PM, Sean Dague s...@dague.net wrote:
 I've spent the better part of the last 2 weeks in the Nova bug tracker
 to try to turn it into something that doesn't cause people to run away
 screaming. I don't remember exactly where we started at open bug count 2
 weeks ago (it was north of 1400, with  200 bugs in new, but it might
 have been north of 1600), but as of this email we're at  1000 open bugs
 (I'm counting Fix Committed as closed, even though LP does not), and ~0
 new bugs (depending on the time of the day).

 == Philosophy in Triaging ==

 I'm going to lay out the philosophy of triaging I've had, because this
 may also set the tone going forward.

 A bug tracker is a tool to help us make a better release. It does not
 exist for it's own good, it exists to help. Which means when evaluating
 what stays in and what leaves we need to evaluate if any particular
 artifact will help us make a better release. But also more importantly
 realize that there is a cost for carrying every artifact in the tracker.
 Resolving duplicates gets non linearly harder as the number of artifacts
 go up. Triaging gets non-linearly hard as the number of artifacts go up.

 With this I was being somewhat pragmatic about closing bugs. An old bug
 that is just a stacktrace is typically not useful. An old bug that is a
 vague sentence that we should refactor a particular module (with no
 specifics on the details) is not useful. A bug reported against a very
 old version of OpenStack where the code has changed a lot in the
 relevant area, and there aren't responses from the author, is not
 useful. Not useful bugs just add debt, and we should get rid of them.
 That makes the chance of pulling a random bug off the tracker something
 that you could actually look at fixing, instead of mostly just stalling out.

 So I closed a lot of stuff as Invalid / Opinion that fell into those camps.

 == Keeping New Bugs at close to 0 ==

 After driving the bugs in the New state down to zero last week, I found
 it's actually pretty easy to keep it at 0.

 We get 10 - 20 new bugs a day in Nova (during a weekday). Of those ~20%
 aren't actually a bug, and can be closed immediately. ~30% look like a
 bug, but don't have anywhere near enough information in them, and
 flipping them to incomplete with questions quickly means we have a real
 chance of getting the right info. ~10% are fixable in  30 minutes worth
 of work. And the rest are real bugs, that seem to have enough to dive
 into it, and can be triaged into Confirmed, set a priority, and add the
 appropriate tags for the area.

On the bugs which would take less than 30 minutes, is that because
they're not bugs, or are they just trivial? It would be cool to be
adding the low-hanging-fruit tag to those bugs if you're not, because
we should just fix them.

 But, more importantly, this means we can filter bug quality on the way
 in. And we can also encourage bug reporters that are giving us good
 stuff, or even easy stuff, as we respond quickly.

 Recommendation #1: we adopt a 0 new bugs policy to keep this from
 getting away from us in the future.

Agreed, this was a goal we used to have back in the day and I'd like
to bring it back.

 == Our worse bug reporters are often core reviewers ==

 I'm going to pick on Dan Prince here, mostly because I have a recent
 concrete example, however in triaging the bug queue much of the core
 team is to blame (including myself).

 https://bugs.launchpad.net/nova/+bug/1368773 is a terrible bug. Also, it
 was set incomplete and no response. I'm almost 100% sure it's a dupe of
 the multiprocess bug we've been tracking down but it's so terse that you
 can't get to the bottom of it.

 There were a ton of 2012 nova bugs that were basically post it notes.
 Oh, we should refactor this function. Full stop. While those are fine
 for personal tracking, their value goes to zero probably 3 months after
 they are files, especially if the reporter stops working on the issue at
 hand. Nova has plenty of wouldn't it be great if we...  ideas. I'm not
 convinced using bugs for those is useful unless we go and close them out
 aggressively if they stall.

 Also, if Nova core can't file a good bug, it's hard to set the example
 for others in our community.

 Recommendation #2: hey, Nova core, lets be better about filing the kinds
 of bugs we want to see! mkay!

 Recommendation #3: Let's create a tag for personal work items or
 something for these class of TODOs people are leaving themselves that
 make them a ton easier to cull later when they stall and no one else has
 enough context to pick them up.

I think we also get a lot of bugs filed almost immediately before a
fix. Sort of like a tracking mechanism for micro-features. Do we want
to continue doing that, or do we want to just let smallish things land
without a bug?

 == Tags ==

 The aggressive tagging that Tracy brought into the project has been
 awesome. It definitely helps slice out into better 

Re: [openstack-dev] [nova] 2 weeks in the bug tracker

2014-09-19 Thread Ben Nemec
On 09/19/2014 08:13 AM, Sean Dague wrote:
 I've spent the better part of the last 2 weeks in the Nova bug tracker
 to try to turn it into something that doesn't cause people to run away
 screaming. I don't remember exactly where we started at open bug count 2
 weeks ago (it was north of 1400, with  200 bugs in new, but it might
 have been north of 1600), but as of this email we're at  1000 open bugs
 (I'm counting Fix Committed as closed, even though LP does not), and ~0
 new bugs (depending on the time of the day).
 
 == Philosophy in Triaging ==
 
 I'm going to lay out the philosophy of triaging I've had, because this
 may also set the tone going forward.
 
 A bug tracker is a tool to help us make a better release. It does not
 exist for it's own good, it exists to help. Which means when evaluating
 what stays in and what leaves we need to evaluate if any particular
 artifact will help us make a better release. But also more importantly
 realize that there is a cost for carrying every artifact in the tracker.
 Resolving duplicates gets non linearly harder as the number of artifacts
 go up. Triaging gets non-linearly hard as the number of artifacts go up.
 
 With this I was being somewhat pragmatic about closing bugs. An old bug
 that is just a stacktrace is typically not useful. An old bug that is a
 vague sentence that we should refactor a particular module (with no
 specifics on the details) is not useful. A bug reported against a very
 old version of OpenStack where the code has changed a lot in the
 relevant area, and there aren't responses from the author, is not
 useful. Not useful bugs just add debt, and we should get rid of them.
 That makes the chance of pulling a random bug off the tracker something
 that you could actually look at fixing, instead of mostly just stalling out.
 
 So I closed a lot of stuff as Invalid / Opinion that fell into those camps.
 
 == Keeping New Bugs at close to 0 ==
 
 After driving the bugs in the New state down to zero last week, I found
 it's actually pretty easy to keep it at 0.
 
 We get 10 - 20 new bugs a day in Nova (during a weekday). Of those ~20%
 aren't actually a bug, and can be closed immediately. ~30% look like a
 bug, but don't have anywhere near enough information in them, and
 flipping them to incomplete with questions quickly means we have a real
 chance of getting the right info. ~10% are fixable in  30 minutes worth
 of work. And the rest are real bugs, that seem to have enough to dive
 into it, and can be triaged into Confirmed, set a priority, and add the
 appropriate tags for the area.
 
 But, more importantly, this means we can filter bug quality on the way
 in. And we can also encourage bug reporters that are giving us good
 stuff, or even easy stuff, as we respond quickly.
 
 Recommendation #1: we adopt a 0 new bugs policy to keep this from
 getting away from us in the future.

We have this policy in TripleO, and to help keep it fresh in people's
minds Roman Podolyaka (IIRC) wrote an untriaged-bot for the IRC channel
that periodically posts a list of any New bugs.  I've found it very
helpful, so it's probably worth getting that into infra somewhere so
other people can use it too.

 
 == Our worse bug reporters are often core reviewers ==
 
 I'm going to pick on Dan Prince here, mostly because I have a recent
 concrete example, however in triaging the bug queue much of the core
 team is to blame (including myself).
 
 https://bugs.launchpad.net/nova/+bug/1368773 is a terrible bug. Also, it
 was set incomplete and no response. I'm almost 100% sure it's a dupe of
 the multiprocess bug we've been tracking down but it's so terse that you
 can't get to the bottom of it.
 
 There were a ton of 2012 nova bugs that were basically post it notes.
 Oh, we should refactor this function. Full stop. While those are fine
 for personal tracking, their value goes to zero probably 3 months after
 they are files, especially if the reporter stops working on the issue at
 hand. Nova has plenty of wouldn't it be great if we...  ideas. I'm not
 convinced using bugs for those is useful unless we go and close them out
 aggressively if they stall.
 
 Also, if Nova core can't file a good bug, it's hard to set the example
 for others in our community.
 
 Recommendation #2: hey, Nova core, lets be better about filing the kinds
 of bugs we want to see! mkay!
 
 Recommendation #3: Let's create a tag for personal work items or
 something for these class of TODOs people are leaving themselves that
 make them a ton easier to cull later when they stall and no one else has
 enough context to pick them up.
 
 == Tags ==
 
 The aggressive tagging that Tracy brought into the project has been
 awesome. It definitely helps slice out into better functional areas.
 Here is the top of our current official tag list (and bug count):
 
 95 compute
 83 libvirt
 74 api
 68 vmware
 67 network
 41 db
 40 testing
 40 volumes
 36 ec2
 35 icehouse-backport-potential
 32 low-hanging-fruit
 31 

Re: [openstack-dev] [nova] 2 weeks in the bug tracker

2014-09-19 Thread Anita Kuno
On 09/19/2014 05:03 PM, Ben Nemec wrote:
 On 09/19/2014 08:13 AM, Sean Dague wrote:
 I've spent the better part of the last 2 weeks in the Nova bug tracker
 to try to turn it into something that doesn't cause people to run away
 screaming. I don't remember exactly where we started at open bug count 2
 weeks ago (it was north of 1400, with  200 bugs in new, but it might
 have been north of 1600), but as of this email we're at  1000 open bugs
 (I'm counting Fix Committed as closed, even though LP does not), and ~0
 new bugs (depending on the time of the day).

 == Philosophy in Triaging ==

 I'm going to lay out the philosophy of triaging I've had, because this
 may also set the tone going forward.

 A bug tracker is a tool to help us make a better release. It does not
 exist for it's own good, it exists to help. Which means when evaluating
 what stays in and what leaves we need to evaluate if any particular
 artifact will help us make a better release. But also more importantly
 realize that there is a cost for carrying every artifact in the tracker.
 Resolving duplicates gets non linearly harder as the number of artifacts
 go up. Triaging gets non-linearly hard as the number of artifacts go up.

 With this I was being somewhat pragmatic about closing bugs. An old bug
 that is just a stacktrace is typically not useful. An old bug that is a
 vague sentence that we should refactor a particular module (with no
 specifics on the details) is not useful. A bug reported against a very
 old version of OpenStack where the code has changed a lot in the
 relevant area, and there aren't responses from the author, is not
 useful. Not useful bugs just add debt, and we should get rid of them.
 That makes the chance of pulling a random bug off the tracker something
 that you could actually look at fixing, instead of mostly just stalling out.

 So I closed a lot of stuff as Invalid / Opinion that fell into those camps.

 == Keeping New Bugs at close to 0 ==

 After driving the bugs in the New state down to zero last week, I found
 it's actually pretty easy to keep it at 0.

 We get 10 - 20 new bugs a day in Nova (during a weekday). Of those ~20%
 aren't actually a bug, and can be closed immediately. ~30% look like a
 bug, but don't have anywhere near enough information in them, and
 flipping them to incomplete with questions quickly means we have a real
 chance of getting the right info. ~10% are fixable in  30 minutes worth
 of work. And the rest are real bugs, that seem to have enough to dive
 into it, and can be triaged into Confirmed, set a priority, and add the
 appropriate tags for the area.

 But, more importantly, this means we can filter bug quality on the way
 in. And we can also encourage bug reporters that are giving us good
 stuff, or even easy stuff, as we respond quickly.

 Recommendation #1: we adopt a 0 new bugs policy to keep this from
 getting away from us in the future.
 
 We have this policy in TripleO, and to help keep it fresh in people's
 minds Roman Podolyaka (IIRC) wrote an untriaged-bot for the IRC channel
 that periodically posts a list of any New bugs.  I've found it very
 helpful, so it's probably worth getting that into infra somewhere so
 other people can use it too.
Get me the url for the source code and the name you want the thing to be
called and I will cook up the patch to get it into stackforge.

Thanks Ben,
Anita.
 

 == Our worse bug reporters are often core reviewers ==

 I'm going to pick on Dan Prince here, mostly because I have a recent
 concrete example, however in triaging the bug queue much of the core
 team is to blame (including myself).

 https://bugs.launchpad.net/nova/+bug/1368773 is a terrible bug. Also, it
 was set incomplete and no response. I'm almost 100% sure it's a dupe of
 the multiprocess bug we've been tracking down but it's so terse that you
 can't get to the bottom of it.

 There were a ton of 2012 nova bugs that were basically post it notes.
 Oh, we should refactor this function. Full stop. While those are fine
 for personal tracking, their value goes to zero probably 3 months after
 they are files, especially if the reporter stops working on the issue at
 hand. Nova has plenty of wouldn't it be great if we...  ideas. I'm not
 convinced using bugs for those is useful unless we go and close them out
 aggressively if they stall.

 Also, if Nova core can't file a good bug, it's hard to set the example
 for others in our community.

 Recommendation #2: hey, Nova core, lets be better about filing the kinds
 of bugs we want to see! mkay!

 Recommendation #3: Let's create a tag for personal work items or
 something for these class of TODOs people are leaving themselves that
 make them a ton easier to cull later when they stall and no one else has
 enough context to pick them up.

 == Tags ==

 The aggressive tagging that Tracy brought into the project has been
 awesome. It definitely helps slice out into better functional areas.
 Here is the top of our current 

Re: [openstack-dev] [nova] 2 weeks in the bug tracker

2014-09-19 Thread Roman Podoliaka
Hi all,

FWIW, a quick and dirty solution is here: http://xsnippet.org/360188/ :)

Thanks,
Roman

On Fri, Sep 19, 2014 at 2:03 PM, Ben Nemec openst...@nemebean.com wrote:
 On 09/19/2014 08:13 AM, Sean Dague wrote:
 I've spent the better part of the last 2 weeks in the Nova bug tracker
 to try to turn it into something that doesn't cause people to run away
 screaming. I don't remember exactly where we started at open bug count 2
 weeks ago (it was north of 1400, with  200 bugs in new, but it might
 have been north of 1600), but as of this email we're at  1000 open bugs
 (I'm counting Fix Committed as closed, even though LP does not), and ~0
 new bugs (depending on the time of the day).

 == Philosophy in Triaging ==

 I'm going to lay out the philosophy of triaging I've had, because this
 may also set the tone going forward.

 A bug tracker is a tool to help us make a better release. It does not
 exist for it's own good, it exists to help. Which means when evaluating
 what stays in and what leaves we need to evaluate if any particular
 artifact will help us make a better release. But also more importantly
 realize that there is a cost for carrying every artifact in the tracker.
 Resolving duplicates gets non linearly harder as the number of artifacts
 go up. Triaging gets non-linearly hard as the number of artifacts go up.

 With this I was being somewhat pragmatic about closing bugs. An old bug
 that is just a stacktrace is typically not useful. An old bug that is a
 vague sentence that we should refactor a particular module (with no
 specifics on the details) is not useful. A bug reported against a very
 old version of OpenStack where the code has changed a lot in the
 relevant area, and there aren't responses from the author, is not
 useful. Not useful bugs just add debt, and we should get rid of them.
 That makes the chance of pulling a random bug off the tracker something
 that you could actually look at fixing, instead of mostly just stalling out.

 So I closed a lot of stuff as Invalid / Opinion that fell into those camps.

 == Keeping New Bugs at close to 0 ==

 After driving the bugs in the New state down to zero last week, I found
 it's actually pretty easy to keep it at 0.

 We get 10 - 20 new bugs a day in Nova (during a weekday). Of those ~20%
 aren't actually a bug, and can be closed immediately. ~30% look like a
 bug, but don't have anywhere near enough information in them, and
 flipping them to incomplete with questions quickly means we have a real
 chance of getting the right info. ~10% are fixable in  30 minutes worth
 of work. And the rest are real bugs, that seem to have enough to dive
 into it, and can be triaged into Confirmed, set a priority, and add the
 appropriate tags for the area.

 But, more importantly, this means we can filter bug quality on the way
 in. And we can also encourage bug reporters that are giving us good
 stuff, or even easy stuff, as we respond quickly.

 Recommendation #1: we adopt a 0 new bugs policy to keep this from
 getting away from us in the future.

 We have this policy in TripleO, and to help keep it fresh in people's
 minds Roman Podolyaka (IIRC) wrote an untriaged-bot for the IRC channel
 that periodically posts a list of any New bugs.  I've found it very
 helpful, so it's probably worth getting that into infra somewhere so
 other people can use it too.


 == Our worse bug reporters are often core reviewers ==

 I'm going to pick on Dan Prince here, mostly because I have a recent
 concrete example, however in triaging the bug queue much of the core
 team is to blame (including myself).

 https://bugs.launchpad.net/nova/+bug/1368773 is a terrible bug. Also, it
 was set incomplete and no response. I'm almost 100% sure it's a dupe of
 the multiprocess bug we've been tracking down but it's so terse that you
 can't get to the bottom of it.

 There were a ton of 2012 nova bugs that were basically post it notes.
 Oh, we should refactor this function. Full stop. While those are fine
 for personal tracking, their value goes to zero probably 3 months after
 they are files, especially if the reporter stops working on the issue at
 hand. Nova has plenty of wouldn't it be great if we...  ideas. I'm not
 convinced using bugs for those is useful unless we go and close them out
 aggressively if they stall.

 Also, if Nova core can't file a good bug, it's hard to set the example
 for others in our community.

 Recommendation #2: hey, Nova core, lets be better about filing the kinds
 of bugs we want to see! mkay!

 Recommendation #3: Let's create a tag for personal work items or
 something for these class of TODOs people are leaving themselves that
 make them a ton easier to cull later when they stall and no one else has
 enough context to pick them up.

 == Tags ==

 The aggressive tagging that Tracy brought into the project has been
 awesome. It definitely helps slice out into better functional areas.
 Here is the top of our current official tag list (and bug count):