On 7/24/05, michael chang <[EMAIL PROTECTED]> wrote: > On 7/22/05, Luis Villa <[EMAIL PROTECTED]> wrote: > > The biggest problem we have with traditional reporting of bug counts > > in bugzilla is that inflow and outflow is largely determined by > > qualities other than actual code quality. In traditional bug tracking, > > number of QA people is relatively fixed, number of tests run > > relatively fixed/known, and amount of triage and fixing are also > > known. On the other hand, we have spikes of all sorts- bugday? yup, > > that made us X% less buggy in one day. Yeah. Right.[1] Sun stops > > filing bugs against HEAD? Or suddenly pours more resources into > > triage? Did our fix rate really go up, or the quality of new releases > > really go down, if those things happen? not really. So we have to > > figure out some way to control for those things. > > If I read this right, this basically says that maybe there should be a > way for e.g. a bug that is triaged as a duplicate or something to not > count as a "fixed" but thereby making it look like many bugs were > fixed, but at the same time for it not to count as "unfixed" (thereby > making it look like nothing happened).
Well, that is definitely part of it. But there are other factors- for example, we always have more users during beta than alpha, so more bugs get reported- even though we're fairly certain the product is actually more stable and after-the-fact analysis suggests that the percentage of 'high' priority bugs is lower later. [This particular issue is the one that scares the most hell out of traditional software managers, because they've been trained that if the curve of the graph is going down, that's when you're approaching release, whereas to us, if the curve of the graph is going up, then you're getting more users ;)] To further complicate things, if you haven't marked the bugs duplicate or invalid yet, how do you account for them when they are new? i.e., even in the best weeks we only barely break even on bug triage, but for stats to be most useful they have to count bugs as they come in. So, a simplified example- in a given week: 120 bugs come in 40 are marked fixed 40 are marked duplicate 20 are invalid/incomplete that leaves 20 bugs open, net. Odds are pretty good that 12 of those remaining bugs are duplicate/invalid/incomplete, so only 8 valid bugs of 120 filed. Do you report 8? 20? so on. > Is there a way to generate statistics so that triaged bugs don't count > as "fixed" bugs per se? Or something like that, e.g. bugs marked as > duplicates don't indicate "fixed" bugs, or something? [So the number > of fixed bugs is compared to the number of open bugs -- since the > triaged/dups are not open bugs, the fixed bugs will still be positive, > but not seemingly inflated.] > > This is the way I see it, and my apologies if my interpretation is nonsense. Not nonsense at all. Luis _______________________________________________ Gnome-bugsquad mailing list [email protected] http://mail.gnome.org/mailman/listinfo/gnome-bugsquad
