First impressions. On Sun, Nov 20, 2011 at 06:37:46PM -0500, Audris Mockus wrote: > To investigate efficiency and effectiveness of issue tracking we > built a tool to visualize and quantify issue resolution practices > based on what is recorded in Bugzilla. In particular, we hope that > something along these lines might be of some use to Gnome.
Seems quite interesting. However, due note that there are multiple teams involved with Bugzilla: 1) bugmasters very small group of administrators 2) bugsquad loosely connected group of people who triage bugs 3) maintainers/developers 4) users The entire triage process is decided by bugsquad themselves. I've cc'ed gnome-bugsquad as I think they'll be interested in this email as well. That is a public mailing list btw. > As an example, we investigated changes to BugBuddy and how it > affected issue reporting and resolution > (http://mockus.org/papers/demo/index.html#BB) > - A description is at http://mockus.org/papers/demo > (pdf at http://mockus.org/papers/demo.pdf) > - A link to a video demonstration : > http://www.youtube.com/watch?v=y9O37OTecbE > - A link to the tool: http://passion-lab.org/pee.html > > We would greatly appreciate any feedback you might have. > Below are some specific questions of particular interest: I will need to look into that a bit more, need more time for that. > 1) Do you think that improving the quality of information > when deciding upon a practice change would be helpful? For me, yes! Though thorough analysis needs to be done. Sometimes the data might indicate one thing, while practically something else happens. > 2) If yes, would you think a tool be of use, e,g., Pe2 Ideally it should be built into Pe2. How does Pe2 gather its data btw? > 3) If yes to both above, what kind of questions does the tool need > to help you easily answer for you to actually use it? I'd like to see: - how often are bugs closed as incomplete - buggyness of GNOME in general over time - pareto chart of top crashing products - all duplicate crashers should be detected automatically - crashers should not be filed at bugzilla, instead they should be on some separate server which only task is to handle the crashers - separate server should forward to bugzilla - pareto chart of products where no action seems to be taken (indicating need to ask maintainer, or lack of maintainer) - anything that indicates sudden trend break, be it positive or negative e.g.: suddenly there are way more bugs fixed for a product than usual, or opposite > 4) Are there factual errors in the BugBuddy account? There are some problems with bug-buddy: - the retrace server is broken, so the change of version 2.19 is broken - we should let the retracing be done by the distribution There are plans to change the crash handling significantly. See: https://live.gnome.org/Design/Apps/Oops https://live.gnome.org/GnomeOS/Design/Whiteboards/ProblemReporting The designers+maintainers want to provide a well integrated problem reporting infrastructure. GNOME is becoming more and more integrated with the OS. OS problems affect the perception people have of GNOME. E.g. if suspend doesn't work, GNOME will be seen as bad. We (GNOME) should work to ensure such lower level problems can be detected, reported and fixed. > 5) Are the two measures sensible summaries for service and > issue quality? > a) quality of service metric: the time it takes to > resolve an issue (e.g, 90% of issues resolved in X months). This relies on two things: - bugsquad to triage the bugs and - maintainers to fix the bug Would also be nice to see it in a control chart. Though think the data is not stable. E.g. new GNOME means new crashers (I assume). Same for when a new distribution is released. > b) issue quality metric: the fraction of issues fixed Ideally I'd like to see the number of crashers per amount of hours spend in the software. But that is impossible to gather at the moment. See for instance: https://crash-stats.mozilla.com/products/Firefox That has crashers per 100 users, also quite interesting. -- Regards, Olav _______________________________________________ gnome-bugsquad mailing list [email protected] http://mail.gnome.org/mailman/listinfo/gnome-bugsquad
