FYI, the searches I've done for unresolved issues excludes all issues that have been identified as duplicates and those having any form of resolution, including being closed. It does not distinguish between unresolved extension requests and defect reports, and between unresolved confirmed and unconfirmed incident reports.
Agreed, finer-grained analysis can provide some clarity on this, and it will be interesting how much noise happens to be removed. Of course, technical debt can be more than known and uncorrected bugs in the code. It has more forms and symptoms, some of which are reflected in bug reports of a different nature. For those playing along at home, what I have relied on in my use of the *metaphorical* term, "technical debt": Martin Fowler. Technical Debt Quadrant. Web site article, 2009-10-14. Available at <http://martinfowler.com/bliki/TechnicalDebtQuadrant.html> and the earlier paper linked there. I use the open bugs as an indicator of technical debt. Whether it is a very good one or not is a different matter. It strikes me that digging deeper is the avenue to clarity on that. What we have here is evidence that there is something being accrued and that needs to be dug into. We can worry about what best to call it when we know more about what it is. My concern is that it is a debt that our users and the supporters of the user-facing lists are paying the interest on [;<). - Dennis > -----Original Message----- > From: Rob Weir [mailto:r...@robweir.com] > Sent: Sunday, January 17, 2016 06:29 > To: dev@openoffice.apache.org; <orc...@apache.org> <orc...@apache.org> > Subject: Re: [REPORT] Issue Clearance Quality + Technical Debt > > On Fri, Jan 15, 2016 at 2:22 PM, Dennis E. Hamilton <orc...@apache.org> > wrote: > > [BCC to Project Management Committee and users@ oo.a.o] > > > > SUMMARY > > > > The top-level analysis of Bugzilla issue handling has been completed > for all issues opened on the project through December 31, 2015. > > > > The complete tabulation is in the PDF document at > <http://s.apache.org/YFT>. > > > > It remains the case that since the establishment of Apache OpenOffice > as an ASF Top Level Project in November, 2012, the accrual of unresolved > issues exceeds 40%. That is, for every 100 new issues, on the average > more than 40 of them will be unresolved indefinitely. > > > > In contrast, although there is a very large number of unresolved > issues that remain in the Bugzilla from its history as part of > OpenOffice.org, that previous technical debt was, proportionally, under > 20%. > > > > It is entirely unfounded to equate unresolved "issues" in Bugzilla > with "technical debt." When I was "leading" a group of QA volunteers a > few years ago to resolve BZ issues it was clear that only a small > percentage of them were actual reports of new bugs in the product. A > very small fraction. Most of them fit into other categories: > > 1) Questions on how to use to product, or misunderstandings about how > the product worked. > > 2) General system-wide issues and user was reaching out to us for free > computer technical support. > > 3) Redundant reports of already known bugs or already requested > enhancements, > > 4) Vague reports which could not be reproduced and which the original > submitter was unresponsive to requests for clarification. > > At that time we reduced these numbers by quite a bit. But note that > this was done without changing a single line of code. It was a paper > exercise of reducing the of open BZ issues only. If closing BZ > issues in these categories does not magically improve product quality, > hopefully you can see how not closing such issues (due to less > organization in the QA activity) does not magically decrease product > quality, or technical debt for that matter. > > Anyone familiar with the discipline of SQE understands that the > meaningful comparison can only be done after triaging issues, > eliminating those that are not reporting bugs, deduplicating bug > reports and attempting to confirm them. That might indicate > "technical debt". But what you're reporting is not that, but "bug > triaging debt" if you feel the need to give it a label. This > distinction is important. > > Regards, > > -Rob > > > > > Some highlights: > > > > * Even though the monthly rate of new issues and comments on issues > has been decreasing significantly since mid-2014, the rate of technical > debt as the proportion of unresolved issues has not improved. > > > > * Although a reduction to 35% unresolved-issue is seen in the last 5 > months of 2015, this may be distorted by issues created and then > resolved in the staging of AOO 4.1.2 release candidates and QA on the > candidates. Results for the first-quarter of 2016 are needed to > determine if this is a new trend or a hiccup. > > > > DETAIL AND QUALITY MATTERS > > > > This is a rough analysis, although the consistent trend is difficult > to explain away. > > > > Refinement requires a closer look at the nature of issues and > understanding of exactly what resolution means, not just what being left > unresolved means. > > > > There is also a suspected disconnect with regard to what is considered > an issue and how the ways of closing an issue are actually applied. > > > > * Closing of a new issue as a duplicate qualifies as a resolution. > The incidence of long-standing issues that continue to receive duplicate > reports is useful to understand in this case, and that requires more > detail. > > > > * Some issues are closed as Resolved Fixed when the fact of the > matter is that there was insufficient detail to understand and confirm > the issue and the reporting party failed to provide additional > information (if it was even requested). > > > > * Some comments on issues tailgate possibly-different problems onto > known ones, although the resemblance may be superficial and the issues > need to be split. > > > > * Enhancement/feature requests are not distinguished. > > > > * Resolved issues are sometimes closed without obtaining confirmation > that incorporation of the identified resolution in distributed code > actually addresses the originally-reported difficulty. > > > > * Some issue reports are closed as not issues because they are > declared user problems and kicked to the Community Forum. > > > > These may all have small effects. We will know only by looking more > closely into Bugzilla details. > > > > The last case deserves more careful attention. > > > > The next-in-line users of Apache OpenOffice distributions consist of > around 50 million users who are mainly individuals and 87% of whom are > using Microsoft Windows. Such casual users, whatever their limited > experience in trouble-shooting and describing problems, are the main > users of this software. The usability issues they encounter are > important to the project; even though they may not involve bugs in the > code, they point to defects in the product. Capturing those experiences > and recognizing them as real issues for the user community is important. > That feedback is important for determining and making available > workarounds and advice. It can also inform changes to the software that > might ameliorate those difficulties for non-experts. Here's a simple > example: having an easy-to-use option for resetting the user profile. > > > > FUTURE STEPS > > > > Along with extending the current analysis into the first quarter of > 2016, exploration of the nature of unresolved and resolved issues will > be introduced. > > > > > > - Dennis > > > > > > > > > >> -----Original Message----- > >> From: Dennis E. Hamilton [mailto:orc...@apache.org] > >> Sent: Wednesday, September 9, 2015 11:33 > >> To: dev@openoffice.apache.org > >> Subject: FW: [DISCUSS] SUSTAINABILITY: Issue Clearance Quality + > >> Technical Debt > >> > >> > >> From the Chair, > >> > >> From "A Maturity Model for Apache Projects," > >> <http://community.apache.org/apache-way/apache-project-maturity- > >> model.html> > >> > >> "QU50: The project strives to respond to documented bug reports > >> in a timely manner." > >> > >> It is also a recommended practice to acknowledge receipt of reports > >> within some minimum time period, sometimes 24 hours, certainly not > more > >> than 72 hours. > >> > >> This note discusses crude estimation of the technical debt that the > >> project is accruing in terms of unresolved issues in the Bugzilla > >> system. It is important to appreciate the statistics as warning > >> indicators. It is necessary to look deeper for better understanding. > >> > >> The risk of not attending to these indications is exposure of users > to a > >> growing set of known defects and related issues that remain in the > >> software indefinitely. > >> > >> DISCUSSION > >> > >> Besides the questions posed by growing technical debt, below, perhaps > >> the most pressing topic for resolution this year is Improved > >> Coordination of Issues, near the end of this lengthy note. This > >> requires a community approach and a group of contributors able to > work > >> with users in providing first-line support in identification of > issues, > >> recording issues, and directing users directly to known solutions or > >> work-arounds. The volunteer users who accomplish that will require > all > >> of our support. They are also our best source of on-the-ground > >> experience with regard to what the major difficulties are. > >> > >> All questions, proposals, corrections, and suggestions are welcome. > >> > >> TECHNICAL DEBT > >> > >> Using crude statistics from an earlier report, every indication is > that > >> the rate of unresolved issues is continuing to accrue as technical > debt > >> with over 40% of arriving issues unresolved. In this case, the debt > is > >> the accumulation of unresolved defects and the cost of a process that > >> reduces the increase while also paying down past debt. > >> > >> As of 2015-08-05 > >> * the oldest unresolved issue is #497 created > >> 2001-03-02 > >> * 24115/121298 (20%) issues unresolved from before > >> November, 2012 > >> * 2232/5139 (43%) issues unresolved of > >> new issues from November, 2012 (after TLP) > >> through July, 2015 > >> * 192/452 (42%) issues unresolved > >> of those created in the first > >> 7 months of 2015 > >> > >> There are a number of factors that impact the quality of these > >> statistics, including > >> > >> * Unresolved issues are not differentiated by type. > >> Feature and enhancement requests are counted among > >> confirmed defects. > >> > >> * The resolved portion includes issues rejected as > >> duplicates, as lacking supporting information, > >> and as usability/support matters rejected as not > >> being bugs, even when obviously about defects. > >> > >> There is every reason to fear that, after such adjustments, the rate > of > >> technical debt accrual is even greater than 42-43%. > >> > >> CONTEXT > >> > >> 2012-11: #121299 First new issue in the Bugzilla of the AOO Top > Level > >> Project. > >> 2015-07: #126439 Last new issue in the Bugzilla at the end of July, > >> 2015. > >> > >> By years at the TLP, (2012 and 2015 partial) > >> > >> 2012 2013 2014 2015 > >> > >> 929 2136 1739 441 BZ items/month > >> 133 198 170 65 New issues/month > >> (averages are rounded to whole numbers) > >> > >> The rate of arrival of new issues is steadily declining from the 2013 > >> peak. That is not necessarily good news. It does not demonstrate > >> better proportional clearing of the smaller number. > >> > >> PROPOSED MITIGATION > >> > >> The project is currently in a crunch to release Apache OpenOffice > 4.1.2. > >> There will be a bump in Bugzilla activity after which there can be a > >> fresh look at this situation and the developers now heads-down on > 4.1.2 > >> can look to the horizon beyond. > >> > >> MONITORING. These data will be updated monthly, using the July 2015 > >> state as a baseline with respect to changes in the technical debt and > >> rate of increase. A new baseline from December 2015 can be used for > >> 2016, continuing on an annual basis thereafter. > >> > >> REVIEW. Some drill-down is required to differentiate between > unresolved > >> feature and enhancement requests in contrast to reports that relate > to > >> defects of some kind. It is also important to understand the > incidence > >> of duplicates for still-unresolved issues and issues that are marked > >> resolved without any action having taken place. > >> > >> IMPROVED COORDINATION OF ISSUES. Issues involve far more than > >> identified bugs. Reported issues are valuable information about > >> experiences of users and others and where their difficulties arise. > >> There is no means to determine how many others experience an issue > and > >> do not report it. > >> It is important that usability issues of many kinds be captured, > >> tracked and mitigated wherever possible. Appropriate categories > should > >> be added to the Bugzilla and issues of such kinds should be embraced > as > >> the gifts that they are. Simple additions to the current Bugzilla > >> classifications will be investigated. > >> In addition, many issues are recognized on the mailing lists and > >> forums. It is important that there be cross-talk among the different > >> volunteer supporters and that there be no direction of users to other > >> places instead of providing some positive assistance. This involves > >> more engagement and communication among those who work with the > >> Bugzilla, the mailing lists, and the forums to ensure that all > avenues > >> are covered. It is desirable for users to be supported with positive > >> assistance no matter how they bring their situation to the attention > of > >> the project. Accomplishing that is open for discussion. "It takes a > >> village." > >> > >> > >> -----Original Message----- > >> From: Dennis E. Hamilton [mailto:orc...@apache.org] > >> Sent: Wednesday, August 5, 2015 19:57 > >> To: dev@openoffice.apache.org > >> Subject: STATE OF AOO: Overall Bugzilla Activity through July 2015 > >> > >> In looking for visible indicators of project activity, I created an > >> overview of Bugzilla activity from November 2012 through July 2015. > >> > >> This is a high-level view of gross activity and does not provide fine > >> details. There is still an interesting picture. > >> > >> My complete tabulation is available in a PDF document at > >> <http://svn.apache.org/repos/asf/openoffice/pmc/project-state/2015- > 07- > >> BZ-OverallActivity-2015-08-05-dh.pdf>. > >> > >> [ ... ] > >> > >> > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: dev-unsubscr...@openoffice.apache.org > >> For additional commands, e-mail: dev-h...@openoffice.apache.org > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: dev-unsubscr...@openoffice.apache.org > > For additional commands, e-mail: dev-h...@openoffice.apache.org > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@openoffice.apache.org > For additional commands, e-mail: dev-h...@openoffice.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@openoffice.apache.org For additional commands, e-mail: dev-h...@openoffice.apache.org
