Re: [openstack-dev] [reno][tripleo] an alternative approach to known issues
On 02/08/2018 07:42 PM, Gabriele Cerami wrote: On 08 Feb, Ben Nemec wrote: So TripleO has a tech debt policy: https://specs.openstack.org/openstack/tripleo-specs/specs/policy/tech-debt-tracking.html (and I'm tagging tripleo on this thread for visibility). I didn't know about this policy. I've been circling around tech debts for more than a month now, and nobody pointed me to it either. Anyway, I find it insufficient. Not specifically the tracking method, but more the guidelines and the example, to understand how to use it correctly. Doing some basic research, I see that in tripleo 31 bugs were marked with tech-debt tag. 15 Were closed, but they were also marked as CRITICAL. This does not match my definition of tech-debt. I would tend to agree. Tech debt is something you can live with for a period of time, and critical bugs are not. The critical tech debt bug open in tripleo at the time I'm writing this is clearly not critical since it's been open for months and nothing has happened with it, nor has it been blocking anyone from deploying or developing TripleO. Of the remaining 16 sometimes it's hard to understand which part is the technical debt, some are really new features requests matching more the feeling "we may have needed to think about this months ago during the design", for some it's just "we don't have a clear idea of what to do" and the rest is "here's a bandaid, we'll think about it later" The policy lacks a definition of what is a technical debt. I understand the issue as it's really difficult to find a unique definition that fits all we want to include. Whatever the definition we want it to be, there are at least three things that I want to see in tech debt bug (or report), and they all try to focus on the "debt" part of the whole "tech debt" concept. - What's the cost of the repayment - What's the cost of the interests - What's the frequency of the interests For me a technical debt is an imperfect implementation that has consequences. Describable and maybe measurable consequences. "I'm using list in this case for simplicity but if we add more items, we may need a more efficient structure, because it will become too slow" The cost of the repayment is the time spent to replace the structure and its methods with something more complex The cost of the interests is the speed lost when the list increases The frequency of the interests is "this list will become very big every three hours" Without these three elements it becomes hard to understand if we want to really repay the debt, and how we prioritize the repayments. Since a tech debt is something that I find really related to the code (Which piece or line of code is the one that has these measurale consequences) I'd really like for the report to be as close as possible to the code. Also sometimes it may just become a design choice based on assumptions. "I know the list is not efficient, but we'll rarely get it big often, and we are sure to clear it out almost immediately" We can maybe discuss further the advantages of the existing bug tracking for the handling of these reports. Absolutely. Policies are not set in stone for all time. They're living documents that can be updated as we find limitations or areas for improvement. Please feel free to propose any updates you think would be helpful to the existing policy. We can hash out the details in Gerrit. I'm not sure I agree. Bugs stay open until they are fixed/won't fixed. Tech debt stays open until it is fixed/won't fixed. We've had bugs open for years for things that are tricky to fix. Arguably those are tech debt too, but in any case I'm not aware of any problems with using the bug tracker to manage them. Remember the "debt" in "technical debt". You're not reporting it correctly if you don't measure the consequences. I don't think the report should really be about the problem or the solution, because then you're really only talking about the full repayment. Of course without any description on the consequences, the tech debt may be equated to a bug, you really have a problem and you want to discuss only its solution. Another difference is that the importance of a bug rarely changes over time, once correctly triaged. With the technical debt instead - A won't fix doesn't mean that the interests are gone. You closed the bug/tech debt and you are not counting the interests anymore. Convenient and deceiving. There is no status currently that could put the bug on hold. Removing it from all the short term consideration, but make it still count for its interests, make it possible to consider and reevaluate at any time. I don't think any bug should be closed as long as we have some interest in fixing it. If it's not high priority then it should be triaged as such, but I wouldn't advocate closing a bug just because we won't have time to get to it this cycle/year/decade. :-) The milestone field might be a good way to indicate that a b
Re: [openstack-dev] [reno][tripleo] an alternative approach to known issues
On 08 Feb, Ben Nemec wrote: > So TripleO has a tech debt policy: > https://specs.openstack.org/openstack/tripleo-specs/specs/policy/tech-debt-tracking.html > (and I'm tagging tripleo on this thread for visibility). I didn't know about this policy. I've been circling around tech debts for more than a month now, and nobody pointed me to it either. Anyway, I find it insufficient. Not specifically the tracking method, but more the guidelines and the example, to understand how to use it correctly. Doing some basic research, I see that in tripleo 31 bugs were marked with tech-debt tag. 15 Were closed, but they were also marked as CRITICAL. This does not match my definition of tech-debt. Of the remaining 16 sometimes it's hard to understand which part is the technical debt, some are really new features requests matching more the feeling "we may have needed to think about this months ago during the design", for some it's just "we don't have a clear idea of what to do" and the rest is "here's a bandaid, we'll think about it later" The policy lacks a definition of what is a technical debt. I understand the issue as it's really difficult to find a unique definition that fits all we want to include. Whatever the definition we want it to be, there are at least three things that I want to see in tech debt bug (or report), and they all try to focus on the "debt" part of the whole "tech debt" concept. - What's the cost of the repayment - What's the cost of the interests - What's the frequency of the interests For me a technical debt is an imperfect implementation that has consequences. Describable and maybe measurable consequences. "I'm using list in this case for simplicity but if we add more items, we may need a more efficient structure, because it will become too slow" The cost of the repayment is the time spent to replace the structure and its methods with something more complex The cost of the interests is the speed lost when the list increases The frequency of the interests is "this list will become very big every three hours" Without these three elements it becomes hard to understand if we want to really repay the debt, and how we prioritize the repayments. Since a tech debt is something that I find really related to the code (Which piece or line of code is the one that has these measurale consequences) I'd really like for the report to be as close as possible to the code. Also sometimes it may just become a design choice based on assumptions. "I know the list is not efficient, but we'll rarely get it big often, and we are sure to clear it out almost immediately" We can maybe discuss further the advantages of the existing bug tracking for the handling of these reports. > I'm not sure I agree. Bugs stay open until they are fixed/won't fixed. Tech > debt stays open until it is fixed/won't fixed. We've had bugs open for > years for things that are tricky to fix. Arguably those are tech debt too, > but in any case I'm not aware of any problems with using the bug tracker to > manage them. Remember the "debt" in "technical debt". You're not reporting it correctly if you don't measure the consequences. I don't think the report should really be about the problem or the solution, because then you're really only talking about the full repayment. Of course without any description on the consequences, the tech debt may be equated to a bug, you really have a problem and you want to discuss only its solution. Another difference is that the importance of a bug rarely changes over time, once correctly triaged. With the technical debt instead - A won't fix doesn't mean that the interests are gone. You closed the bug/tech debt and you are not counting the interests anymore. Convenient and deceiving. There is no status currently that could put the bug on hold. Removing it from all the short term consideration, but make it still count for its interests, make it possible to consider and reevaluate at any time. - A tech debt really can get more and more costly to repay. If someone else implement something over you "imperfect" code, the cost of the repayment just doubled, because you have to fix a stack of code now. Marking the code with a # TD may warn someone "be aware that someone is trying to build over a problem" - The frequency of interests may increase also over time, and the importance may raise as we are paying too much interests, and may be better to start considering full repayment. - One of the solution to a technical debt is "conversion": you just render the imperfect solution just less imperfect, that is you don't fully repay it, you repay just a little to lower the interests cost or frequency. It's not a workaround, it's not a fix, you're just reducing its impact. How do you report that in a bug tracking system ? > I'm kind of split on the idea of templates for Reno. On the one hand I > could see it being useful for complex things, but on the other I wonder if > something co
Re: [openstack-dev] [reno][tripleo] an alternative approach to known issues
So TripleO has a tech debt policy: https://specs.openstack.org/openstack/tripleo-specs/specs/policy/tech-debt-tracking.html (and I'm tagging tripleo on this thread for visibility). It essentially comes down to: open a bug, tag it "tech-debt", and reference it in the code near the tech debt. I kind of like that approach because it makes use of the existing integration between Gerrit and Launchpad, and we don't have to invent a new system for triaging tech debt. It just gets treated as the appropriate level bug. I guess my question then would be whether there is sufficient advantage to inventing a new system in Reno when we already have systems in place that seem suited to this. I have a few specific thoughts below too. On 02/08/2018 04:43 PM, Gabriele Cerami wrote: Hi, sometimes it happens, while reviewing a patch, to find an issue that is not quite a bug, because it doesn't limit functionality, but may represent a problem in some corner case, or with some possible future modification in some component involved in the patch; it may best be described as a weakness in the code, which may happen only under certain circumstances. The author, for some time or complexity constraint is creating a technical debt, or making a micro design choice. How to keep track of the issue ? How, after 6 month when there's time and bandwidth to look at the problem again, can this note be found and issue dealt in the way it deserves ? How to help prioritize then the list of issues left behind during the duration of a release ? Nobody is going to read all the comments on all the merged patches in the past months, to find all the objections. Also technical debts cannot be treated like bugs, because they have a different life span. A bug is opened and closed for good after a while. I'm not sure I agree. Bugs stay open until they are fixed/won't fixed. Tech debt stays open until it is fixed/won't fixed. We've had bugs open for years for things that are tricky to fix. Arguably those are tech debt too, but in any case I'm not aware of any problems with using the bug tracker to manage them. A technical debt may be carried for long time, and it would be perfectly natural to mark it as something to just live with, and pay the interest for, because the time required to solve it it's not worth it. And despite that, it's important to keep track of them because an eventual reevaluation of the interests cost or a change in the surroundings (a new requirement that breaks an assumption) may lead to a different decision after some time. The way technical debts are treated right now officially is by adding a TODO note inside the code, or maybe adding a "issue" field in release notes. I would like to expand this TODO note, and the known issue field, make it become something more structured. I thought about reno, to create a technical debt register/micro design document. A developer would generate a UUID, put on the code a comment # TD: and then add the description in reno. A simple yaml associative array with three or four keys: UUID, description, consequences, options, which may describe either the problem or the micro design choice and assumption without which the code may show these weaknesses. The description would stay with the code, submitted with the same patch with which it was introduced. Then when it's time, a report on all these description could be created to evaluate, prioritize and eventually close the gap that was created, or just mark that as "prefer to just deal with the consequences" One may later incur in a problem a number of times, find the piece of code responsible, and see that the problem is know, and immediately raise its impact to request a reevaluation. Or we may realize that the code that creates a certain amount of weaknesses is going to be deleted, and we can close all the items related to it. The creation and handling of such items could add too much of a burden to the developer, for these reasons, I would prefer to automate some part of the creation, for example the UUID generation, date expansion, status change on the item. I used this, to try out how this automation could work https://review.openstack.org/538233 which could add basic logic to the templates, to automate some of the tasks. This idea certainly requires refinement (for example what happens when the weakness is discovered at a later time), but I would like to understand if it's possible to use reno for this approach. Any feedback would be highly appreciated. I'm kind of split on the idea of templates for Reno. On the one hand I could see it being useful for complex things, but on the other I wonder if something complex enough to require a template actually belongs in release notes or if it should go in formal documentation. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openst
