I'm fine with that, but in that case we should have a priority for release blockers, below which bugs get automatically bumped to the next release (and which becomes the burndown list).
On Fri, Oct 25, 2019 at 1:58 PM Kenneth Knowles <k...@apache.org> wrote: > > My takeaway from this thread is that priorities should have a shared > community intuition and/or policy around how they are treated, which could > eventually be formalized into SLOs. > > At a practical level, I do think that build breaks are higher priority than > release blockers. If you are on this thread but not looking at the PR, here > is the verbiage I added about urgency: > > P0/Blocker: "A P0 issue is more urgent than simply blocking the next release" > P1/Critical: "Most critical bugs should block release" > P2/Major: "No special urgency is associated" > ... > > Kenn > > On Fri, Oct 25, 2019 at 11:46 AM Robert Bradshaw <rober...@google.com> wrote: >> >> We cut a release every 6 weeks, according to schedule, making it easy >> to plan for, and the release manager typically sends out a warning >> email to remind everyone. I don't think it makes sense to do that for >> every ticket. Blockers should be reserved for things we really >> shouldn't release without. >> >> On Fri, Oct 25, 2019 at 11:33 AM Pablo Estrada <pabl...@google.com> wrote: >> > >> > I mentioned on the PR that I had been using the 'blocker' priority along >> > with the 'fix version' field to mark issues that I want to get in the >> > release. >> > Of course, this little practice of mine only matters much around release >> > branch cutting time - and has been useful for me to track which things I >> > want to ensure getting into the release / bump to the next /etc. >> > I've also found it to be useful as a way to communicate with the release >> > manager without having to sync directly. >> > >> > What would be a reasonable way to tell the release manager "I'd like to >> > get this feature in. please talk to me if you're about to cut the branch" >> > - that also uses the priorities appropriately? - and that allows the >> > release manager to know when a fix version is "more optional" / "less >> > optional"? >> > >> > On Wed, Oct 23, 2019 at 12:20 PM Kenneth Knowles <k...@apache.org> wrote: >> >> >> >> I finally got around to writing some of this up. It is minimal. Feedback >> >> is welcome, especially if what I have written does not accurately >> >> represent the community's approach. >> >> >> >> https://github.com/apache/beam/pull/9862 >> >> >> >> Kenn >> >> >> >> On Mon, Feb 11, 2019 at 3:21 PM Daniel Oliveira <danolive...@google.com> >> >> wrote: >> >>> >> >>> Ah, sorry, I missed that Alex was just quoting from our Jira >> >>> installation (didn't read his email closely enough). Also I wasn't aware >> >>> about those pages on our website. >> >>> >> >>> Seeing as we do have definitions for our priorities, I guess my main >> >>> request would be that they be made more discoverable somehow. I don't >> >>> think the tooltips are reliable, and the pages on the website are >> >>> informative, but hard to find. Since it feels a bit lazy to say "this >> >>> isn't discoverable enough" without suggesting any improvements, I'd like >> >>> to propose these two changes: >> >>> >> >>> 1. We should write a Beam Jira Guide with basic information about our >> >>> Jira. I think the bug priorities should go in here, but also anything >> >>> else we would want someone to know before filing any Jira issues, like >> >>> how our components are organized or what the different issue types mean. >> >>> This guide could either be written in the website or the wiki, but I >> >>> think it should definitely be linked in >> >>> https://beam.apache.org/contribute/ so that newcomers read it before >> >>> getting their Jira account approved. The goal here being to have a >> >>> reference for the basics of our Jira since at the moment it doesn't seem >> >>> like we have anything for this. >> >>> >> >>> 2. The existing info on Post-commit and pre-commit policies doesn't seem >> >>> very discoverable to someone monitoring the Pre/Post-commits. I've >> >>> reported a handful of test-failures already and haven't seen this link >> >>> mentioned much. We should try to find a way to funnel people towards >> >>> this link when there's an issue, the same way we try to funnel people >> >>> towards the contribution guide when they write a PR. As a note, while >> >>> writing this email I remembered this link that someone gave me before >> >>> (https://s.apache.org/beam-test-failure). That mentions the Post-commit >> >>> policies page, so maybe it's just a matter of pasting that all over our >> >>> Jenkins builds whenever we have a failing test? >> >>> >> >>> PS: I'm also definitely for SLOs, but I figure it's probably better >> >>> discussed in a separate thread so I'm trying to stick to the subject of >> >>> priority definitions. >> >>> >> >>> On Mon, Feb 11, 2019 at 9:17 AM Scott Wegner <sc...@apache.org> wrote: >> >>>> >> >>>> Thanks for driving this discussion. I also was not aware of these >> >>>> existing definitions. Once we agree on the terms, let's add them to our >> >>>> Contributor Guide and start using them. >> >>>> >> >>>> +1 in general; I like both Alex and Kenn's definitions; Additional >> >>>> wordsmithing could be moved to a Pull Request. Can we make the >> >>>> definitions useful for both the person filing a bug, and the assignee, >> >>>> i.e. >> >>>> >> >>>> <Priority Level>: <Criteria for what types of issues should be >> >>>> assigned>. <Expectations for responding to a Priority Level issue> >> >>>> >> >>>> On Sun, Feb 10, 2019 at 7:49 PM Kenneth Knowles <k...@google.com> wrote: >> >>>>> >> >>>>> The content that Alex posted* is the definition from our Jira >> >>>>> installation anyhow. >> >>>>> >> >>>>> I just searched around, and there's >> >>>>> https://community.atlassian.com/t5/Jira-questions/According-to-Jira-What-is-Blocker-Critical-Major-Minor-and/qaq-p/668774 >> >>>>> which makes clear that this is really user-defined, since Jira has >> >>>>> many deployments with their own configs. >> >>>>> >> >>>>> I guess what I want to know about this thread is what action is being >> >>>>> proposed? >> >>>>> >> >>>>> Previously, there was a thread that resulted in >> >>>>> https://beam.apache.org/contribute/precommit-policies/ and >> >>>>> https://beam.apache.org/contribute/postcommits-policies/. These have >> >>>>> test failures and flakes as Critical. I agree with Alex that these >> >>>>> should be Blocker. They disrupt the work of the entire community, so >> >>>>> we need to drop everything and get green again. >> >>>>> >> >>>>> Other than that, I think what you - Daniel - are suggesting is that >> >>>>> the definition might be best expressed as SLOs. I asked on >> >>>>> u...@infra.apache.org about how we could have those and the answer is >> >>>>> the homebrew >> >>>>> https://svn.apache.org/repos/infra/infrastructure/trunk/projects/status/sla/jira/. >> >>>>> If anyone has time to dig into that and see if it can work for us, >> >>>>> that would be cool. >> >>>>> >> >>>>> Kenn >> >>>>> >> >>>>> *Blocker: Blocks development and/or testing work, production could not >> >>>>> run >> >>>>> Critical: Crashes, loss of data, severe memory leak. >> >>>>> Major (Default): Major loss of function. >> >>>>> Minor: Minor loss of function, or other problem where easy workaround >> >>>>> is present. >> >>>>> Trivial: Trivial Cosmetic problem like misspelt words or misaligned >> >>>>> text. >> >>>>> >> >>>>> >> >>>>> On Sun, Feb 10, 2019 at 7:20 PM Daniel Oliveira >> >>>>> <danolive...@google.com> wrote: >> >>>>>> >> >>>>>> Are there existing meanings for the priorities in Jira already? I >> >>>>>> wasn't able to find any info on either the Beam website or wiki about >> >>>>>> it, so I've just been prioritizing issues based on gut feeling. If >> >>>>>> not, I think having some well-defined priorities would be nice, at >> >>>>>> least for our test-failures, and especially if we wanna have some >> >>>>>> SLOs like I've seen being thrown about. >> >>>>>> >> >>>>>> On Fri, Feb 8, 2019 at 3:06 PM Kenneth Knowles <k...@apache.org> >> >>>>>> wrote: >> >>>>>>> >> >>>>>>> I've been thinking about this since working on the release. If I >> >>>>>>> ignore the names I think: >> >>>>>>> >> >>>>>>> P0: get paged, stop whatever you planned on doing, work late to fix >> >>>>>>> P1: continually update everyone on status and shouldn't sit around >> >>>>>>> unassigned >> >>>>>>> P2: most things here; they can be planned or picked up by whomever >> >>>>>>> P3: nice-to-have things, maybe starter tasks or lesser cleanup, but >> >>>>>>> no driving need >> >>>>>>> Sometimes there's P4 but I don't value it. Often P3 is a >> >>>>>>> deprioritized thing from P2, so more involved and complex, while P4 >> >>>>>>> is something easy and not important filed just as a reminder. Either >> >>>>>>> way, they are both not on the main path of work. >> >>>>>>> >> >>>>>>> I looked into it and the Jira priority scheme determines the set of >> >>>>>>> priorities as well as the default. Ours is shared by 635 projects. >> >>>>>>> Probably worth keeping. The default priority is Major which would >> >>>>>>> correspond with P2. We can expect the default to be where most >> >>>>>>> issues end up. >> >>>>>>> >> >>>>>>> P0 == Blocker: get paged, stop whatever you planned on doing, work >> >>>>>>> late to fix >> >>>>>>> P1 == Critical: continually update everyone on status and shouldn't >> >>>>>>> sit around unassigned >> >>>>>>> P0 == Major (default): most things here; they can be planned or >> >>>>>>> picked up by whomever >> >>>>>>> P3 == Minor: nice-to-have things, maybe starter tasks or lesser >> >>>>>>> cleanup, but no driving need >> >>>>>>> Trivial: Maybe this is attractive to newcomers as it makes it sound >> >>>>>>> easy. >> >>>>>>> >> >>>>>>> Kenn >> >>>>>>> >> >>>>>>> On Thu, Feb 7, 2019 at 4:08 PM Alex Amato <ajam...@google.com> wrote: >> >>>>>>>> >> >>>>>>>> Hello Beam community, I was thinking about this and found some >> >>>>>>>> information to share/discuss. Would it be possible to confirm my >> >>>>>>>> thinking on this: >> >>>>>>>> >> >>>>>>>> There are 5 priorities in the JIRA system today (tooltip link): >> >>>>>>>> >> >>>>>>>> Blocker Blocks development and/or testing work, production could >> >>>>>>>> not run >> >>>>>>>> Critical Crashes, loss of data, severe memory leak. >> >>>>>>>> Major Major loss of function. >> >>>>>>>> Minor Minor loss of function, or other problem where easy >> >>>>>>>> workaround is present. >> >>>>>>>> Trivial Cosmetic problem like misspelt words or misaligned text. >> >>>>>>>> >> >>>>>>>> How should JIRA issues be prioritized for pre/post commit test >> >>>>>>>> failures? >> >>>>>>>> >> >>>>>>>> I think Blocker >> >>>>>>>> >> >>>>>>>> What about the flakey failures? >> >>>>>>>> >> >>>>>>>> Blocker as well? >> >>>>>>>> >> >>>>>>>> How should non test issues be prioritized? (E.g. feature to >> >>>>>>>> implement or bugs not regularly breaking tests). >> >>>>>>>> >> >>>>>>>> I suggest Minor, but its not clear how to distinguish between these. >> >>>>>>>> >> >>>>>>>> Below is my thinking: But I wanted to know what the Apache/Beam >> >>>>>>>> community generally thinks about these priorities. >> >>>>>>>> >> >>>>>>>> Blocker: Expect to be paged. Production systems are down. >> >>>>>>>> Critical: Expect to be contacted by email or a bot to fix this. >> >>>>>>>> Major: Some loss of function in the repository, can issues that >> >>>>>>>> need to be addressed soon are here. >> >>>>>>>> Minor: Most issues will be here, important issues within this will >> >>>>>>>> get picked up and completed. FRs, bugs. >> >>>>>>>> Trivial: Unlikely to be implemented, far too many issues in this >> >>>>>>>> category. FRs, bugs. >> >>>>>>>> >> >>>>>>>> Thanks for helping to clear this up >> >>>>>>>> Alex >> >>>> >> >>>> >> >>>> >> >>>> -- >> >>>> >> >>>> >> >>>> >> >>>> >> >>>> Got feedback? tinyurl.com/swegner-feedback
