On Fri, Sep 6, 2019 at 10:32 AM Vincent Massol <vinc...@massol.net> wrote: > > Hi Simon, > > > On 6 Sep 2019, at 10:27, Simon Urli <simon.u...@xwiki.com> wrote: > > > > Hi all, > > > > On 05/09/2019 17:40, Simon Urli wrote: > >> On 05/09/2019 17:24, Thomas Mortagne wrote: > >>> On Thu, Sep 5, 2019 at 3:43 PM Simon Urli <simon.u...@xwiki.com> wrote: > >>>> > >>>> Hi everyone, > >>>> > >>>> reopening this thread since I started to close some flicker issues as > >>>> part of BFD and got comments for those. > >>>> > >>>> So the last mails on this threads suggested to close the flicker issues > >>>> if we didn't manage to reproduce them locally after a repeated tests, > >>>> and that we didn't see them after a while. > >>>> > >>>> We didn't vote for those suggestion and I assumed a bit quick that I > >>>> could close some flicker issues that I personally don't remember about > >>>> on the CI after having tested them locally. > >>>> My point for doing that is the same as for the first mail I posted on > >>>> this thread: those flickers are old, and the code did change enough for > >>>> those to be fixed in a way or another. > >>> > >>> Being old does not always means the code leading to those failures > >>> changed that much. > >>> > >>>> > >>>> Now I might be completely wrong, and the flicker to happen again, but I > >>>> don't think it's a problem since we can really easily open back the > >>>> issues if it's the case. > >>>> > >>>> The other solution IMO is to indeed keep the issue open and in fact to > >>>> never really close them, because we just don't have time to investigate > >>>> each of them properly. > >>>> > >>>> I really don't see any value of keeping things open and don't act on > >>>> them, that's why I suggest to close them after doing the checks we > >>>> suggested before: > >>>> 1. try to repeat locally the failure; > >>> > >>> This is totally useless IMO unless you make sure that your computer is > >>> made super slow some way since that's the reason for most of the > >>> flickering tests. > >>> > >>>> 2. check that we didn't encounter those flickers since last cycle. > >>> > >>> This one is enough for me but the hard part is to knowing that. > >> Ok, so the proposal is now to check only the age since last time we saw > >> them of the open flickers before closing them. > >>> > >>>> > >>>> So first question, do we all agree on that? > >>>> > >>>> Then for the second check, Vincent suggested to add some tooling: it > >>>> will be best, but it takes time to do. So on the meantime, as Thomas > >>>> also suggested, we could add a check in the release plan to create or > >>>> update all jira issues that concerns flickers. It would allow us to keep > >>>> some information about the liveness of our flickers. > >>>> > >>>> So second question, do you agree on that? > >>> > >>> Depends what it exactly means. Have some dedicated jira field to > >>> indicate when you saw it last ? Comment that you just saw that test > >>> failing again ? > >> My suggestion was about a dedicated JIRA field if possible. > > > > So, ok if I create a new custom field in JIRA for flickers, called "Date of > > last failure for flicker”? > > [snip] > > I don’t see how it’ll help since it’ll never be up to date, and the old value > will remain making us think it’s not been flickering for a long time.
In my mind the idea is not so much to use this field as a criteria to close an issue but as a criteria to not close it. > > Thanks > -Vincent > -- Thomas Mortagne
