I think majority of the flaky test tags were put in one shot in one commit. So the timer will expire on all tests in one shot. Also we have stopped marking things flaky, if something fails in CI, we immediately try to fix it. If there is a flakiness element in the test, the test is immediately modified. And slowly we are also cleaning up the existing flaky tests.
Regards Naba On Fri, Nov 3, 2017 at 10:33 AM Patrick Rhomberg <prhomb...@pivotal.io> wrote: > Hello, all! > > I was considering doing some git archeology centered around identifying > how long a any given test class containing a @Flaky has had that > annotation. Ultimately, I think it would be good to add a test that would > fail when any one test has been flaky for too long. I feel like many of > our flaky tests have fallen by the wayside, and this could provide the > impetus to resolve these issues in a timely fashion. > This leads naturally to the question: How long should a test be allowed > to remain marked Flaky? Certainly, flaky tests are most often of the > non-deterministic, hard-to-reproduce variety, so some leeway is deserved. > Two weeks? One month? > Thoughts? > > Imagination is Change. > ~Patrick Rhomberg >
