Ivan, We already have IgniteReproducingSuite, and there was a corresponding TC configuration, but it was removed (maybe by accident). We could resurrect TC configuration. + We can improve Suite to provide test class name by parameter instead of changing code.
On Mon, Feb 11, 2019 at 11:22 AM Павлухин Иван <[email protected]> wrote: > Vlad, > > Your case is valid as well. And DebugSuite will not help there. But I > talk about another case. And I have already reproduced several tests > after repetitive execution of a single test on TC. So, my case also > seems valid. > > пн, 11 февр. 2019 г. в 10:43, Vladislav Pyatkov <[email protected]>: > > > > Hi, > > > > I think more test falling on TC due to context in which running. > > If you added it in debug suite, it can stop failing anymore like local > run. > > > > On Mon, Feb 11, 2019 at 9:37 AM Павлухин Иван <[email protected]> > wrote: > > > > > Hi, > > > > > > During a couple of last weeks I was fixing several flaky tests. > > > Sometimes it was quite hard to reproduce a test locally. So, one > > > option was running a particular test on TC several times in a row. To > > > setup such run I did code modifications in several places. > > > > > > I thought about how to simplify the thing. And I came up with some > > > sort of solution which I would like to share. Basically it is custom > > > junit runner DebugSuite and a configuration annotation > > > DebugSuite.Config which allows to choose a method to run and number of > > > executions. You can see a draft in PR [1]. > > > > > > As always there are several options to solve a problem. One > > > alternative way is creating something similar to parameterized build > > > job Jenkins employs [2] (I have not checked for TC analog yet) and > > > using maven features to run single test repeatedly (have not checked > > > as well). But all in all we need to answer following questions: > > > 1. Do we need such tool? (Or perhaps we already have something and > > > there is no need to reinvent the wheel.) > > > 2. What is the best way for us to implement the tool? > > > > > > [1] https://github.com/apache/ignite/pull/6076 > > > [2] https://wiki.jenkins.io/display/JENKINS/Parameterized+Build > > > > > > -- > > > Best regards, > > > Ivan Pavlukhin > > > > > > > > > -- > > Vladislav Pyatkov > > > > -- > Best regards, > Ivan Pavlukhin >
