Hi Kevin, > Is there a way to retry a flakey test?
In general I agree with Martin Todorov that tests should be small and atomic, and flakiness is a sign of larger problems. However, I also agree with you that sometimes flaky tests are a reality: my group has run into this with behavior of the JVM garbage collector differing on Windows, for example, in a rather nondeterministic way. In that case, retrying tests ~5-10 times was a good solution for us. But rather than configuring Surefire or Failsafe to do this, I'd suggest (if possible) to write it into the test itself. Put the test in a for loop, and then assert at the end that numSuccesses > 0 or whatever. You then also issue a warning if numSuccesses != numAttempts, as you requested. In short: it gives you the flexibility you need. Regards, Curtis On Fri, Sep 19, 2014 at 11:24 AM, Kevin Burton <[email protected]> wrote: > Is there a way to retry a flakey test? > > I’d basically like to have a flag that retries a failing test 2 or 3 times. > > Flakey and non deterministic tests are a fact of life in more complicated > systems. > > The main problem being that they are impossible to setup again because > they’re usually race conditions. > > If I could just retry the test a second time , that would solve this issue. > > Would be nice to have the build warned that the test is failing though.. > > -- > > Founder/CEO Spinn3r.com > Location: *San Francisco, CA* > blog: http://burtonator.wordpress.com > … or check out my Google+ profile > <https://plus.google.com/102718274791889610666/posts> > <http://spinn3r.com> >
