+1
On 11/06/2019 09:34, Thomas Mortagne wrote:
+1
On Mon, Jun 10, 2019 at 11:43 AM Vincent Massol <[email protected]> wrote:
Hi devs,
I was looking for a rerun optin in JUnit5 or Surefire but it seems it doesn’t
exist for Junit5:
* https://github.com/junit-team/junit5/issues/1558
* https://issues.apache.org/jira/browse/SUREFIRE-1584
However, there’s an extension to do that:
* https://github.com/artsok/rerunner-jupiter
Also WIP for JUnit Pioneer:
* https://github.com/junit-pioneer/junit-pioneer/pull/110
So my proposal FTM is to try https://github.com/artsok/rerunner-jupiter and to
start putting a repeat of 3 for JUnit5 tests that we know are flaky.
For JUnit4 tests we could simply use:
*
https://maven.apache.org/surefire/maven-surefire-plugin/examples/rerun-failing-tests.html
Note that this would also allow us to correct flaky tests that don’t set up the
initial conditions (fixture) correctly.
Second note: This is not an endorsement that we should accept flaky tests.
They’re a plague and we need to work on them and fix them. This is just a
momentary workaround to identify how flaky a test is and to try to have a
better solution towards moving to a Continuous Delivery solution.
WDYT?
Thanks
-Vincent
--
Simon Urli
Software Engineer at XWiki SAS
[email protected]
More about us at http://www.xwiki.com
