On Fri, Aug 1, 2014 at 5:44 PM, Qingzhou Luo <[email protected]> wrote: > We recently made some contribution to Maven Surefire to add the new > option -rereunFailingTestsCount.
Sounds very valuable. > the first part of our plugin is to integrate with this new feature from > Maven Surefire. It will parse the report and display flaky tests, output > information of all the re-runs(stacktrace, output, etc) on test result page. Very nice. We could definitely use this on https://jenkins.ci.cloudbees.com/job/core/job/jenkins-core/ for example. Currently we rely on JUnit-based means of suppressing flaky tests: https://github.com/jenkinsci/jenkins/commit/d29a90150c839855d4ff0c084c3cd838b48fab5c https://github.com/jenkinsci/jenkins/commit/7bbf36b7f0de43db908b23278f39ad8cc573186c https://github.com/jenkinsci/jenkins/commit/6fd76ea2e882d7439d3d0fb50df79aee3c073155 which could result in hiding genuine regressions. Of course you could do the rerunning logic in a custom JUnit runner but this would not compose well, and displaying these notes in Jenkins is a good idea. > 2. For each failing build we provide a "Deflake" action. It is different > from rebuilding the project, as it will: > > 1) Checkout the exact revision of that failing build (GIT) > 2) Only run all those failed tests. Pity that this can only work in constrained circumstances: that you are using Git, that you have a freestyle project, and that there are no particular side effects of running the build (I presume with $MAVEN_OPTS or something set to specify which tests to run). > https://github.com/google/jenkins-flaky-test-report > > Will push our code to it very soon. So here I want to get a repo from > Jenkins so we can push our code to Jenkins too. By the way it is discouraged to have two repositories with the same plugin—better to have the @jenkinsci repo host definitive sources. Otherwise there is endless confusion when people want to contribute pull requests and so on. Of course Git lets you reconcile changes eventually but at the cost of delays and conflicts. -- You received this message because you are subscribed to the Google Groups "Jenkins Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
