Hi, I'd like to host our new plug-in.
Plugin name: flaky-test-handler My github account: qingzhouluo/seriousamlqz (Please add both, thanks). Description: This plugin is designed to handle flaky tests. Currently it aims for Git and Maven. 1. We recently made some contribution to Maven Surefire to add the new option -rereunFailingTestsCount. It was already accepted, and may be released with 2.18 in the near future: https://github.com/apache/maven-surefire/commit/fefaae7f0534a59f52c046a64c96987e8561dd48 It basically let users to choose to re-run failing tests up to N times, and if it passes any of those N times, it will be marked as a "Flake" and the build will be marked as successful. The generated test report will contain each re-run information. So the first part of our plugin is to integrate with this new feature from Maven Surefire. It will parse the report and display flaky tests, output information of all the re-runs(stacktrace, output, etc) on test result page. https://drive.google.com/file/d/0Bzfyw2NYCLFBV2lnMVlrWVhueGc https://drive.google.com/file/d/0Bzfyw2NYCLFBZXFxSThlMWlaeVU 2. For each failing build we provide a "Deflake" action. It is different from rebuilding the project, as it will: 1) Checkout the exact revision of that failing build (GIT) 2) Only run all those failed tests. So this will give developer a good idea about how many tests are flaky for this build. https://drive.google.com/file/d/0Bzfyw2NYCLFBR3RDM3psQjhIRUk/ https://drive.google.com/file/d/0Bzfyw2NYCLFBeTFjQk5sQW5XR3M/ 3. Aggregate information from both 1 and 2, and display them to the users. For each git revision and each test, we count how many times it passed, how many times it failed. So if a test passed and failed for the both revision, we count it as a flake; if it always failed, then it's a fail, otherwise it's a pass. https://drive.google.com/file/d/0Bzfyw2NYCLFBRlI2SHpubTltd28/ https://drive.google.com/file/d/0Bzfyw2NYCLFBVWJNWFAweEl1Y28/ So this help users to see what are my most flaky tests (they can compute some kind of flakiness), so they can fix them first. It basically serves as the test health page. We have a repo on github, however it was empty right now. https://github.com/google/jenkins-flaky-test-report Will push our code to it very soon. So here I want to get a repo from Jenkins so we can push our code to Jenkins too. Thank you very much! Regards, Qingzhou -- You received this message because you are subscribed to the Google Groups "Jenkins Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
