Thanks for supporting, Bruno. Still waiting for approval.. On Saturday, August 2, 2014 12:02:04 PM UTC-7, Bruno Meneguello wrote: > > Amazing idea! In my company we have developed a RunConfiguration to do > exactly that and implemented a plugin to do the same thing (but haven't > opened it because it become a very specific implementation) > > Thumbs up for the initiative! > Em 01/08/2014 21:44, "Qingzhou Luo" <[email protected] <javascript:>> > escreveu: > >> Hi, >> >> I'd like to host our new plug-in. >> >> Plugin name: flaky-test-handler >> My github account: qingzhouluo/seriousamlqz (Please add both, thanks). >> >> Description: >> >> This plugin is designed to handle flaky tests. Currently it aims for Git >> and Maven. >> >> 1. We recently made some contribution to Maven Surefire to add the new >> option -rereunFailingTestsCount. It was already accepted, and may be >> released with 2.18 in the near future: >> >> >> https://github.com/apache/maven-surefire/commit/fefaae7f0534a59f52c046a64c96987e8561dd48 >> >> It basically let users to choose to re-run failing tests up to N times, >> and if it passes any of those N times, it will be marked as a "Flake" and >> the build will be marked as successful. The generated test report will >> contain each re-run information. >> >> So the first part of our plugin is to integrate with this new feature >> from Maven Surefire. It will parse the report and display flaky tests, >> output information of all the re-runs(stacktrace, output, etc) on test >> result page. >> >> https://drive.google.com/file/d/0Bzfyw2NYCLFBV2lnMVlrWVhueGc >> https://drive.google.com/file/d/0Bzfyw2NYCLFBZXFxSThlMWlaeVU >> >> 2. For each failing build we provide a "Deflake" action. It is different >> from rebuilding the project, as it will: >> >> 1) Checkout the exact revision of that failing build (GIT) >> 2) Only run all those failed tests. >> >> So this will give developer a good idea about how many tests are flaky >> for this build. >> >> https://drive.google.com/file/d/0Bzfyw2NYCLFBR3RDM3psQjhIRUk/ >> https://drive.google.com/file/d/0Bzfyw2NYCLFBeTFjQk5sQW5XR3M/ >> >> 3. Aggregate information from both 1 and 2, and display them to the users. >> >> For each git revision and each test, we count how many times it passed, >> how many times it failed. So if a test passed and failed for the both >> revision, we count it as a flake; if it always failed, then it's a fail, >> otherwise it's a pass. >> >> https://drive.google.com/file/d/0Bzfyw2NYCLFBRlI2SHpubTltd28/ >> https://drive.google.com/file/d/0Bzfyw2NYCLFBVWJNWFAweEl1Y28/ >> >> So this help users to see what are my most flaky tests (they can compute >> some kind of flakiness), so they can fix them first. It basically serves as >> the test health page. >> >> We have a repo on github, however it was empty right now. >> https://github.com/google/jenkins-flaky-test-report >> >> Will push our code to it very soon. So here I want to get a repo from >> Jenkins so we can push our code to Jenkins too. >> >> Thank you very much! >> >> Regards, >> >> Qingzhou >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Jenkins Developers" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> For more options, visit https://groups.google.com/d/optout. >> >
-- You received this message because you are subscribed to the Google Groups "Jenkins Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
