Hello Quingzhou, I have just created https://github.com/jenkinsci/flaky-test-handler for you and granted commit access.
Welcome aboard! Vincent 2014-08-05 1:25 GMT+02:00 Qingzhou Luo <[email protected]>: > Thanks for supporting, Bruno. Still waiting for approval.. > > > On Saturday, August 2, 2014 12:02:04 PM UTC-7, Bruno Meneguello wrote: > >> Amazing idea! In my company we have developed a RunConfiguration to do >> exactly that and implemented a plugin to do the same thing (but haven't >> opened it because it become a very specific implementation) >> >> Thumbs up for the initiative! >> Em 01/08/2014 21:44, "Qingzhou Luo" <[email protected]> escreveu: >> >>> Hi, >>> >>> I'd like to host our new plug-in. >>> >>> Plugin name: flaky-test-handler >>> My github account: qingzhouluo/seriousamlqz (Please add both, thanks). >>> >>> Description: >>> >>> This plugin is designed to handle flaky tests. Currently it aims for Git >>> and Maven. >>> >>> 1. We recently made some contribution to Maven Surefire to add the new >>> option -rereunFailingTestsCount. It was already accepted, and may be >>> released with 2.18 in the near future: >>> >>> https://github.com/apache/maven-surefire/commit/ >>> fefaae7f0534a59f52c046a64c96987e8561dd48 >>> >>> It basically let users to choose to re-run failing tests up to N times, >>> and if it passes any of those N times, it will be marked as a "Flake" and >>> the build will be marked as successful. The generated test report will >>> contain each re-run information. >>> >>> So the first part of our plugin is to integrate with this new feature >>> from Maven Surefire. It will parse the report and display flaky tests, >>> output information of all the re-runs(stacktrace, output, etc) on test >>> result page. >>> >>> https://drive.google.com/file/d/0Bzfyw2NYCLFBV2lnMVlrWVhueGc >>> https://drive.google.com/file/d/0Bzfyw2NYCLFBZXFxSThlMWlaeVU >>> >>> 2. For each failing build we provide a "Deflake" action. It is different >>> from rebuilding the project, as it will: >>> >>> 1) Checkout the exact revision of that failing build (GIT) >>> 2) Only run all those failed tests. >>> >>> So this will give developer a good idea about how many tests are flaky >>> for this build. >>> >>> https://drive.google.com/file/d/0Bzfyw2NYCLFBR3RDM3psQjhIRUk/ >>> https://drive.google.com/file/d/0Bzfyw2NYCLFBeTFjQk5sQW5XR3M/ >>> >>> 3. Aggregate information from both 1 and 2, and display them to the >>> users. >>> >>> For each git revision and each test, we count how many times it passed, >>> how many times it failed. So if a test passed and failed for the both >>> revision, we count it as a flake; if it always failed, then it's a fail, >>> otherwise it's a pass. >>> >>> https://drive.google.com/file/d/0Bzfyw2NYCLFBRlI2SHpubTltd28/ >>> https://drive.google.com/file/d/0Bzfyw2NYCLFBVWJNWFAweEl1Y28/ >>> >>> So this help users to see what are my most flaky tests (they can compute >>> some kind of flakiness), so they can fix them first. It basically serves as >>> the test health page. >>> >>> We have a repo on github, however it was empty right now. >>> https://github.com/google/jenkins-flaky-test-report >>> >>> Will push our code to it very soon. So here I want to get a repo from >>> Jenkins so we can push our code to Jenkins too. >>> >>> Thank you very much! >>> >>> Regards, >>> >>> Qingzhou >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Jenkins Developers" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> >>> For more options, visit https://groups.google.com/d/optout. >>> >> -- > You received this message because you are subscribed to the Google Groups > "Jenkins Developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Jenkins Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
