Hello Quingzhou, I have just created https://github.com/jenkinsci/flaky-test-handler for you and granted commit access.
Welcome aboard! Vincent 2014-08-05 1:25 GMT+02:00 Qingzhou Luo <qingzhou...@google.com>: > Thanks for supporting, Bruno. Still waiting for approval.. > > > On Saturday, August 2, 2014 12:02:04 PM UTC-7, Bruno Meneguello wrote: > >> Amazing idea! In my company we have developed a RunConfiguration to do >> exactly that and implemented a plugin to do the same thing (but haven't >> opened it because it become a very specific implementation) >> >> Thumbs up for the initiative! >> Em 01/08/2014 21:44, "Qingzhou Luo" <qingz...@google.com> escreveu: >> >>> Hi, >>> >>> I'd like to host our new plug-in. >>> >>> Plugin name: flaky-test-handler >>> My github account: qingzhouluo/seriousamlqz (Please add both, thanks). >>> >>> Description: >>> >>> This plugin is designed to handle flaky tests. Currently it aims for Git >>> and Maven. >>> >>> 1. We recently made some contribution to Maven Surefire to add the new >>> option -rereunFailingTestsCount. It was already accepted, and may be >>> released with 2.18 in the near future: >>> >>> https://github.com/apache/maven-surefire/commit/ >>> fefaae7f0534a59f52c046a64c96987e8561dd48 >>> >>> It basically let users to choose to re-run failing tests up to N times, >>> and if it passes any of those N times, it will be marked as a "Flake" and >>> the build will be marked as successful. The generated test report will >>> contain each re-run information. >>> >>> So the first part of our plugin is to integrate with this new feature >>> from Maven Surefire. It will parse the report and display flaky tests, >>> output information of all the re-runs(stacktrace, output, etc) on test >>> result page. >>> >>> https://drive.google.com/file/d/0Bzfyw2NYCLFBV2lnMVlrWVhueGc >>> https://drive.google.com/file/d/0Bzfyw2NYCLFBZXFxSThlMWlaeVU >>> >>> 2. For each failing build we provide a "Deflake" action. It is different >>> from rebuilding the project, as it will: >>> >>> 1) Checkout the exact revision of that failing build (GIT) >>> 2) Only run all those failed tests. >>> >>> So this will give developer a good idea about how many tests are flaky >>> for this build. >>> >>> https://drive.google.com/file/d/0Bzfyw2NYCLFBR3RDM3psQjhIRUk/ >>> https://drive.google.com/file/d/0Bzfyw2NYCLFBeTFjQk5sQW5XR3M/ >>> >>> 3. Aggregate information from both 1 and 2, and display them to the >>> users. >>> >>> For each git revision and each test, we count how many times it passed, >>> how many times it failed. So if a test passed and failed for the both >>> revision, we count it as a flake; if it always failed, then it's a fail, >>> otherwise it's a pass. >>> >>> https://drive.google.com/file/d/0Bzfyw2NYCLFBRlI2SHpubTltd28/ >>> https://drive.google.com/file/d/0Bzfyw2NYCLFBVWJNWFAweEl1Y28/ >>> >>> So this help users to see what are my most flaky tests (they can compute >>> some kind of flakiness), so they can fix them first. It basically serves as >>> the test health page. >>> >>> We have a repo on github, however it was empty right now. >>> https://github.com/google/jenkins-flaky-test-report >>> >>> Will push our code to it very soon. So here I want to get a repo from >>> Jenkins so we can push our code to Jenkins too. >>> >>> Thank you very much! >>> >>> Regards, >>> >>> Qingzhou >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Jenkins Developers" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to jenkinsci-de...@googlegroups.com. >>> >>> For more options, visit https://groups.google.com/d/optout. >>> >> -- > You received this message because you are subscribed to the Google Groups > "Jenkins Developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to jenkinsci-dev+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Jenkins Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-dev+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
