I personally don't see an issue with those feature additions, but Kristian does the lions share of the work on the Surefire plugin so it would be his call. I would definitely support the additions of those features and would help test.
On May 28, 2014, at 1:46 PM, Qingzhou Luo <[email protected]> wrote: > Hi Jason, > > Thanks for your reply. We plan to first extend surefire, so users can put a > boolean parameter rerunFailintTests and an int parameter > rerunFailintTestsCount, to tell surefire whether to rerun failing tests > immediately after they fail. If a test passes in any of those reruns if will > be marked as pass, but all the failing attempts will still be recorded and > may be presented to users in a different way. > > We plan to finish everything in next 1-2 months. So we wonder would you be > willing to accept our contribution in the end, and patch our code into > surefire? > > Thanks, > > Qingzhou > > > On Wed, May 28, 2014 at 5:41 AM, Jason van Zyl <[email protected]> wrote: > > On May 27, 2014, at 1:14 PM, Qingzhou Luo <[email protected]> wrote: > >> Hi, >> >> I am an intern at Google. The first step of my internship project is to add >> the ability to Maven to automatically rerun failing tests a few times, to >> see if they ever pass in any of the reruns. It is useful because in many >> cases a test fails because it is flaky, not because there is a bug in the >> new source code change. >> > > I think generally being able to re-run failed tests is useful. I use this > feature in the IDE all the time in a fail/fix/run/repeat scenario. If you're > trying to identify flaky tests I think that's useful especially if you can > collect statistics. While In the long term you may not want to rely on the > re-running of failed tests that later succeed without alteration as the basis > for good testing, but it's a good identification tool, but a useful feature > in its own right. > >> We think the right way to achieve this is to modify surefire plugin of >> maven. We want to add it as a part of the configuration of surefire, so >> users can decide whether they want to enable this feature, and how many >> times they want to rerun failing tests. We plan to open-source our >> contribution, and hopefully can merge our code into surefire master branch >> in the end. Therefore, we are wondering do you have any >> comments/suggestions/opinions regarding this? We appreciate any input. >> > > Maybe a simple mechanism where you store the failed tests in a file and then > have a flag to run only the failed tests. I think this would be very useful. > Then possible a small, configurable loop around the main execution of tests > if you wanted to specify how many times to run the tests. > >> Thank you very much! >> >> Regards, >> >> Qingzhou > > Thanks, > > Jason > > ---------------------------------------------------------- > Jason van Zyl > Founder, Apache Maven > http://twitter.com/jvanzyl > http://twitter.com/takari_io > --------------------------------------------------------- > > The modern conservative is engaged in one of man's oldest exercises in moral > philosophy; that is, > the search for a superior moral justification for selfishness. > > -- John Kenneth Galbraith > > > > > > > > > > Thanks, Jason ---------------------------------------------------------- Jason van Zyl Founder, Apache Maven http://twitter.com/jvanzyl http://twitter.com/takari_io --------------------------------------------------------- Selfish deeds are the shortest path to self destruction. -- The Seven Samuari, Akira Kurosawa
