I agree with Zoltan. The continuously braking tests make it very hard to spot real issues. Any thoughts on doing it automatically?
> On Feb 22, 2018, at 10:47 AM, Zoltan Haindrich <k...@rxd.hu> wrote: > > * > > Hello, > > * > * > > ** > > In the last couple weeks the number of broken tests have started to go > up...and even tho I run bisect/etc from time to time ; sometimes people don’t > react to my comments/tickets/etc. > > Because keeping this many failing tests makes it easier for a new one to slip > in...I think reverting the patch introducing the test failures would also > help in some case. > > I think it would help a lot to prevent further test breaks to revert the > patch if any of the following conditions is met: > > * > * > > C1) if the notification/comment about the fact that the patch indeed broken a > test somehow have been unanswered for at least 24 hours. > > C2) if the patch is in for 7 days; but the test failure is still not > addressed (note that in this case there might be a conversation about fixing > it...but in this case ; to enable other people to work in a cleaner > environment is more important than a single patch - and if it can't be fixed > in 7 days...well it might not get fixed in a month). > > * > * > > I would like to also note that I've seen a few tickets which have been picked > up by people who were not involved in creating the original change - and > although the intention was good, they might miss the context of the original > patch and may "fix" the tests in the wrong way: accept a q.out which is > inappropriate or ignore the test... > > * > * > > would it be ok to implement this from now on? because it makes my efforts > practically useless if people are not reacting… > > * > * > > note: just to be on the same page - this is only about running a single test > which falls on its own - I feel that flaky tests are an entirely different > topic. > > * > * > > cheers, > > Zoltan > > ** > *
