Re: Better management of flakies

https://issues.apache.org/jira/browse/HBASE-15651 On Fri, Apr 1, 2016 at 1:49 PM, Stack wrote: > On Fri, Apr 1, 2016 at 10:05 AM, Sean Busbey wrote: > > > > Maybe better if we ensure everything needed to run our jobs is scripts > > in dev-support so

Re: Better management of flakies

On Fri, Apr 1, 2016 at 10:05 AM, Sean Busbey wrote: > Maybe better if we ensure everything needed to run our jobs is scripts > in dev-support so that the commiter-only change can just be moving > jenkins to execute them. > > I like this idea Sean. St.Ack > On Fri,

Re: Better management of flakies

On Tue, Mar 29, 2016 at 4:46 PM, Matteo Bertozzi wrote: > don't we have already a flakey detector? HBASE-8018 > > https://github.com/apache/hbase/commit/4dc52261a19ed1055b837b136c97ea36e362ba6e > > I remember that script. It used to work for me but had forgotten about

Re: Better management of flakies

If you create a jira that describes what we need to implement, I'll find time to put in changes. Maybe better if we ensure everything needed to run our jobs is scripts in dev-support so that the commiter-only change can just be moving jenkins to execute them. On Fri, Apr 1, 2016 at 12:01 PM,

Re: Better management of flakies

So who's setting up the jobs? I don't have perms to do it. On Wed, Mar 30, 2016 at 11:24 AM, Apekshit Sharma wrote: > So tackling the two parts individually: > 1) Detection : A good automatic flaky detector will no doubt save manual > effort. The one mentioned by Matteo seems

Re: Better management of flakies

So tackling the two parts individually: 1) Detection : A good automatic flaky detector will no doubt save manual effort. The one mentioned by Matteo seems like a good one which we can hook up with post-commit job. I see a minor problem though, we'll should use about 20 runs at least, but the rate

Re: Better management of flakies

bq. We start demoting tests to flaky list which never get fixed. I think this may happen. Jenkins jobs for 1.x builds are relatively stable - meaning, we would have either green Java 7 build or green Java 8 build (if not both). Broken test(s) would be quickly identified and fixed. I don't think

Re: Better management of flakies

don't we have already a flakey detector? HBASE-8018 https://github.com/apache/hbase/commit/4dc52261a19ed1055b837b136c97ea36e362ba6e On Tue, Mar 29, 2016 at 4:39 PM, Sean Busbey wrote: > How about if we make a script for determining flakey status, based on > rate of failing

Re: Better management of flakies

How about if we make a script for determining flakey status, based on rate of failing in either post-commit or in the pre-existing branch check for precommit. The script could live in dev-support and work by effectively grabbing the test reports from those jobs. We can then run that script in a

Re: Better management of flakies

Where/How do we maintain the list of tests that are flagged as flakes? On Tue, Mar 29, 2016 at 6:25 PM, Apekshit Sharma wrote: > Proposal: > Maintain a list of flaky tests which can be ignored in main builds (for > patches) and setup a new job which will periodically run those

Better management of flakies

Proposal: Maintain a list of flaky tests which can be ignored in main builds (for patches) and setup a new job which will periodically run those flaky tests. Benefits: - Cleaner main builds. Less runs per patch, ideally no re-runs because of timeouts and other flakiness. Less frustration.