[
https://issues.apache.org/jira/browse/HBASE-15651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Appy updated HBASE-15651:
-------------------------
Attachment: HBASE-15651-master.patch
Here's a formal patch.
I have made the following changes:
- Handle multiple-configuration jobs. Se we can directly pass
'https://builds.apache.org/view/All/job/HBase-Trunk_matrix'
- Pass multiple jobs by separating urls by space. This is needed because in
final setup, we'll need to run it on two jenkins jobs: flaky-tests job (which
runs only flaky tests) and post-commit job.
- max runs flag : So if jenkins has history of many runs, we can tune this know
to only consider some amount of recent ones.
- '-mvn' flag: This will write two strings (to different files) which can be
directly passed to -Dtest and -Dtest.exclude.pattern maven flags to
include/exclude flaky tests. These files will be archived as artifacts and can
be directly curl'ed into other jenkins jobs.
> Track our flaky tests and use them to improve our build environment
> -------------------------------------------------------------------
>
> Key: HBASE-15651
> URL: https://issues.apache.org/jira/browse/HBASE-15651
> Project: HBase
> Issue Type: Improvement
> Reporter: Appy
> Assignee: Appy
> Attachments: HBASE-15651-master.patch, flakies.py
>
>
> So i have written this simple script (attached) which looks at history of the
> [post commit build|https://builds.apache.org/view/All/job/HBase-Trunk_matrix]
> and outputs a list of flaky tests with some numbers.
> Next steps:
> 1. Setup a jenkins job (say *find-flaky-tests*) to run this script daily. We
> can either directly pull these results into other jobs using curl on this
> job's artifacts, or commit the list of flaky test to repo (idk if it's
> possible to commit something from jenkins job).
> We'll collect results from both *post-commit* job (to add new flakies) and
> *flaky-tests* job (to delete tests which are no more flaky).
> 2. Change *pre-commit* and *post-commit* jobs to ignore these tests using
> --exclude maven flag. Someone familiar with yetus might be able to do it
> easily.
> 3. Setup a new job (say *flaky-tests*) to run only these flaky tests.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
