[
https://issues.apache.org/jira/browse/HBASE-15651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15250191#comment-15250191
]
Dima Spivak commented on HBASE-15651:
-------------------------------------
One more round of nits and then I'm done, I promise. :)
- Line length of 31 is huge, bring that down to 100 chars.
- For the {{\-\-max-builds}} arg, set {{metavar='n'}} to pretty up the help
message. Also consider setting a default value there so that it's clear from
the help message how many jobs will be analyzed if it's left out.
- Alphabetize the order of the args.
- On line 46, have logging as follows to make Pylint happy: {{logger.info("No
test results for %s", build_url)}}
- Just noticed this, but on line 54, truncating test names using {{\[24:\]}}
can get a little hairy. A simpler way would be:
{noformat}
test_name = "{0}#{1}".format(test['className'].rsplit('.',1)[-1], test['name'])
{noformat}
- In lines 69-70, instead of logging an error and then raising an empty
Exception, just put the error in the Exception: {{raise Exception("Bad url
(\{0\}).".format(url))}}.
That's all I got. Sorry for being so type-A, this is an awesome contribution,
[~appy].
> Track our flaky tests and use them to improve our build environment
> -------------------------------------------------------------------
>
> Key: HBASE-15651
> URL: https://issues.apache.org/jira/browse/HBASE-15651
> Project: HBase
> Issue Type: Improvement
> Reporter: Appy
> Assignee: Appy
> Attachments: HBASE-15651-master-v2.patch,
> HBASE-15651-master-v3.patch, HBASE-15651-master.patch, flakies.py
>
>
> So i have written this simple script (attached) which looks at history of the
> [post commit build|https://builds.apache.org/view/All/job/HBase-Trunk_matrix]
> and outputs a list of flaky tests with some numbers.
> Next steps:
> 1. Setup a jenkins job (say *find-flaky-tests*) to run this script daily. We
> can either directly pull these results into other jobs using curl on this
> job's artifacts, or commit the list of flaky test to repo (idk if it's
> possible to commit something from jenkins job).
> We'll collect results from both *post-commit* job (to add new flakies) and
> *flaky-tests* job (to delete tests which are no more flaky).
> 2. Change *pre-commit* and *post-commit* jobs to ignore these tests using
> --exclude maven flag. Someone familiar with yetus might be able to do it
> easily.
> 3. Setup a new job (say *flaky-tests*) to run only these flaky tests.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
