[
https://issues.apache.org/jira/browse/HBASE-15651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15248805#comment-15248805
]
Dima Spivak commented on HBASE-15651:
-------------------------------------
Great stuff overall, [~appy]. Some thoughts:
- Best practice is to set shebang to {{/usr/bin/env python}} to handle any
virtualenvs someone might wanna use.
- With three options and any number of positional arguments, you're better off
using argparse for command line parsing as opposed to manually manipulating
sys.argv. Will make it more resilient and maybe even shorter, too.
- Remove space between function name and parentheses on line 63.
- Instead of prints, import logging module and use throughout for status
updates. This makes automation easier in the future.
- Speaking of making life easier with modules, requests tends to be way easier
to read than urllib2
- Good stuff using Jenkins' tree endpoint. Not many people know how awesome it
is. :)
- Instead of using {{sys.exit(1)}} when errors happen, raise an Exception to
let people use this as a library in the future.
- Maybe add {{max-runs}} as an optional argument to let this be controlled
during invocation.
- Can you add a comment describing how you know whether a particular job is a
multi-configuration job or not? In my experience, it's been through looking for
a {{runs}} key in the JSON of a particular build.
- Use with statement when writing file to handle problems that might arise.
- You import {{sets}} and {{os}}, but I don't see you using either.
> Track our flaky tests and use them to improve our build environment
> -------------------------------------------------------------------
>
> Key: HBASE-15651
> URL: https://issues.apache.org/jira/browse/HBASE-15651
> Project: HBase
> Issue Type: Improvement
> Reporter: Appy
> Assignee: Appy
> Attachments: HBASE-15651-master.patch, flakies.py
>
>
> So i have written this simple script (attached) which looks at history of the
> [post commit build|https://builds.apache.org/view/All/job/HBase-Trunk_matrix]
> and outputs a list of flaky tests with some numbers.
> Next steps:
> 1. Setup a jenkins job (say *find-flaky-tests*) to run this script daily. We
> can either directly pull these results into other jobs using curl on this
> job's artifacts, or commit the list of flaky test to repo (idk if it's
> possible to commit something from jenkins job).
> We'll collect results from both *post-commit* job (to add new flakies) and
> *flaky-tests* job (to delete tests which are no more flaky).
> 2. Change *pre-commit* and *post-commit* jobs to ignore these tests using
> --exclude maven flag. Someone familiar with yetus might be able to do it
> easily.
> 3. Setup a new job (say *flaky-tests*) to run only these flaky tests.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
