On 12/13/2016 09:41 AM, Martin Babinsky wrote:
https://github.com/freeipa/freeipa/pull/177 was recently merged
despite causing nearly half of the tests in our Travis CI gating to
fail. This broke Travis CI for all other PR that were rebased after
this merge, causing false negative errors everywhere.
Fraser reverted the offending commits in
https://github.com/freeipa/freeipa/pull/329 which restored Travis to
original state (never mind PEP8 errors they were in the original code
Regarding this issues I have two questions:
should I merge https://github.com/freeipa/freeipa/pull/329 and thus
revert the breakage in order to unblock other contributors? Given the
current traffic I think it is sufficient to wait for us to investigate
and produce a fix. If not, please scream loudly.
what can we improve to make the results of CI more visible to
contributors? I think that I should sit down with Martin 2 and
investigate the possibility to send notifications about negative CI
results (sufficient IMO) to the mailing list.
In the meanwhile I would like to ask all reviewers to carefully check
the output of failed Travis CI runs. If the job fails, you will see
the results at the very end of the log. There are two sections: PEP8
errors and test output. You can expand both of them to see what went
wrong and report it to the PR author if necessary.
The reviewer and author can then use the very same tool used in CI 
to reproduce the failures locally. Using '--no-cleanup' option during
the run  leaves behind a running container which you can attach to
and investigate further.
If you have any additional questions/suggestions about Travis feel
free to contact me.
I would personally wait for a proper fix in the a) case. We've been able
to perform well even without Travis up until recently plus 99% of the
pull requests are created and reviewed by team members who are aware of
as for b): I can't quite imagine how sending mails to the devel list
would work but I assume you'd come up with some kind of sane solution.
Manage your subscription for the Freeipa-devel mailing list:
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code