Karl Rupp via petsc-dev <[email protected]> writes: > With some effort we can certainly address 1.) and to some extent 3.), > probably 4.) as well, but I don't know how to solve 2.) and 5.) with > Jenkins. Given that a significant effort is required for 1.), 3.) and > 4.) anyway, I'm starting to get more and more comfortable with the idea > of rolling our own CI infrastructure (which has been suggested in some > of Barry's snarky remarks already ;-) ). Small Python scripts for > executing the tests and pushing results to Bitbucket as well as a > central result storage can replicate our existing setup with a few lines > of codes, while being much more flexible.
I think further commitment to Bitbucket would be a liability. On existing open source CI tools, I think looking at how the project itself uses CI is a good indicator. Some examples of recent PRs with test failures, see what needs to be done to narrow down what failed. https://github.com/buildbot/buildbot/pull/4726 https://github.com/drone/drone/pull/2363 https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/27652 (click Expand on the test failure) https://github.com/jenkinsci/jenkins/pull/3991
