Hi Patrick,
Once tuning is complete, how is one intended to interpret the nice green
check marks? "The library compiles" or "All the tests passed"?
So far the green checkmark essentially means "The library compiles". We
are working towards "All the tests passed", but that requires some more
tinkering in processing test output. That is, we need to extract
information about failed tests and present that in a prominent way.
Best regards,
Karli
I ask because in the demo PR there is the reassuring check mark and "3
of 3 builds passed", even though failed tests are reported (timeouts).
2018-07-20 3:35 GMT+02:00 Karl Rupp <[email protected]
<mailto:[email protected]>>:
Hi all,
we now have a first step towards full continuous integration via
Jenkins completed. Thus, every new pull request that is (re-)based
on a commit in master not older than today will be automatically
tested with a subset of common tests that are intended to expose the
most frequent issues. This, in particular, includes configurations
with 64 bit integers as well as complex arithmetic.
The integration of Jenkins into Bitbucket is smooth: You will notice
on our demo pull request
https://bitbucket.org/petsc/petsc/pull-requests/1039/jenkinsfile-for-build-pipelines-tied-to/diff
<https://bitbucket.org/petsc/petsc/pull-requests/1039/jenkinsfile-for-build-pipelines-tied-to/diff>
that on the right it says "3 of 3 builds passed". If you click on
the link, you will get further details on the individual builds and
find further links to the test output stored on the Jenkins server.
Implications on our development workflow: Currently 'next' gets
(ab)used for all kinds of portability tests. As a consequence, every
buggy merge clogs the whole integration pipeline, making it hard to
integrate other PRs. With the Jenkins server in place, all pull
requests will receive a good share of portability testing *before*
they reach next. This reduces the burden on next, (hopefully)
leading to faster code integration.
Corollary: I strongly encourage all PETSc developers to use issue
pull requests rather than merging to next directly (use your own
judgment for exceptions!).
Please note that we are still fine-tuning various aspects of the
Jenkins infrastructure (location of the Jenkins server, which test
nodes to use, which configurations to test, etc.). Most of these
things are changes under the hood, though. If something still
bubbles up and causes the testing to choke, please be considerate
with us ;-)
Finally, I'd like to explicitly thank Alp Dener for his help on
getting Jenkins to run smoothly. Any credit should go to him.
Best regards,
Karli
