On 20/08/2013 2:54 a.m., Kinkie wrote:
On Mon, Aug 19, 2013 at 4:44 PM, Amos Jeffries <squ...@treenet.co.nz> wrote:
On 20/08/2013 12:56 a.m., Kinkie wrote:
Hi all,
Now that the matrix jobs feature has matured in jenkins, I'd like to
restructure the jobs we use for our continuous integration testing to
reduce their number.
I propose to use these jobs:
- X.Y-matrix -> gcc-only (for now), build a specific revision
- trunk-compilers-matrix -> uses a combination filter to test on any
unix-like platform using any available compiler (gcc, clang, icc, msvc
to come as soon as the windows port reaches stability)
- anybranch-wholefarm-matrix -> same as above, but with some
parameters (a branch to be used, whether to use ccache or not, and an
address to be notified).
Ideally trunk-compilers-matrix should replace most 3.HEAD-* jobs,
while anybranch-wholefarm-matrix can replace most experimental jobs
(after all, a patch is just a push to lp away). The only nonmatrix
jobs that would remain are
- coverity-submit
- mswin
- possibly some support jobs
What do you think?
I plan to go ahead on Aug 25th, unless there are no objections.
Thanks!
We have several problems with the existing matrix jobs before it makes much
sense to use them this widely:
* at the top level it appears that the stable releases are failing to build
(red icon) if any one of the test nodes fail.
* there is no soft-fail type indication when a failure was with Jenkins
infrastructure.
Have the matrix support advanced far enough to resolve those issues for us
yet?
Not really.
Some advance was made with the Elastic Axis plugin (skip over
disconnected nodes) but that plugin can't be used due to other bugs.
Then the reason we had for 3.HEAD builds being separated still stands -
we need to easily and clearly see which OS to target bug fixing towards.
I am happy for the compiler-specific builds to be matrix'd though so at
most we have one 3.HEAD job per OS indicating whether it builds on all
compilers that systems users will be using.
They currently provide a false impression of instability and prevents us
testing all stable versions against all nodes simply because the newer nodes
will be for OS which the stable was never coded to build on properly (ie 3.1
wont build on FreeBSD 9/10 after compiler changes). It would be good to have
such a picture of the versions buildability but I hope we can avoid giving
the impression that none of our stables ever work.
I think it may be worth a try, but not removing the existing jobs until the
new ones have a proven record of usefulness.
How about dropping only relatively rare configurations? e.g. icc,
ALPHA-PATCH, old FreeBSD variants? That'd already go a long way
towards simplification without losing sight of the big picture.
The only reason ALPHA-PATCH are not a matrix is that I could not find a
way to propigate the uploaded file to all the matrix sub-jobs. It got
uploaded and applied on the keystone build but the others all ran
without the patch, with the obvious useless results.
Amos
