Hi Tim,
On 17/05/2013 13:42, Tim Brown wrote:
We have been using Jenkins for a while now (nearly two years) and have
come up against a problem. We need to run unit tests on Multiple OSs and
against multiple resources. We also need to lock each of those resources
while they are in use and not let other jobs use them. Our jobs
currently look as Follows:
Yep, same here, we have many environments in a matrix, but each point in
the matrix requires a database server (postgres version is one of our
axes) and host to run on (linux distro version is another axis) and only
one point in the matrix can run on a host at one time.
As such we have looked into the following plugins that we thought could
help us:
Throttle Concurrent Jobs:
Well, Matrix jobs have the feel of a horrible hack (check out the fact
that the parent job executes all the pre-build steps, even though
they're never used...) but they're too useful not to use...
Throttle Concurrent Jobs tries to work around some of these hacks, but
it's buggy and the conflicting usecases of some users haven't been
properly coded for in the configuration options...
We found that this worked at the child level,
I think that's what you want though... Each point in the matrix is a
child, and it's each child that you want to lock resources, right?
You wan the children to run in parallel to speed things up, but not
trample on each other, right?
there is a fix for this:
https://issues.jenkins-ci.org/browse/JENKINS-12240 but even then this
does not fix our problem (as we add one Category Name per matrix job and
this applied to all child jobs).
I've given up on the Throttle Concurrent Jobs plugin, it's buggy, and
doesn't work. We've fallen back to having a bunch of slaves with
database servers on them that only have one executor. They're tagged up
with db server version and os version and we use those as axes in the
matrix. It works fine, but it means that other builds (C++ compiles,
python and JS static code analysis, etc) can't run on those hosts, which
is a shame as they're quite beefy hosts with plenty of spare cycles...
Seeing as none of these seem to fit what we are trying to do we were
thinking of add the concept of a that can be managed in a similar way to
nodes (Manage Jenkins -> Manage Resource -> add/Configure Resource).
This would allow people to set labels, names and Number of Executors.
The we could add an Axis to Matrix Jobs (e.g. ‘Resources’) to allow
people to select a group of slaves and a groups of resources in their jobs.
I think the config is all there already with labels on nodes.
I think two things are needed:
- a plugin that actually lets you say how many jobs using a particular
label can run on a machine at one time (ie: 1 in both your and my usecases).
- fix the crazy in matrix parent jobs that mean they execute all their
pre-build steps (which can easily trample child jobs on the same node!)
cheers,
Chris
--
Simplistix - Content Management, Batch Processing & Python Consulting
- http://www.simplistix.co.uk
--
You received this message because you are subscribed to the Google Groups "Jenkins
Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/groups/opt_out.
