> > So the simple solution I see is having a throttle on how many jobs can be > added into the queue. Some logic will ask the build how many current builds > are running and how many are currently in the queue, and when the number > drops bellow a threshold we add 1 more to the queue. >
I don't think such double-stage queue is a good idea. Jenkins has many features depending on the full task list in the queue: Cloud provisioning, Load balancing, Priorities. All this features won't work properly with the approach. Since "Throttle Concurrent Builds" plugin operates with Jenkins.Queue only, it is not a right place to implement such scheduling in any case. Matrix Project Plugin could be enhanced to limit the number of configurations being submitted to the queue. This approach may address particular use-cases (an indermediate option between the default behavior and "run sequentially" option) even if the concerns above also apply to such approach. I still think that the queue pagination and filtering are the right solution of this issue, which is mostly related to UI/UX. You can try to introduce the new feature in Matrix project. Best regards, Oleg Nenashev 2015-10-06 21:13 GMT+03:00 Dima Kovalenko <[email protected]>: > Hey Olen, and everyone. > > So the thing I’m trying to avoid is a situation like this > http://i.imgur.com/Joz7LmS.png > > Imagine an iOS app that needs to be tested on all of the locales, and > versions, etc… etc… You know that the build will take 24 hours for full > regression because you can only purchase so many mac nodes. > > Putting all of the builds in the queue is first quite ugly but also > putting them into the queue can make Jenkins unusable ( > https://issues.jenkins-ci.org/browse/JENKINS-20046) > > So one option is to break the build down into smaller chunks and daisy > chain them, but then you no longer have a nice matrix view at a glance to > see problem spots (The main reason to use a multi build matrix) > http://emptysqua.re/blog/media/2013/04/pymongo-jenkins-matrix.png > > So the simple solution I see is having a throttle on how many jobs can be > added into the queue. Some logic will ask the build how many current builds > are running and how many are currently in the queue, and when the number > drops bellow a threshold we add 1 more to the queue. > > This will also prevent a situation where you have 10 nodes, and 1 team has > 30 configurations, putting everyone else at the bottom of the queue > (granted, this is a duplication of priority sorter and accelerated build > now plugin). But by limiting how many jobs are in the queue for a given > build we can allow a build with a large queue to run for a while without > taking over everyone’s resources. > > From: Oleg Nenashev <[email protected]> > Date: Tue, 6 Oct 2015 20:45:19 +0300 > To: Dima Kovalenko <[email protected]> > Cc: JenkinsCI Developers <[email protected]>, Tom Fennelly < > [email protected]> > Subject: Re: Throttle Concurrent Builds Plugin Question > > Hi Dmitry, > > Added jenkinsci-dev mailing list to Cc. It's a right place to discuss > particular improvements (especially if you add plugin owners to Cc). > > I am trying to set up a multi-configuration matrix build, that will have >> 250 to 500 builds. However, we only have 100 nodes to run at the time, and >> I do not want to see the remaining 400 builds in the Queue. .... Is it >> possible to user your plugin to restrict how many builds get put into the >> queue? > > > No, it's not possible right now. > > If this feature is not implemented, would you be interested in a PR with >> the implementation? > > > There are two ways to hide items in the queue: > > - Decline the build scheduling. > - This build won't be executed at all > - Matrix project will likely fail the build after it > - I doubt this feature complies with the plugin design > - Filter out the job from the web interface > - AFAIK there is no extension points in Jenkins core allowing to do > it > - This feature could be useful > > What actually do toy try to achieve? IIRC there is a PR from Tom Fennelly > (in Cc), which provides a pagination of the build queue. > https://github.com/jenkinsci/jenkins/pull/1641 . Probably it may address > your use-case if you want just to reduce the size of the list. > Best regards, > Oleg > > 2015-10-06 17:19 GMT+03:00 Dima Kovalenko <[email protected]>: > >> Hey Oleg, >> >> I am trying to set up a multi-configuration matrix build, that will have >> 250 to 500 builds. However, we only have 100 nodes to run at the time, and >> I do not want to see the remaining 400 builds in the Queue. >> >> Is it possible to user your plugin to restrict how many builds get put >> into the queue? Sorry to send email directly but could not find much >> information about this config online. If this feature is not implemented, >> would you be interested in a PR with the implementation? >> >> Thanks >> > > -- You received this message because you are subscribed to the Google Groups "Jenkins Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAPfivLB%3DQ913VGqOt3VREDsnvAMtNrKHSkSgmOh8%2BKCcte7UyA%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
