Sadly, last time I checked, the answer was yes. There is a lock free load balancing algorithm I found a paper on that I want to try, but day job has my effort directed elsewhere at the minute...
On Tuesday 18 October 2016, Pavel Janousek <[email protected]> wrote: > On the other hand, is it really necessary to call all associated Listeners > (like in [1].update() - L222, L236, L242) when the Queue is still locked > (is it guaranteed and documented for a ProvisioningListener contract)? It > can be a quite time spent operation/call... > > [1] https://github.com/jenkinsci/jenkins/blob/master/core/src/ > main/java/hudson/slaves/NodeProvisioner.java > > -- > Pavel Janousek > Senior Jenkins QA Engineer > > > ----- Original Message ----- > > From: "Stephen Connolly" <[email protected] <javascript:;> > > > > To: [email protected] <javascript:;> > > Sent: Monday, October 17, 2016 11:51:10 PM > > Subject: Re: Adding a node with Queue.withLock in Jenkins > > > > On 17 October 2016 at 22:42, Surya Gaddipati <[email protected] > <javascript:;>> > > wrote: > > > > > Hi Stephen, > > > > > > Thank you for your quick response. I don't want to belabor this more > than > > > it needs to be . > > > > > > >If you change the list of nodes during that time, the queue thread > will > > > get a concurrent modification exception (best case) and die > > > > > > I am guessing you are referring to this line > > > <https://github.com/jenkinsci/jenkins/blob/master/core/src/ > main/java/hudson/model/Queue.java#L1388> > > > > > > > > > for (Computer c : Jenkins.getInstance().getComputers()) > > > > > > This is not same collection that addnode modifies, its a copy . So it > > > seems unlikely that would cause a concurrent modification exception. > > > > > > > > It's when you then end up down at > > https://github.com/jenkinsci/jenkins/blob/master/core/src/ > main/java/hudson/model/Queue.java#L1525-L1526 > > that the issues start cropping up. > > > > In theory, add should be safe and only remove requiring the lock... in > > practice we found that a lot of the cloud plugins blow up if we remove > the > > add lock. If you want to remove the add lock for your Jenkins and have > > complete control over the plugins installed, you may get lucky > > > > > > > > > > > it will assign work to a node that no longer exists, except for a > > > phantom object reference that it held onto... or worse > > > > > > This won't happen in this specific case I am describing here since a > > > computer is tied to *one* particular build , it cannot be assigned to a > > > phantom node. > > > > > > > > > My main motivation is to remove queue locking as much as possible as > our > > > Jenkins instance has major scalability issues, almost *all* of it > stemming > > > from overzealous queue locking. > > > > > > > > > > It is a long term goal to remove the queue lock > > > > > > I would love for queue.lock to go away but do you think it realistic > to > > > expect that to go away anytime soon given how deeply embedded it is > into > > > the core structure of the code. eg: Here it looks like we are > acquiring a > > > queue lock > > > <https://github.com/jenkinsci/jenkins/blob/master/core/src/ > main/java/hudson/model/Executor.java#L337>within > > > a queue lock > > > <https://github.com/jenkinsci/jenkins/blob/master/core/src/ > main/java/hudson/model/Executor.java#L351> > > > ? > > > > > > > > > > > > > > > > > > On Monday, October 17, 2016 at 1:54:11 PM UTC-5, Stephen Connolly > wrote: > > > > > >> > > >> > > >> On Monday 17 October 2016, Surya Gaddipati <[email protected] > <javascript:;>> wrote: > > >> > > >>> Thanks Stephen for your quick response. > > >>> > > >>> > as otherwise the scheduling will blow up in your face. > > >>> > > >>> Curious, What do you mean by this ? > > >>> > > >> > > >> When the queue starts scheduling it has to iterate the list of nodes > to > > >> build up the candidate nodes for the load balancing algorithm. > > >> > > >> If you change the list of nodes during that time, the queue thread > will > > >> get a concurrent modification exception (best case) and die... or it > will > > >> assign work to a node that no longer exists, except for a phantom > object > > >> reference that it held onto... or worse > > >> > > >> It is a long term goal to remove the queue lock as it impacts > scalability > > >> when you have 1000's of executors that can match a job... but even > then > > >> the > > >> impact is not so large that removing the lock would be a priority. > > >> > > >> For now, use the queue lock methods, when we remove the need for a > lock > > >> they will become no-ops that the JVM will inline away for plugins > compiled > > >> against current cores > > >> > > >> > > >>> thanks again. > > >>> > > >>> On Monday, October 17, 2016 at 9:53:23 AM UTC-5, Surya Gaddipati > wrote: > > >>>> > > >>>> Hi all, > > >>>> > > >>>> I am working on plugin > > >>>> <https://github.com/suryagaddipati/jenkins-docker-swarm-plugin> > that > > >>>> creates a single use computer/node whose lifecycle is tied to a > single > > >>>> build . I am currently adding the node like this > > >>>> > > >>>> Jenkins.getInstance().addNode(node); > > >>>> > > >>>> > > >>>> However that method requires multiple Queue locks while doing so. > > >>>> > > >>>> > > >>>> I believe in my particular case there is no need for queue locking > > >>>> since only a single build can ever be scheduled on the computer via > > >>>> LabelAssignmentAction. > > >>>> > > >>>> I wanted to check , > > >>>> > > >>>> 1. if that assumption is correct > > >>>> > > >>>> 2. If the team is open to accepting a patch to jenkins.instance > for a > > >>>> new method which adds a node without the Queue lock. > > >>>> > > >>>> > > >>>> Thank you. > > >>>> > > >>>> > > >>>> --Surya > > >>>> > > >>> -- > > >>> You received this message because you are subscribed to the Google > > >>> Groups "Jenkins Developers" group. > > >>> To unsubscribe from this group and stop receiving emails from it, > send > > >>> an email to [email protected] > <javascript:;>. > > >>> To view this discussion on the web visit > https://groups.google.com/d/ms > > >>> gid/jenkinsci-dev/6a9cb43d-8eab-4413-89e9-f85775f5bc03%40goo > > >>> glegroups.com > > >>> <https://groups.google.com/d/msgid/jenkinsci-dev/6a9cb43d- > 8eab-4413-89e9-f85775f5bc03%40googlegroups.com?utm_medium= > email&utm_source=footer> > > >>> . > > >>> For more options, visit https://groups.google.com/d/optout. > > >>> > > >> > > >> > > >> -- > > >> Sent from my phone > > >> > > > -- > > > You received this message because you are subscribed to the Google > Groups > > > "Jenkins Developers" group. > > > To unsubscribe from this group and stop receiving emails from it, send > an > > > email to [email protected] <javascript:;>. > > > To view this discussion on the web visit https://groups.google.com/d/ > > > msgid/jenkinsci-dev/8daeff83-4cbb-4654-8886-c78586655d91% > > > 40googlegroups.com > > > <https://groups.google.com/d/msgid/jenkinsci-dev/8daeff83- > 4cbb-4654-8886-c78586655d91%40googlegroups.com?utm_medium= > email&utm_source=footer> > > > . > > > > > > For more options, visit https://groups.google.com/d/optout. > > > > > > > -- > > You received this message because you are subscribed to the Google Groups > > "Jenkins Developers" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [email protected] <javascript:;>. > > To view this discussion on the web visit > > https://groups.google.com/d/msgid/jenkinsci-dev/CA%2BnPnMwMv85% > 3D9nTLOYfiJUzAiiassAEhLDocD%3DOMncMmd9yJ-Q%40mail.gmail.com. > > For more options, visit https://groups.google.com/d/optout. > > > > -- > You received this message because you are subscribed to the Google Groups > "Jenkins Developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] <javascript:;>. > To view this discussion on the web visit https://groups.google.com/d/ > msgid/jenkinsci-dev/382396873.181768.1476774525919.JavaMail. > zimbra%40redhat.com. > For more options, visit https://groups.google.com/d/optout. > -- Sent from my phone -- You received this message because you are subscribed to the Google Groups "Jenkins Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CA%2BnPnMyHfLEMMxrVCO77nzw70U1HkJbHZaWLh5%3DC6RJzuQdb7w%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
