remoting has tons of improvements between those two versions 3.33 vs 4.5, if you agents ar JNLP the new default protocols is JNLP 4 that is it's better than JNLP 3 IIRC it has an important performance issue
El jueves, 1 de octubre de 2020 a las 12:33:53 UTC+2, Dominik escribió: > We upgraded our Jenkins instance yesterday (from version 2.190.1 to > 2.249.1) and the heavy load on the master "magically disappeared". > The exact same test suite now runs (and 'spams' all the agents), but the > load on master stays below a value of 3 (with 4 CPUs). > > On Friday, September 25, 2020 at 5:05:11 PM UTC+2 Dominik wrote: > >> Hi Kamil, >> >> Thanks for the link. Although we do use pipelines for many projects, the >> set of test jobs I mentioned are plain old 'freestyle' jobs. >> They include one or multiple Maven build steps which run some more or >> less heavy unit and integration tests (which all happens on agents anyway). >> When looking at top/htop on the server during those tests, it's really >> just the Jenkins main process rotating at 380-400% CPU usage (of 4 CPUs). >> >> Regards, >> Dominik >> >> On Friday, September 25, 2020 at 3:28:23 PM UTC+2 [email protected] >> wrote: >> >>> Hi Dominik, >>> >>> High load on the master might be caused by incorrectly implemented >>> Jenkinsfiles which contain complex logic, thus inducing load on the master >>> instead of the nodes. Furthermore, this load will not be visible in the >>> node overview. I think this is the best point to start investigating this >>> issue from, see this piece of documentation for explanation: >>> https://www.jenkins.io/doc/book/pipeline/pipeline-best-practices/#making-sure-to-use-groovy-code-in-pipelines-as-glue >>> >>> Best regards, >>> Kamil >>> >>> On Friday, September 25, 2020 at 2:26:47 PM UTC+2 Dominik wrote: >>> >>>> Hi, >>>> >>>> We are running Jenkins with about 20 build agents on separate VMs (with >>>> up to 4 executors each). The master has 4 CPUs, so I would expect a load >>>> of >>>> about 4 when there's "a lot of work" to be done. >>>> >>>> However, no job is ever scheduled on the master itself, and still we >>>> can observe very high load levels on the master when many jobs are running >>>> concurrently on the agents. For example, one of our test suites launches >>>> about 30 jobs (lasting 5-20min each) in a 1h time slot and at that moment >>>> the master reaches a load of up to 40, with all CPUs fully occupied! >>>> However, looking at the UI, all jobs clearly run on the agents, and the >>>> master simply shows as 'Idle'. >>>> >>>> When I look at Jenkins' own monitoring, I see a huge amount of >>>> *Computer.threadPoolForRemoting >>>> *and similar threads with a summary stating *Threads on master: *Number >>>> = 415, Maximum = 511, Total started = 7,731,717. >>>> >>>> Can anyone give some insight on what could be going on? What can be the >>>> reasons for high load on the master when all the jobs are running on >>>> agents? >>>> >>>> Regards, >>>> Dominik >>>> >>> -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/8b729806-0cf6-4fdd-a598-236d623220f1n%40googlegroups.com.
