Dear mates, I'm recently in charge of quite big Jenkins configuration based on a central master (no build on master) with :
>7000 Jobs 136 active Slaves mostly tied build 150-200 Total Slaves ~450 potential executors Users are on many R&D sites in Europe, slaves are spread over the WAN each site has a Nexus proxy next to the slaves. SCM servers (SVN/ClearCase/Git) are not on the same LAN... At the moment the console is quite slow and we had recently memory trouble that end-up adding 4 more Gb of RAM to the server and review the JVM setup below. At the moment it looks better, but I can't be sure because most builds are schedule at night... I installed the monitoring plugin. The strange thing is that Jenkins is using more memory on the Windows server then just the total of XMX + MaxPermSize (7+1 Gb), and ends-up using more then 10 Gb... any idea why? I found a memory leak affecting the PermGen memory, the number of loaded classes rise continuously and fills-up PermGem memory at the same time. I have to restart Jenkins weekly. It saw also slave build making surefire reports over 200Mb going back to the server. I'm also convinced that I should migrate to a more recent version of Jenkins, but with such a big config and so much users, it's not so easy! *What do you think of such a configuration?* Do you think the server is well sized or under-sized, should I use several master? Any TIPs for large scale administration is welcome! *My configuration :* VMWare VM Win 2K8R2 64bits 12 Gb VRAM 4 VCPU Xeon X5660 2,8 Ghz JENKINS : 1.424.1 TOMCAT : 6.0.29 JDK : 1.7.0_03 Tomcat JVM Setup as a NT Service : -Xms7g -Xmx7g -Xss128kb -XX:+UseLargePages -XX:PermSize=1g -XX:MaxPermSize=1g -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:+CMSIncrementalMode -XX:+CMSIncrementalPacing -XX:CMSInitiatingOccupancyFraction=33 -XX:CMSIncrementalDutyCycleMin=10 -XX:CMSIncrementalDutyCycle=50 -XX:ParallelGCThreads=8 -XX:MaxGCPauseMillis=2000 -XX:GCTimeRatio=10 -XX:+DisableExplicitGC -Xloggc:K:\apache-tomcat-6.0.29\logs\gc.log -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+UseCompressedOops -Duser.language=US -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
