Got to the bottom of this via stackoverflow <http://stackoverflow.com/questions/33027437/insufficient-memory-on-jenkins-server-startup>: it was an issue with the glibc distributed with Red Hat Enterprise Linux 6 as discussed here <https://issues.apache.org/jira/browse/HADOOP-7154>. The issue can be worked around via explicit setting of the environment variable MALLOC_ARENA_MAX, in my case export MALLOC_ARENA_MAX=2. Without explicit setting of this variable, the JVM will apparently attempt to spawn (8 x cpu core count) threads, each consuming 64M. My 40 core case would have required northward of 10 gigs of virtual ram, exceeding (by itself) the ulimit on my machine of 8 gigs. Setting this to 2 reduces VM consumption to around 128 megs, though I'll probably increase it to something like 16 when I start using Jenkins seriously.
On Tuesday, October 6, 2015 at 10:00:36 AM UTC-4, Trevor Williams wrote: > > Ugh, correcting spelling of the subject. > > Me fail English? That's unpossible! > > On Monday, October 5, 2015 at 7:27:01 PM UTC-4, Trevor Williams wrote: >> >> Hi all, >> >> First time user of Jenkins here, and having a bit of trouble getting it >> started. From the Linux shell I run a command like: >> >> java -Xms512m -Xmx512m -jar jenkins.war >> >> and consistently get an error like: >> >> # There is insufficient memory for the Java Runtime Environment to >> continue. >> # pthread_getattr_np >> # An error report file with more information is saved as: >> # /home/twilliams/.jenkins/hs_err_pid36290.log >> >> First, the basics: >> >> - Jenkins 1.631 >> - Running via the jetty embedded in the war file >> - OpenJDK 1.7.0_51 >> - Oracle Linux (3.8.13-55.1.5.el6uek.x86_64) >> - 386 GB ram >> - 40 cores >> >> I get the same problem with a number of other configurations as well: >> using Java Hotspot 1.8.0_60, running through Apache Tomcat, and using all >> sorts of different values for -Xms/-Xmx/-Xss and similar options. >> >> I've done a fair bit of research and *think* I know what the problem is, >> but am at a loss as how to solve it. I suspect that I'm running into the >> virtual memory overcommit issue mentioned here >> <http://serverfault.com/questions/317115/jenkins-ci-cannot-allocate-memory>; >> the relevant bits from ulimit: >> >> --($:)-- ulimit -a >> ... >> max locked memory (kbytes, -l) 64 >> max memory size (kbytes, -m) 8388608 >> stack size (kbytes, -s) 8192 >> virtual memory (kbytes, -v) 8388608 >> ... >> >> If I double the virtual memory limit as root, I can start Jenkins, but >> I'd rather not run Jenkins as the root user. >> >> Another workaround: a soon-to-be-decommissioned machine with 48 GB ram >> and 24 cores can start Jenkins without issue, though (I suspect) just >> barely: according to htop, its virtual memory footprint is just over 8 GB. >> I suspect, as a result, that the memory overcommit issue is scaling with >> the number of processors on the machine, presumably the result of Jenkins >> starting a number of threads proportional to the number of processors >> present on the host machine. I roughly captured the thread count via ps >> -eLf | grep jenkins | wc -l and found that the thread count spikes at >> around 114 on the 40 core machine, and 84 on the 24 core machine. >> >> Does this explanation seem sound? Provided it does... >> >> 1. Is there any way to configure Jenkins to reduce the number of >> threads it spawns at startup? >> 2. Are there any VMs available that don't suffer from the overcommit >> issue, or some configuration option to address it? >> >> The sanest option at this point may be to just run Jenkins in a >> virtualized environment to limit the resources at its disposal to something >> reasonable, but at this point I'm interested in this problem on an >> intellectual level and want to know how to get this recalcitrant >> configuration to behave. >> >> Thanks, >> Trevor >> > -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/aba1b6a4-6461-457f-a826-d3aa4b0bb0fd%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
