Issue #13038 has been updated by Michael Stahnke. Status changed from Unreviewed to Needs More Information Assignee set to Erik Dalén
Are you on RHEL or CentOS or something similar? We've seen issues with specific kernels on EL 5 systems. ---------------------------------------- Bug #13038: Hung agent processes https://projects.puppetlabs.com/issues/13038#change-56489 Author: Erik Dalén Status: Needs More Information Priority: Normal Assignee: Erik Dalén Category: Target version: Affected Puppet version: 2.7.11 Keywords: select hung process stuck Branch: Since upgrading to 2.7.11 (from 2.6.2) we have experienced a number or hosts where the puppet process has gotten stuck. Doing a strace shows the following: Process 15272 attached - interrupt to quit select(0, NULL, NULL, NULL, {0, 386506}) = 0 (Timeout) select(0, NULL, NULL, NULL, {1, 0}) = 0 (Timeout) select(0, NULL, NULL, NULL, {1, 0}) = 0 (Timeout) select(0, NULL, NULL, NULL, {1, 0}) = 0 (Timeout) select(0, NULL, NULL, NULL, {1, 0}) = 0 (Timeout) select(0, NULL, NULL, NULL, {1, 0}) = 0 (Timeout) select(0, NULL, NULL, NULL, {1, 0}) = 0 (Timeout) select(0, NULL, NULL, NULL, {1, 0}) = 0 (Timeout) select(0, NULL, NULL, NULL, {1, 0}) = 0 (Timeout) select(0, NULL, NULL, NULL, {1, 0}) = 0 (Timeout) select(0, NULL, NULL, NULL, {1, 0}) = 0 (Timeout) select(0, NULL, NULL, NULL, {1, 0}^C <unfinished ...> Process 15272 detached lsof output: # lsof -p 15272 COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME puppet 15272 root cwd DIR 254,0 4096 917506 /root puppet 15272 root rtd DIR 254,0 4096 2 / puppet 15272 root txt REG 254,0 5424 794665 /usr/bin/ruby1.8 puppet 15272 root mem REG 254,0 80712 1180151 /lib/libresolv-2.11.2.so puppet 15272 root mem REG 254,0 22928 1180172 /lib/libnss_dns-2.11.2.so puppet 15272 root mem REG 254,0 37984 922400 /usr/lib/ruby/1.8/x86_64-linux/zlib.so puppet 15272 root mem REG 254,0 10784 925695 /usr/lib/ruby/1.8/x86_64-linux/shadow.so puppet 15272 root mem REG 254,0 8648 922394 /usr/lib/ruby/1.8/x86_64-linux/digest/md5.so puppet 15272 root mem REG 254,0 45824 922412 /usr/lib/ruby/1.8/x86_64-linux/socket.so puppet 15272 root mem REG 254,0 93936 793016 /usr/lib/libz.so.1.2.3.4 puppet 15272 root mem REG 254,0 1693344 797851 /usr/lib/libcrypto.so.0.9.8 puppet 15272 root mem REG 254,0 349248 797852 /usr/lib/libssl.so.0.9.8 puppet 15272 root mem REG 254,0 298664 922401 /usr/lib/ruby/1.8/x86_64-linux/openssl.so puppet 15272 root mem REG 254,0 15304 922415 /usr/lib/ruby/1.8/x86_64-linux/racc/cparse.so puppet 15272 root mem REG 254,0 12712 922397 /usr/lib/ruby/1.8/x86_64-linux/digest.so puppet 15272 root mem REG 254,0 12808 922392 /usr/lib/ruby/1.8/x86_64-linux/digest/sha1.so puppet 15272 root mem REG 254,0 14392 791358 /usr/lib/gconv/UTF-16.so puppet 15272 root mem REG 254,0 17064 922409 /usr/lib/ruby/1.8/x86_64-linux/iconv.so puppet 15272 root mem REG 254,0 19352 922402 /usr/lib/ruby/1.8/x86_64-linux/strscan.so puppet 15272 root mem REG 254,0 271400 922421 /usr/lib/ruby/1.8/x86_64-linux/nkf.so puppet 15272 root mem REG 254,0 12680 922406 /usr/lib/ruby/1.8/x86_64-linux/syslog.so puppet 15272 root mem REG 254,0 47616 1180155 /lib/libnss_files-2.11.2.so puppet 15272 root mem REG 254,0 43552 1180152 /lib/libnss_nis-2.11.2.so puppet 15272 root mem REG 254,0 89064 1180168 /lib/libnsl-2.11.2.so puppet 15272 root mem REG 254,0 31616 1180165 /lib/libnss_compat-2.11.2.so puppet 15272 root mem REG 254,0 5344 922420 /usr/lib/ruby/1.8/x86_64-linux/fcntl.so puppet 15272 root mem REG 254,0 11032 922399 /usr/lib/ruby/1.8/x86_64-linux/etc.so puppet 15272 root mem REG 254,0 16560 922398 /usr/lib/ruby/1.8/x86_64-linux/thread.so puppet 15272 root mem REG 254,0 119984 922403 /usr/lib/ruby/1.8/x86_64-linux/syck.so puppet 15272 root mem REG 254,0 21120 922413 /usr/lib/ruby/1.8/x86_64-linux/stringio.so puppet 15272 root mem REG 254,0 1432968 1180158 /lib/libc-2.11.2.so puppet 15272 root mem REG 254,0 530736 1180167 /lib/libm-2.11.2.so puppet 15272 root mem REG 254,0 35104 1180169 /lib/libcrypt-2.11.2.so puppet 15272 root mem REG 254,0 14696 1180157 /lib/libdl-2.11.2.so puppet 15272 root mem REG 254,0 31744 1180160 /lib/librt-2.11.2.so puppet 15272 root mem REG 254,0 131260 1180159 /lib/libpthread-2.11.2.so puppet 15272 root mem REG 254,0 927840 794548 /usr/lib/libruby1.8.so.1.8.7 puppet 15272 root mem REG 254,0 128744 1180170 /lib/ld-2.11.2.so puppet 15272 root mem REG 254,0 26048 793220 /usr/lib/gconv/gconv-modules.cache puppet 15272 root 0r FIFO 0,8 0t0 765633332 pipe puppet 15272 root 1u REG 254,0 0 393323 /tmp/tmpfs2dR5B (deleted) puppet 15272 root 2u REG 254,0 0 393323 /tmp/tmpfs2dR5B (deleted) puppet 15272 root 3r FIFO 0,8 0t0 765635635 pipe puppet 15272 root 4w FIFO 0,8 0t0 765635635 pipe puppet 15272 root 5u unix 0xffff8801b88f0f00 0t0 765635637 socket -- You have received this notification because you have either subscribed to it, or are involved in it. To change your notification preferences, please click here: http://projects.puppetlabs.com/my/account -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/puppet-bugs?hl=en.
