Jira (PUP-1570) Puppet Agent hanging on applying configuration

[Brian Stolz (JIRA)]

Title: Message Title

Brian Stolz created an issue   Puppet / PUP-1570 Puppet Agent hanging on applying configuration Issue Type: Bug Affects Versions: 3.4.2 Assignee: Eric Sorenson Created: 02/Feb/14 11:17 AM Environment: ubuntu Priority: Normal Reporter: Brian Stolz Probably worth noting, I run a setup with a puppetmaster and the agents connecting to that to get their manifests. My puppet agents get stuck applying configuration every once and awhile. For example: root 769 0.0 2.8 42616 29292 ? S Jan16 0:02 puppet agent: applying configurat root 28554 0.4 2.5 38244 25996 ? Ssl Jan12 136:58 /usr/bin/ruby /usr/bin/puppet agent It is currently February 2nd, so that worker has been stuck for over 2 weeks. An strace on pid 769 gives: sudo strace -p 769 Process 769 attached - interrupt to quit select(9, [8], NULL, NULL, NULL An strace of pid gives (TRUNCATED) select(7, [5], [], [], {0, 37820}) = 0 (Timeout) clock_gettime(CLOCK_MONOTONIC, {7655144, 458961330}) = 0 select(7, [5], [], [], {0, 0}) = 0 (Timeout) waitpid(769, 0xbfde87bc, WNOHANG) = 0 clock_gettime(CLOCK_MONOTONIC, {7655144, 459362511}) = 0 select(7, [5], [], [], {0, 0}) = 0 (Timeout) clock_gettime(CLOCK_MONOTONIC, {7655144, 459607993}) = 0 clock_gettime(CLOCK_MONOTONIC, {7655144, 459728396}) = 0 select(7, [5], [], [], {0, 59879}) = 0 (Timeout) clock_gettime(CLOCK_MONOTONIC, {7655144, 520035825}) = 0 select(7, [5], [], [], {0, 0}) = 0 (Timeout) waitpid(769, 0xbfde87bc, WNOHANG) = 0 clock_gettime(CLOCK_MONOTONIC, {7655144, 520416241}) = 0 select(7, [5], [], [], {0, 0}) = 0 (Timeout) clock_gettime(CLOCK_MONOTONIC, {7655144, 520631447}) = 0 clock_gettime(CLOCK_MONOTONIC, {7655144, 520747775}) = 0 select(7, [5], [], [], {0, 59883}) = 0 (Timeout) clock_gettime(CLOCK_MONOTONIC, {7655144, 581028381}) = 0 select(7, [5], [], [], {0, 0}) = 0 (Timeout) waitpid(769, 0xbfde87bc, WNOHANG) = 0 clock_gettime(CLOCK_MONOTONIC, {7655144, 581324512}) = 0 select(7, [5], [], [], {0, 0}) = 0 (Timeout) clock_gettime(CLOCK_MONOTONIC, {7655144, 581516746}) = 0 clock_gettime(CLOCK_MONOTONIC, {7655144, 581634011}) = 0 select(7, [5], [], [], {0, 59882}) = 0 (Timeout) clock_gettime(CLOCK_MONOTONIC, {7655144, 641928723}) = 0 select(7, [5], [], [], {0, 0}) = 0 (Timeout) waitpid(769, 0xbfde87bc, WNOHANG) = 0 clock_gettime(CLOCK_MONOTONIC, {7655144, 642200491}) = 0 select(7, [5], [], [], {0, 0}) = 0 (Timeout) clock_gettime(CLOCK_MONOTONIC, {7655144, 642371221}) = 0 clock_gettime(CLOCK_MONOTONIC, {7655144, 642460015}) = 0 select(7, [5], [], [], {0, 59911}) = 0 (Timeout) clock_gettime(CLOCK_MONOTONIC, {7655144, 702822877}) = 0 select(7, [5], [], [], {0, 0}) = 0 (Timeout) waitpid(769, 0xbfde87bc, WNOHANG) = 0 clock_gettime(CLOCK_MONOTONIC, {7655144, 703168884}) = 0 select(7, [5], [], [], {0, 0}) = 0 (Timeout) clock_gettime(CLOCK_MONOTONIC, {7655144, 703388116}) = 0 clock_gettime(CLOCK_MONOTONIC, {7655144, 703480552}) = 0 select(7, [5], [], [], {0, 59907}) = 0 (Timeout) clock_gettime(CLOCK_MONOTONIC, {7655144, 763785916}) = 0 select(7, [5], [], [], {0, 0}) = 0 (Timeout) waitpid(769, 0xbfde87bc, WNOHANG) = 0 clock_gettime(CLOCK_MONOTONIC, {7655144, 764168802}) = 0 select(7, [5], [], [], {0, 0}) = 0 (Timeout) clock_gettime(CLOCK_MONOTONIC, {7655144, 764396951}) = 0 clock_gettime(CLOCK_MONOTONIC, {7655144, 764496825}) = 0 select(7, [5], [], [], {0, 59900}) = 0 (Timeout) clock_gettime(CLOCK_MONOTONIC, {7655144, 824800600}) = 0 select(7, [5], [], [], {0, 0}) = 0 (Timeout) waitpid(769, 0xbfde87bc, WNOHANG) = 0 clock_gettime(CLOCK_MONOTONIC, {7655144, 825105655}) = 0 select(7, [5], [], [], {0, 0}) = 0 (Timeout) clock_gettime(CLOCK_MONOTONIC, {7655144, 825301581}) = 0 clock_gettime(CLOCK_MONOTONIC, {7655144, 825463069}) = 0 select(7, [5], [], [], {0, 59838}) = 0 (Timeout) I will not kill this, and will leave it running. Let me know what else you need to debug this. Add Comment   This message was sent by Atlassian JIRA (v6.1.4#6159-sha1:44eaede) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/groups/opt_out.