Issue #2089 has been reported by dlgtho.
----------------------------------------
Bug #2089: puppet client on nodes hangs after various networking glitches
http://projects.reductivelabs.com/issues/2089
Author: dlgtho
Status: Unreviewed
Priority: Normal
Assigned to:
Category:
Target version:
Complexity: Unknown
Affected version: 0.24.7
Keywords:
sometimes i find puppet on nodes stuck. usually it is several nodes together.
they hang there doing "nothing". i restart those in order to get them running
again.
it seems network glitches are responsible for this behavior.
here is a quick trace(it's the same on all nodes):
hey:~# ps aux| grep puppet
root 12629 0.5 2.1 129820 88896 ? Ss 03:41 2:26 ruby
/usr/sbin/puppetd -w 0
hey:~# strace -f -p 12629
Process 12629 attached - interrupt to quit
select(12, [9 10], [], [], {0, 184000}) = 0 (Timeout)
select(12, [9 10], [], [], {0, 1477}) = 0 (Timeout)
select(12, [9 10], [], [], {0, 0}) = 0 (Timeout)
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
select(12, [9 10], [], [], {1, 999998}) = 0 (Timeout)
select(12, [9 10], [], [], {0, 1706}) = 0 (Timeout)
select(12, [9 10], [], [], {0, 0}) = 0 (Timeout)
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
select(12, [9 10], [], [], {1, 999997} <unfinished ...>
Process 12629 detached
hey:~# file /proc/12629/fd/{9,10}
/proc/12629/fd/9: broken symbolic link to `socket:[93425425]'
/proc/12629/fd/10: broken symbolic link to `socket:[93847992]'
hey:~# lsof -n | egrep "(93425425|93847992)"
ruby 12629 root 9u IPv4 93425425 TCP
*:8139 (LISTEN)
ruby 12629 root 10u IPv4 93847992 TCP
localip:46803->puppetmaster:8140 (ESTABLISHED)
hey:# file /proc/12629/fd/*
0: symbolic link to `/dev/null'
1: symbolic link to `/dev/null'
10: broken symbolic link to `socket:[93847992]'
2: symbolic link to `/dev/null'
3: broken symbolic link to `pipe:[93425407]'
4: broken symbolic link to `pipe:[93425407]'
5: broken symbolic link to `socket:[93425419]'
6: symbolic link to `/var/log/puppet/http.log'
7: symbolic link to `/var/log/puppet/http.log'
8: symbolic link to `/var/log/puppet/http.log'
9: broken symbolic link to `socket:[93425425]'
i'm ready to provide more info(if requested) next time it happens.
----------------------------------------
You have received this notification because you have either subscribed to it,
or are involved in it.
To change your notification preferences, please click here:
http://reductivelabs.com/redmine/my/account
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Bugs" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/puppet-bugs?hl=en
-~----------~----~----~----~------~----~------~--~---
