Issue #2089 has been updated by jamtur01.

Category set to plumbing
Status changed from Unreviewed to Needs more information
Target version set to unplanned

Can you try this on 0.24.8?
----------------------------------------
Bug #2089: puppet client on nodes hangs after various networking glitches
http://projects.reductivelabs.com/issues/2089

Author: dlgtho
Status: Unreviewed
Priority: Normal
Assigned to: 
Category: plumbing
Target version: unplanned
Complexity: Unknown
Affected version: 0.24.7
Keywords: 


sometimes i find puppet on nodes stuck. usually it is several nodes together.
they hang there doing "nothing". i restart those in order to get them running 
again.
it seems network glitches are responsible for this behavior.
here is a quick trace(it's the same on all nodes):

hey:~# ps aux| grep puppet
root     12629  0.5  2.1 129820 88896 ?        Ss   03:41   2:26 ruby 
/usr/sbin/puppetd -w 0

hey:~# strace -f -p 12629
Process 12629 attached - interrupt to quit
select(12, [9 10], [], [], {0, 184000}) = 0 (Timeout)
select(12, [9 10], [], [], {0, 1477})   = 0 (Timeout)
select(12, [9 10], [], [], {0, 0})      = 0 (Timeout)
rt_sigprocmask(SIG_BLOCK, NULL, [], 8)  = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8)  = 0
select(12, [9 10], [], [], {1, 999998}) = 0 (Timeout)
select(12, [9 10], [], [], {0, 1706})   = 0 (Timeout)
select(12, [9 10], [], [], {0, 0})      = 0 (Timeout)
rt_sigprocmask(SIG_BLOCK, NULL, [], 8)  = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8)  = 0
select(12, [9 10], [], [], {1, 999997} <unfinished ...>
Process 12629 detached

hey:~# file /proc/12629/fd/{9,10}
/proc/12629/fd/9:  broken symbolic link to `socket:[93425425]'
/proc/12629/fd/10: broken symbolic link to `socket:[93847992]'

hey:~# lsof -n | egrep "(93425425|93847992)"
ruby      12629     root    9u     IPv4           93425425                  TCP 
*:8139 (LISTEN)
ruby      12629     root   10u     IPv4           93847992                  TCP 
localip:46803->puppetmaster:8140 (ESTABLISHED)

hey:# file /proc/12629/fd/*
0:  symbolic link to `/dev/null'
1:  symbolic link to `/dev/null'
10: broken symbolic link to `socket:[93847992]'
2:  symbolic link to `/dev/null'
3:  broken symbolic link to `pipe:[93425407]'
4:  broken symbolic link to `pipe:[93425407]'
5:  broken symbolic link to `socket:[93425419]'
6:  symbolic link to `/var/log/puppet/http.log'
7:  symbolic link to `/var/log/puppet/http.log'
8:  symbolic link to `/var/log/puppet/http.log'
9:  broken symbolic link to `socket:[93425425]'

i'm ready to provide more info(if requested) next time it happens.


----------------------------------------
You have received this notification because you have either subscribed to it, 
or are involved in it.
To change your notification preferences, please click here: 
http://reductivelabs.com/redmine/my/account

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups 
"Puppet Bugs" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to 
[email protected]
For more options, visit this group at 
http://groups.google.com/group/puppet-bugs?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply via email to