Issue #961 has been updated by madduck.
With the help of Luke (hi Luke!), we've found the following:
The problem seems to be in lib/puppet/network/xmlrpc/client.rb at line 40:
<pre>
begin
-> call("%s.%s" % [namespace, method.to_s],*args)
rescue [...]
</pre>
It seems that the connection after the call() method is never closed. The
following patch verifies and fixes that, but it only closes the connection if
there isn't an exception (it's only a proof-of-concept):
<pre>
--- a/lib/puppet/network/xmlrpc/client.rb
+++ b/lib/puppet/network/xmlrpc/client.rb
@@ -39,6 +39,10 @@ module Puppet::Network
Puppet.debug "Calling %s.%s" % [namespace, method]
begin
result = call("%s.%s" % [namespace, method.to_s],*args)
+ if http.started?
+ Puppet.err "HTTP instance still open after call to
%s.%s" % [namespace, method]
+ http.finish
+ end
result
rescue OpenSSL::SSL::SSLError => detail
if detail.message =~ /bad write retry/
Puppet.warning "Transient SSL write error;
restarting connection and retrying"
</pre>
This outputs a whole lot of errors for me, so Luke has agreed to
rewrite/refactor/fix this, and I will test.
Whether it fixes the Debian bug is currently unknown.
----------------------------------------
Bug #961: puppetd creating too many/not closing TCP connections
http://projects.reductivelabs.com:80/issues/961
Author: col_s
Status: Re-opened
Priority: Normal
Assigned to: luke
Category: network
Target version:
Complexity: Unknown
Patch: None
Affected version: 0.24.1
Keywords:
With a single client puppetd run, established concurrent TCP connections
between the client and the puppetmaster often exceeds 100.
If two or three clients are run in parallel, and sometimes even with a single
client, the number of connections exceeds 256. On my Solaris 10 puppetmaster,
the maximum number of
open files defaults to 256, when this limit is exceeded for the puppetmasterd
process, no further connections can be opened and we receive the following
error message on the client(s):
<pre>
warning: Other end went away; restarting connection and retrying
</pre>
Before starting a client run, the number of connections is zero:
<pre>
$ netstat -an | grep 8140.*ESTABLISH | wc -l
0
</pre>
Soon after I start puppetd on the client:
<pre>
puppetmaster $ netstat -an | grep 8140.*ESTABLISH | wc -l
101
</pre>
Prior to 0.24.0 I don't believe puppetd opened many connections to the
puppetmaster concurrently.
I have needed to increase the limit on maximum open files on the puppetmaster,
but this is not an ideal workaround as having all those sockets open consumes a
lot of system resources and i'm still very limited in the number of clients we
can run concurrently. So I think that this is a major issue.
----------------------------------------
You have received this notification because you have either subscribed to it,
or are involved in it.
To change your notification preferences, please click here:
http://reductivelabs.com/redmine/my/account
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Bugs" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/puppet-bugs?hl=en
-~----------~----~----~----~------~----~------~--~---
