Issue #2691 has been updated by Christian Hofstaedtler.
You could - as a possible workaround - try to set Apache's Timeout to something larger than 300 seconds and see if that "helps" (i.e. make the "Broken Pipe" error go away). Is it slow too with puppetmasterd instead of Passenger or is it only slow with Passenger? ---------------------------------------- Bug #2691: "Could not retrieve catalog: HTTP-Error: 500 Internal Server Error" with tagged exported resources http://projects.reductivelabs.com/issues/2691 Author: Marc Fournier Status: Unreviewed Priority: Normal Assigned to: Category: Target version: Affected version: 0.25.1rc1 Keywords: Branch: The problem happens with this sort of manifest (in fact, this simple example works fine): <pre> node a { @@file { "/tmp/foo": content => "fjskfjs\n", tag => "foofile", } } node b { File <<| tag == 'foofile' |>> } </pre> In my case, I have a about 20 nodes, each one having: <pre> @@nagios_host { $fqdn: tag => "nagiosserver.domain.tld", ... } @@nagios_service { "a": tag => "nagiosserver.domain.tld", ... } @@nagios_service { "b": tag => "nagiosserver.domain.tld", ... } @@nagios_service { "c": tag => "nagiosserver.domain.tld", ... } @@nagios_service { "d": tag => "nagiosserver.domain.tld", ... } @@nagios_service { "e": tag => "nagiosserver.domain.tld", ... } </pre> I suspect the problem occurs when many exported resources are involved (if 20x5 can be considered many). The problem happens when running puppet on a node containing: <pre> Nagios_host <<| tag == "nagiosserver.domain.tld" |>> Nagios_service <<| tag == nagiosserver.domain.tld" |>> </pre> On the puppetmaster, one ruby process consumes 100% CPU during a few minutes, then the client says: <pre> err: Could not call puppetmaster.getconfig: #<RuntimeError: HTTP-Error: 500 Internal Server Error> err: Could not retrieve catalog: HTTP-Error: 500 Internal Server Error </pre> the ruby process continues running at 100%, and after a while we get this backtrace: <pre> [ pid=3338 file=ext/apache2/Hooks.cpp:547 time=2009-10-01 11:16:46.189 ]: Backend process 3465 did not return a valid HTTP response. It returned no data. *** Exception Errno::EPIPE in Passenger RequestHandler (Broken pipe) (process 3465): from /usr/lib/ruby/gems/1.8/gems/passenger-2.2.2/lib/phusion_passenger/rack/request_handler.rb:83:in `write' from /usr/lib/ruby/gems/1.8/gems/passenger-2.2.2/lib/phusion_passenger/rack/request_handler.rb:83:in `process_request' from /usr/lib/ruby/gems/1.8/gems/passenger-2.2.2/lib/phusion_passenger/abstract_request_handler.rb:203:in `main_loop' from /usr/lib/ruby/gems/1.8/gems/passenger-2.2.2/lib/phusion_passenger/rack/application_spawner.rb:110:in `run' from /usr/lib/ruby/gems/1.8/gems/passenger-2.2.2/lib/phusion_passenger/rack/application_spawner.rb:67:in `spawn_application' from /usr/lib/ruby/gems/1.8/gems/passenger-2.2.2/lib/phusion_passenger/utils.rb:181:in `safe_fork' from /usr/lib/ruby/gems/1.8/gems/passenger-2.2.2/lib/phusion_passenger/rack/application_spawner.rb:60:in `spawn_application' from /usr/lib/ruby/gems/1.8/gems/passenger-2.2.2/lib/phusion_passenger/rack/application_spawner.rb:45:in `spawn_application' from /usr/lib/ruby/gems/1.8/gems/passenger-2.2.2/lib/phusion_passenger/spawn_manager.rb:158:in `spawn_application' from /usr/lib/ruby/gems/1.8/gems/passenger-2.2.2/lib/phusion_passenger/spawn_manager.rb:282:in `handle_spawn_application' from /usr/lib/ruby/gems/1.8/gems/passenger-2.2.2/lib/phusion_passenger/abstract_server.rb:337:in `__send__' from /usr/lib/ruby/gems/1.8/gems/passenger-2.2.2/lib/phusion_passenger/abstract_server.rb:337:in `main_loop' from /usr/lib/ruby/gems/1.8/gems/passenger-2.2.2/lib/phusion_passenger/abstract_server.rb:187:in `start_synchronously' from /usr/lib/ruby/gems/1.8/gems/passenger-2.2.2/bin/passenger-spawn-server:61 </pre> The problem can be produced with 0.24.8 and 0.25.0 clients, and 0.25.0 as well as 0.25.1rc1 server, with passenger and these gems installed: activerecord (2.3.2) activesupport (2.3.2) fastthread (1.0.7) passenger (2.2.2) rack (1.0.0) rake (0.8.7) It should be noted that this works fine: <pre> Nagios_host <<| |>> Nagios_service <<| |>> </pre> I've also noticed the following error from time to time on 0.25.0. But I'm unable to reproduce it, and never seen it with 0.25.1rc1, so I'm not sure it's related: <pre> /srv/puppet/lib/puppet/util/settings/file_setting.rb:19:in `group=' /srv/puppet/lib/puppet/util/settings/setting.rb:44:in `send' /srv/puppet/lib/puppet/util/settings/setting.rb:44:in `initialize' /srv/puppet/lib/puppet/util/settings/setting.rb:38:in `each' /srv/puppet/lib/puppet/util/settings/setting.rb:38:in `initialize' /srv/puppet/lib/puppet/util/settings.rb:398:in `new' /srv/puppet/lib/puppet/util/settings.rb:398:in `newsetting' /srv/puppet/lib/puppet/util/settings.rb:533:in `setdefaults' /srv/puppet/lib/puppet/util/settings.rb:518:in `each' /srv/puppet/lib/puppet/util/settings.rb:518:in `setdefaults' /srv/puppet/lib/puppet/reports/store.rb:15:in `mkclientdir' /srv/puppet/lib/puppet/reports/store.rb:35:in `process' /srv/puppet/lib/puppet/network/handler/report.rb:66:in `process' /srv/puppet/lib/puppet/network/handler/report.rb:59:in `each' /srv/puppet/lib/puppet/network/handler/report.rb:59:in `process' /srv/puppet/lib/puppet/network/handler/report.rb:33:in `report' /usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `to_proc' /srv/puppet/lib/puppet/network/xmlrpc/processor.rb:52:in `call' /srv/puppet/lib/puppet/network/xmlrpc/processor.rb:52:in `protect_service' /srv/puppet/lib/puppet/network/xmlrpc/processor.rb:85:in `setup_processor' /usr/lib/ruby/1.8/xmlrpc/server.rb:336:in `call' /usr/lib/ruby/1.8/xmlrpc/server.rb:336:in `dispatch' /usr/lib/ruby/1.8/xmlrpc/server.rb:323:in `each' /usr/lib/ruby/1.8/xmlrpc/server.rb:323:in `dispatch' /usr/lib/ruby/1.8/xmlrpc/server.rb:366:in `call_method' /usr/lib/ruby/1.8/xmlrpc/server.rb:378:in `handle' /srv/puppet/lib/puppet/network/xmlrpc/processor.rb:44:in `process' /srv/puppet/lib/puppet/network/http/rack/xmlrpc.rb:35:in `process' /srv/puppet/lib/puppet/network/http/rack.rb:48:in `call' /usr/lib/ruby/gems/1.8/gems/passenger-2.2.2/lib/phusion_passenger/rack/request_handler.rb:81:in `process_request' /usr/lib/ruby/gems/1.8/gems/passenger-2.2.2/lib/phusion_passenger/abstract_request_handler.rb:203:in `main_loop' /usr/lib/ruby/gems/1.8/gems/passenger-2.2.2/lib/phusion_passenger/rack/application_spawner.rb:110:in `run' /usr/lib/ruby/gems/1.8/gems/passenger-2.2.2/lib/phusion_passenger/rack/application_spawner.rb:67:in `spawn_application' /usr/lib/ruby/gems/1.8/gems/passenger-2.2.2/lib/phusion_passenger/utils.rb:181:in `safe_fork' /usr/lib/ruby/gems/1.8/gems/passenger-2.2.2/lib/phusion_passenger/rack/application_spawner.rb:60:in `spawn_application' /usr/lib/ruby/gems/1.8/gems/passenger-2.2.2/lib/phusion_passenger/rack/application_spawner.rb:45:in `spawn_application' /usr/lib/ruby/gems/1.8/gems/passenger-2.2.2/lib/phusion_passenger/spawn_manager.rb:158:in `spawn_application' /usr/lib/ruby/gems/1.8/gems/passenger-2.2.2/lib/phusion_passenger/spawn_manager.rb:282:in `handle_spawn_application' /usr/lib/ruby/gems/1.8/gems/passenger-2.2.2/lib/phusion_passenger/abstract_server.rb:337:in `__send__' /usr/lib/ruby/gems/1.8/gems/passenger-2.2.2/lib/phusion_passenger/abstract_server.rb:337:in `main_loop' /usr/lib/ruby/gems/1.8/gems/passenger-2.2.2/lib/phusion_passenger/abstract_server.rb:187:in `start_synchronously' /usr/lib/ruby/gems/1.8/gems/passenger-2.2.2/bin/passenger-spawn-server:61 </pre> -- You have received this notification because you have either subscribed to it, or are involved in it. To change your notification preferences, please click here: http://reductivelabs.com/redmine/my/account --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/puppet-bugs?hl=en -~----------~----~----~----~------~----~------~--~---
