Hi, I have a puppet infrastructure running 2.6.14 under Passenger with about 30 nodes. I'm just spinning up a new Cent 6.3, puppet 2.7.19 master. Right now I have the master built and configured using the same configuration as my 2.6.14 setup, with a few tweaks for the new version. Puppet on the master built itself fine. I have puppetd stopped on the master. I tried pointing one node at it, but I'm getting: err: Could not retrieve catalog from remote server: execution expired
I disabled passenger and ran through the builtin webrick, and confirmed that only one node is talking to the master. No change. No helpful information in the master log. I'm running the master with: /usr/sbin/puppetmasterd --no-daemonize --debug --logdest console --verbose --trace and some relevant puppet.conf portions on the master: storeconfigs = true thin_storeconfigs = true reports = log,puppet_dashboard reportfrom = root syslogfacility = user # Remote Inventory Service facts_terminus = rest inventory_server = tv-puppetma1.techtarget.com # fqdn of master inventory_port = 8140 The exact behavior I'm seeing is that when I run puppetd on the node, all works well up to seeing "debug: catalog supports formats: marshal pson raw yaml; using pson" on the node and "debug: facts supports formats: b64_zlib_yaml pson raw yaml; using pson" on the master. At that point, it hangs for 2 minutes (120 seconds exactly), with no output on either the node or master. puppetmasterd is using ~25% CPU and ~14 memory. After that time, I see the below stack trace on the master, followed by a whole slew of the same for the facts, and "err: Could not retrieve catalog from remote server: execution expired" on the node. Any ideas? I'm going to try adding resources to the master VM, but I find it hard to believe that this is just caused by compiling a single catalog on the master... Any advice or insight would be greatly appreciated. Thanks, Jason Antman Trace from master: /usr/lib/ruby/1.8/timeout.rb:64:in `rbuf_fill' /usr/lib/ruby/1.8/net/protocol.rb:134:in `rbuf_fill' /usr/lib/ruby/1.8/net/protocol.rb:116:in `readuntil' /usr/lib/ruby/1.8/net/protocol.rb:126:in `readline' /usr/lib/ruby/1.8/net/http.rb:2028:in `read_status_line' /usr/lib/ruby/1.8/net/http.rb:2017:in `read_new' /usr/lib/ruby/1.8/net/http.rb:1051:in `request' /usr/lib/ruby/1.8/net/http.rb:1037:in `request' /usr/lib/ruby/1.8/net/http.rb:543:in `start' /usr/lib/ruby/1.8/net/http.rb:1035:in `request' /usr/lib/ruby/1.8/net/http.rb:857:in `put' /usr/lib/ruby/site_ruby/1.8/puppet/indirector/rest.rb:98:in `send' /usr/lib/ruby/site_ruby/1.8/puppet/indirector/rest.rb:98:in `http_request' /usr/lib/ruby/site_ruby/1.8/puppet/indirector/rest.rb:76:in `http_put' /usr/lib/ruby/site_ruby/1.8/puppet/indirector/rest.rb:158:in `save' /usr/lib/ruby/site_ruby/1.8/puppet/indirector/indirection.rb:271:in `save' /usr/lib/ruby/site_ruby/1.8/puppet/node/facts.rb:20:in `save' /usr/lib/ruby/site_ruby/1.8/puppet/indirector/catalog/compiler.rb:26:in `extract_facts_from_request' /usr/lib/ruby/site_ruby/1.8/puppet/indirector/catalog/compiler.rb:31:in `find' /usr/lib/ruby/site_ruby/1.8/puppet/indirector/indirection.rb:195:in `find' /usr/lib/ruby/site_ruby/1.8/puppet/network/http/handler.rb:109:in `do_find' /usr/lib/ruby/site_ruby/1.8/puppet/network/http/handler.rb:68:in `send' /usr/lib/ruby/site_ruby/1.8/puppet/network/http/handler.rb:68:in `process' /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick/rest.rb:24:in `service' /usr/lib/ruby/1.8/webrick/httpserver.rb:104:in `service' /usr/lib/ruby/1.8/webrick/httpserver.rb:65:in `run' /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:45:in `listen' /usr/lib/ruby/1.8/webrick/server.rb:173:in `call' /usr/lib/ruby/1.8/webrick/server.rb:173:in `start_thread' /usr/lib/ruby/1.8/webrick/server.rb:162:in `start' /usr/lib/ruby/1.8/webrick/server.rb:162:in `start_thread' /usr/lib/ruby/1.8/webrick/server.rb:95:in `start' /usr/lib/ruby/1.8/webrick/server.rb:92:in `each' /usr/lib/ruby/1.8/webrick/server.rb:92:in `start' /usr/lib/ruby/1.8/webrick/server.rb:23:in `start' /usr/lib/ruby/1.8/webrick/server.rb:82:in `start' /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:42:in `listen' /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:41:in `initialize' /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:41:in `new' /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:41:in `listen' /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:38:in `synchronize' /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:38:in `listen' /usr/lib/ruby/site_ruby/1.8/puppet/network/server.rb:126:in `listen' /usr/lib/ruby/site_ruby/1.8/puppet/network/server.rb:141:in `start' /usr/lib/ruby/site_ruby/1.8/puppet/daemon.rb:124:in `start' /usr/lib/ruby/site_ruby/1.8/puppet/application/master.rb:202:in `main' /usr/lib/ruby/site_ruby/1.8/puppet/application/master.rb:146:in `run_command' /usr/lib/ruby/site_ruby/1.8/puppet/application.rb:309:in `run' /usr/lib/ruby/site_ruby/1.8/puppet/application.rb:416:in `hook' /usr/lib/ruby/site_ruby/1.8/puppet/application.rb:309:in `run' /usr/lib/ruby/site_ruby/1.8/puppet/application.rb:407:in `exit_on_fail' /usr/lib/ruby/site_ruby/1.8/puppet/application.rb:309:in `run' /usr/sbin/puppetmasterd:4 err: /usr/lib/ruby/1.8/timeout.rb:64:in `rbuf_fill' /usr/lib/ruby/1.8/net/protocol.rb:134:in `rbuf_fill' /usr/lib/ruby/1.8/net/protocol.rb:116:in `readuntil' /usr/lib/ruby/1.8/net/protocol.rb:126:in `readline' /usr/lib/ruby/1.8/net/http.rb:2028:in `read_status_line' /usr/lib/ruby/1.8/net/http.rb:2017:in `read_new' /usr/lib/ruby/1.8/net/http.rb:1051:in `request' /usr/lib/ruby/1.8/net/http.rb:1037:in `request' /usr/lib/ruby/1.8/net/http.rb:543:in `start' /usr/lib/ruby/1.8/net/http.rb:1035:in `request' /usr/lib/ruby/1.8/net/http.rb:857:in `put' /usr/lib/ruby/site_ruby/1.8/puppet/indirector/rest.rb:98:in `send' /usr/lib/ruby/site_ruby/1.8/puppet/indirector/rest.rb:98:in `http_request' /usr/lib/ruby/site_ruby/1.8/puppet/indirector/rest.rb:76:in `http_put' /usr/lib/ruby/site_ruby/1.8/puppet/indirector/rest.rb:158:in `save' /usr/lib/ruby/site_ruby/1.8/puppet/indirector/indirection.rb:271:in `save' /usr/lib/ruby/site_ruby/1.8/puppet/node/facts.rb:20:in `save' /usr/lib/ruby/site_ruby/1.8/puppet/indirector/catalog/compiler.rb:26:in `extract_facts_from_request' /usr/lib/ruby/site_ruby/1.8/puppet/indirector/catalog/compiler.rb:31:in `find' /usr/lib/ruby/site_ruby/1.8/puppet/indirector/indirection.rb:195:in `find' /usr/lib/ruby/site_ruby/1.8/puppet/network/http/handler.rb:109:in `do_find' /usr/lib/ruby/site_ruby/1.8/puppet/network/http/handler.rb:68:in `send' /usr/lib/ruby/site_ruby/1.8/puppet/network/http/handler.rb:68:in `process' /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick/rest.rb:24:in `service' /usr/lib/ruby/1.8/webrick/httpserver.rb:104:in `service' /usr/lib/ruby/1.8/webrick/httpserver.rb:65:in `run' /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:45:in `listen' /usr/lib/ruby/1.8/webrick/server.rb:173:in `call' /usr/lib/ruby/1.8/webrick/server.rb:173:in `start_thread' /usr/lib/ruby/1.8/webrick/server.rb:162:in `start' /usr/lib/ruby/1.8/webrick/server.rb:162:in `start_thread' /usr/lib/ruby/1.8/webrick/server.rb:95:in `start' /usr/lib/ruby/1.8/webrick/server.rb:92:in `each' /usr/lib/ruby/1.8/webrick/server.rb:92:in `start' /usr/lib/ruby/1.8/webrick/server.rb:23:in `start' /usr/lib/ruby/1.8/webrick/server.rb:82:in `start' /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:42:in `listen' /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:41:in `initialize' /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:41:in `new' /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:41:in `listen' /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:38:in `synchronize' /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:38:in `listen' /usr/lib/ruby/site_ruby/1.8/puppet/network/server.rb:126:in `listen' /usr/lib/ruby/site_ruby/1.8/puppet/network/server.rb:141:in `start' /usr/lib/ruby/site_ruby/1.8/puppet/daemon.rb:124:in `start' /usr/lib/ruby/site_ruby/1.8/puppet/application/master.rb:202:in `main' /usr/lib/ruby/site_ruby/1.8/puppet/application/master.rb:146:in `run_command' /usr/lib/ruby/site_ruby/1.8/puppet/application.rb:309:in `run' /usr/lib/ruby/site_ruby/1.8/puppet/application.rb:416:in `hook' /usr/lib/ruby/site_ruby/1.8/puppet/application.rb:309:in `run' /usr/lib/ruby/site_ruby/1.8/puppet/application.rb:407:in `exit_on_fail' /usr/lib/ruby/site_ruby/1.8/puppet/application.rb:309:in `run' /usr/sbin/puppetmasterd:4 err: execution expired -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/MmfNpa2LnIoJ. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
