Hi Louise, In my case the error was triggered by a faulty manifest which contained a variable ::$fqdn instead of $::fqdn. So it might be worth looking at the manifests and check for faulty vars.
Laurent On Dec 10, 2013 5:29 PM, "Louise Baker" <[email protected]> wrote: > > Hi, > > Apologies for the delay in replying. I completely missed the posts! > > I am yet to find a solution. I ended up reverting to 2.7.23 until I had > more time to investigate the issue. Laurent, have you had any success in > the past week or so? > > Thanks for the heads up James. I'm not explicitly setting vardir, but > will certainly do so. The puppet master is running passenger on RHEL 6. > I will try 3.3.2, however if I can't be assured the problem won't occur > again when I roll it into production, I am hesitant. Because of the > randomness it is really hard to replicate, and didn't show up in my test > environment. Perhaps it relates to load on the puppet master??? > > Lou > > On Fri, Nov 29, 2013 at 10:39 PM, <[email protected]> wrote: > >> Hi, >> >> For reference, I've just upgraded my puppet masters from 2.7.22 to 3.3.2 >> and haven't seen any errors of this kind. >> >> I presume you are running with passenger? I am too. CentOS EL6 masters. >> >> Maybe there is a change between 3.3.1 and 3.3.2 that will resolve this >> for you both. >> >> I have seen one nasty other error though. >> >> If you aren't setting the vardir explicitly in all clients puppet.conf, >> I'd suggest doing so before upgrading. >> After my upgrade, I had to manually intervene on almost all client nodes >> because puppet was failing to run. Bug filed at >> http://projects.puppetlabs.com/issues/23311 >> >> J >> >> On Friday, 29 November 2013 04:27:20 UTC, Laurent Domb wrote: >>> >>> I am running into the exact same issue with 3.3.1 Did you find a >>> solution for it? >>> >>> On Thursday, October 24, 2013 1:54:28 AM UTC-4, Lou wrote: >>>> >>>> Hello, >>>> >>>> I have a rhel 6 puppet master with the following packages installed: >>>> >>>> facter.x86_64 1:1.7.3-1.el6 >>>> hiera.noarch 1.2.1-1.el6 >>>> puppet.noarch 3.3.1-1.el6 >>>> puppet-server.noarch 3.3.1-1.el6 >>>> ruby.x86_64 1.8.7.352-12.el6_4 >>>> >>>> I have recently upgraded the puppet master from 2.7.19 to 3.3.1, >>>> downloaded from the puppetlabs yum repo. >>>> >>>> I am now randomly seeing the following errors: >>>> >>>> 1. On the node I get: >>>> … >>>> Debug: catalog supports formats: b64_zlib_yaml dot pson raw yaml; using >>>> pson >>>> Error: Could not retrieve catalog from remote server: Error 500 on >>>> SERVER: <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> >>>> <html><head> >>>> <title>500 Internal Server Error</title> >>>> </head><body> >>>> <h1>Internal Server Error</h1> >>>> <p>The server encountered an internal error or >>>> misconfiguration and was unable to complete >>>> your request.</p> >>>> <p>Please contact the server administrator, >>>> root@localhost and inform them of the time the error occurred, >>>> and anything you might have done that may have >>>> caused the error.</p> >>>> <p>More information about this error may be available >>>> in the server error log.</p> >>>> <hr> >>>> <address>Apache/2.2.15 (Red Hat) Server at <pmaster> Port 8140</address> >>>> </body></html> >>>> >>>> /opt/csw/lib/ruby/site_ruby/1.8/puppet/indirector/rest.rb:185:in >>>> `is_http_200?' >>>> /opt/csw/lib/ruby/site_ruby/1.8/puppet/indirector/rest.rb:100:in `find' >>>> /opt/csw/lib/ruby/site_ruby/1.8/puppet/indirector/indirection.rb:197:in >>>> `find' >>>> /opt/csw/lib/ruby/site_ruby/1.8/puppet/configurer.rb:243:in >>>> `retrieve_new_catalog' >>>> /opt/csw/lib/ruby/site_ruby/1.8/puppet/util.rb:351:in `thinmark' >>>> /opt/csw/lib/ruby/1.8/benchmark.rb:308:in `realtime' >>>> /opt/csw/lib/ruby/site_ruby/1.8/puppet/util.rb:350:in `thinmark' >>>> /opt/csw/lib/ruby/site_ruby/1.8/puppet/configurer.rb:242:in >>>> `retrieve_new_catalog' >>>> /opt/csw/lib/ruby/site_ruby/1.8/puppet/configurer.rb:67:in >>>> `retrieve_catalog' >>>> /opt/csw/lib/ruby/site_ruby/1.8/puppet/configurer.rb:107:in >>>> `prepare_and_retrieve_catalog' >>>> /opt/csw/lib/ruby/site_ruby/1.8/puppet/configurer.rb:159:in `run' >>>> /opt/csw/lib/ruby/site_ruby/1.8/puppet/agent.rb:45:in `run' >>>> /opt/csw/lib/ruby/site_ruby/1.8/puppet/agent/locker.rb:20:in `lock' >>>> /opt/csw/lib/ruby/site_ruby/1.8/puppet/agent.rb:45:in `run' >>>> /opt/csw/lib/ruby/1.8/sync.rb:230:in `synchronize' >>>> /opt/csw/lib/ruby/site_ruby/1.8/puppet/agent.rb:45:in `run' >>>> /opt/csw/lib/ruby/site_ruby/1.8/puppet/agent.rb:119:in `with_client' >>>> /opt/csw/lib/ruby/site_ruby/1.8/puppet/agent.rb:42:in `run' >>>> /opt/csw/lib/ruby/site_ruby/1.8/puppet/agent.rb:84:in `run_in_fork' >>>> /opt/csw/lib/ruby/site_ruby/1.8/puppet/agent.rb:41:in `run' >>>> /opt/csw/lib/ruby/site_ruby/1.8/puppet/application.rb:179:in `call' >>>> /opt/csw/lib/ruby/site_ruby/1.8/puppet/application.rb:179:in >>>> `controlled_run' >>>> /opt/csw/lib/ruby/site_ruby/1.8/puppet/agent.rb:39:in `run' >>>> /opt/csw/lib/ruby/site_ruby/1.8/puppet/application/agent.rb:353:in >>>> `onetime' >>>> /opt/csw/lib/ruby/site_ruby/1.8/puppet/application/agent.rb:327:in >>>> `run_command' >>>> /opt/csw/lib/ruby/site_ruby/1.8/puppet/application.rb:364:in `run' >>>> /opt/csw/lib/ruby/site_ruby/1.8/puppet/application.rb:456:in >>>> `plugin_hook' >>>> /opt/csw/lib/ruby/site_ruby/1.8/puppet/application.rb:364:in `run' >>>> /opt/csw/lib/ruby/site_ruby/1.8/puppet/util.rb:504:in `exit_on_fail' >>>> /opt/csw/lib/ruby/site_ruby/1.8/puppet/application.rb:364:in `run' >>>> /opt/csw/lib/ruby/site_ruby/1.8/puppet/util/command_line.rb:132:in >>>> `run' >>>> /opt/csw/lib/ruby/site_ruby/1.8/puppet/util/command_line.rb:86:in >>>> `execute' >>>> /opt/csw/bin/puppet:4 >>>> Warning: Not using cache on failed catalog >>>> Error: Could not retrieve catalog; skipping run >>>> >>>> 2. In /var/log/messages I see errors such as: >>>> >>>> Oct 24 14:01:57 <pmaster> puppet-master[13114]: Compiled catalog for >>>> <node> in environment production in 33.30 seconds >>>> Oct 24 14:02:11 <pmaster> puppet-master[13114]: YAML in network >>>> requests is deprecated and will be removed in a future version. See >>>> http://links.puppetlabs.com/deprecate_yaml_on_network >>>> Oct 24 14:02:11 <pmaster> puppet-master[13114]: (at >>>> /usr/lib/ruby/site_ruby/1.8/puppet/network/http/handler.rb:252:in >>>> `response_formatter_for') >>>> Oct 24 14:02:11 <pmaster> puppet-master[13114]: YAML in network >>>> requests is deprecated and will be removed in a future version. See >>>> http://links.puppetlabs.com/deprecate_yaml_on_network >>>> Oct 24 14:02:11 <pmaster> puppet-master[13114]: (at >>>> /usr/lib/ruby/site_ruby/1.8/puppet/network/http/handler.rb:65:in >>>> `request_format') >>>> >>>> 3. And in the apache error log I get errors such as: >>>> >>>> [Thu Oct 24 14:04:50 2013] [error] [client xx.xx.xx.xx] Premature end >>>> of script headers: <node1> >>>> [ pid=8205 thr=140243091982304 file=ext/apache2/Hooks.cpp:819 >>>> time=2013-10-24 14:04:50.263 ]: The backend application (process 13114) did >>>> not send a valid HTTP response; instead, it sent nothing at all. It is >>>> possible that it has crashed; please check whether there are crashing bugs >>>> in this application. >>>> /usr/lib/ruby/site_ruby/1.8/puppet/util/tagging.rb:42: [BUG] >>>> rb_gc_mark(): unknown data type 0x20(0x495a580) non object >>>> ruby 1.8.7 (2011-06-30 patchlevel 352) [x86_64-linux] >>>> >>>> [Thu Oct 24 14:05:58 2013] [error] [client xx.xx.xx.xx] Premature end >>>> of script headers: <node2> >>>> [ pid=8269 thr=140243091982304 file=ext/apache2/Hooks.cpp:819 >>>> time=2013-10-24 14:05:58.762 ]: The backend application (process 11392) did >>>> not send a valid HTTP response; instead, it sent nothing at all. It is >>>> possible that it has crashed; please check whether there are crashing bugs >>>> in this application. >>>> /usr/lib/ruby/site_ruby/1.8/puppet/util/tagging.rb:42: [BUG] >>>> Segmentation fault >>>> ruby 1.8.7 (2011-06-30 patchlevel 352) [x86_64-linux] >>>> >>>> --- >>>> The nodes are a mix of rhel5 and 6 and solaris 10 servers, mostly >>>> running 2.7.19. I have upgraded some rhel6 nodes to 3.3.1, and some solaris >>>> 10 servers to 3.2.4 (as that is the latest csw package available). >>>> >>>> The vast majority of the errors occur with the solaris servers running >>>> 2.7.19, but I have had a 3.2.4 fail consistently with the same error. >>>> Annoyingly I have had the error on a rhel server once, so I can’t say for >>>> certain the Solaris servers are the problem :/ >>>> >>>> I have searched for similar errors but cannot find anything exact. Any >>>> help would be greatly appreciated. >>>> >>>> Many thanks, >>>> Lou >>>> >>> -- >> You received this message because you are subscribed to the Google Groups >> "Puppet Users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/puppet-users/1c43515d-a4a8-4594-a59d-bdd6b785a4ec%40googlegroups.com >> . >> >> For more options, visit https://groups.google.com/groups/opt_out. >> > > -- > You received this message because you are subscribed to a topic in the > Google Groups "Puppet Users" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/puppet-users/yXXuVN3Bb0w/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/puppet-users/CAPWqdEZVxtqSH2jmg1OkJBggcYmjCU7iL59Od0XRQFgoUAbfsA%40mail.gmail.com > . > For more options, visit https://groups.google.com/groups/opt_out. > -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAL4%2BNOeEy1UgpOS2jA2%3DrO8y56o%3DsEMr1n%2BAe_89ur8J_qF3jQ%40mail.gmail.com. For more options, visit https://groups.google.com/groups/opt_out.
