Hi We are tried to update puppet master from 2.6.6 to 2.7.18 and saw similar picture. Catalog compile time took ~60-120 seconds. Before this update compilation took about ~1-5 second. After upgrade average CPU usage increased from 10-20% up to 20-40%, during some activity (compilation?) ruby process puppetmasterd take 100% too.
All, is there are some magic to solve or this is expected behaviour? Thanks! --- Alexander On Saturday, December 15, 2012 10:23:49 PM UTC+4, treydock wrote: > > > > On Sunday, December 9, 2012 1:41:34 PM UTC-6, treydock wrote: >> >> I am finding that the puppetlabs-apache module is somehow adding 30-60 >> seconds onto a host's catalog compile time when the puppetmaster has no >> other hosts contacting or generating catalogs. The Puppetmaster is setup >> to use Puppet-2.7.18 - Apache & Passenger. RIght now only 2 hosts are even >> configured to use this new PM, the PM itself and a Foreman host. With >> neither hosts contacting the PM, and disabling the "include apache" and >> "require apache::mod::dev" in puppetlabs-passenger, I get "Config >> retrieval: 2.45". >> >> Catalogs had hostnames replaced, puppetmaster.tld is the CNAME for Puppet >> master and puppetmaster-host.tld is the actual hostname. >> >> Catalog: http://pastebin.com/V29JmB8J >> # puppet agent --{summarize,test,debug,evaltrace,noop} | perl -pe >> 's/^/localtime().": "/e' >> <snip> >> Sun Dec 9 13:05:35 2012: notice: Finished catalog run in 2.43 seconds >> Sun Dec 9 13:05:35 2012: Changes: >> Sun Dec 9 13:05:35 2012: Events: >> Sun Dec 9 13:05:35 2012: Total: 1 >> Sun Dec 9 13:05:35 2012: Noop: 1 >> Sun Dec 9 13:05:35 2012: Resources: >> Sun Dec 9 13:05:35 2012: Out of sync: 1 >> Sun Dec 9 13:05:35 2012: Total: 22 >> Sun Dec 9 13:05:35 2012: Skipped: 6 >> Sun Dec 9 13:05:35 2012: Time: >> Sun Dec 9 13:05:35 2012: Resources: 0.00 >> Sun Dec 9 13:05:35 2012: Filebucket: 0.00 >> Sun Dec 9 13:05:35 2012: Package: 0.00 >> Sun Dec 9 13:05:35 2012: File: 0.00 >> Sun Dec 9 13:05:35 2012: Firewall: 0.01 >> Sun Dec 9 13:05:35 2012: Exec: 0.14 >> Sun Dec 9 13:05:35 2012: Last run: 1355079935 >> Sun Dec 9 13:05:35 2012: Config retrieval: 2.45 >> Sun Dec 9 13:05:35 2012: Total: 2.61 >> Sun Dec 9 13:05:35 2012: Version: >> Sun Dec 9 13:05:35 2012: Config: 1355077701 >> Sun Dec 9 13:05:35 2012: Puppet: 2.7.18 >> >> Uncommenting "include apache" and "require apache::mod::dev" in >> puppetlabs-passenger about 40 seconds are added to the compile time. I >> went through disabled all modules and parameters, and found that enabling >> the passenger module alone with Apache included caused the huge delay. >> >> Catalog: http://pastebin.com/wwcKgX5b >> # puppet agent --{summarize,test,debug,evaltrace,noop} | perl -pe >> 's/^/localtime().": "/e' >> <snip> >> Sun Dec 9 13:13:08 2012: notice: Finished catalog run in 2.23 seconds >> Sun Dec 9 13:13:08 2012: Changes: >> Sun Dec 9 13:13:08 2012: Events: >> Sun Dec 9 13:13:08 2012: Total: 4 >> Sun Dec 9 13:13:08 2012: Noop: 4 >> Sun Dec 9 13:13:08 2012: Resources: >> Sun Dec 9 13:13:08 2012: Out of sync: 4 >> Sun Dec 9 13:13:08 2012: Skipped: 6 >> Sun Dec 9 13:13:08 2012: Total: 80 >> Sun Dec 9 13:13:08 2012: Time: >> Sun Dec 9 13:13:08 2012: Filebucket: 0.00 >> Sun Dec 9 13:13:08 2012: Resources: 0.00 >> Sun Dec 9 13:13:08 2012: Package: 0.00 >> Sun Dec 9 13:13:08 2012: Firewall: 0.01 >> Sun Dec 9 13:13:08 2012: File: 0.01 >> Sun Dec 9 13:13:08 2012: A2mod: 0.03 >> Sun Dec 9 13:13:08 2012: Service: 0.07 >> Sun Dec 9 13:13:08 2012: Exec: 0.14 >> Sun Dec 9 13:13:08 2012: Last run: 1355080388 >> Sun Dec 9 13:13:08 2012: Config retrieval: 42.79 >> Sun Dec 9 13:13:08 2012: Total: 43.06 >> Sun Dec 9 13:13:08 2012: Version: >> Sun Dec 9 13:13:08 2012: Config: 1355080261 >> Sun Dec 9 13:13:08 2012: Puppet: 2.7.18 >> >> >> The install is via RPM. I'm using the Fedora SRPMs rebuilt for CentOS 6. >> The Puppet master has two environments, development and production with >> this as the puppet.conf >> >> # cat /etc/puppet/puppet.conf >> --- >> [main] >> server = puppetmaster.tld >> logdir = /var/log/puppet >> vardir = /var/lib/puppet >> ssldir = /var/lib/puppet/ssl >> rundir = /var/run/puppet >> factpath = $vardir/lib/facter >> pluginsync = true >> templatedir = $confdir/templates >> >> >> [agent] >> report = true >> environment = production >> preferred_serialization_format = yaml >> >> [master] >> modulepath = >> /etc/puppet/environments/$environment/modules:/etc/puppet/modules >> manifest = /etc/puppet/manifests/site.pp >> environment = production >> autosign = false >> ssl_client_header = SSL_CLIENT_S_DN >> ssl_client_verify_header = SSL_CLIENT_VERIFY >> certname = puppetmaster.tld >> dns_alt_names = puppetmaster.tld,puppetmaster-host.tld >> report = true >> reports = store >> #reports = store, foreman >> >> [production] >> manifest = /etc/puppet/manifests/site.pp >> >> >> Has anyone run into this issue with puppetlabs-apache or know why my >> compile times are so long? I have a Puppet 2.6.17 server (Also >> Apache+Passenger) that has one host, for example, with 524 resources , and >> only a 21.36 second catalog compile time. >> >> Thanks >> - Trey >> > > Continuing to debug this issue it seems the apache::mod::default is the > bulk of the compile time. I'm curious if this is a symptom of the module > itself or something with 2.7.x. I moved this puppet master to the EPEL > 2.6.17 RPMs and now catalogs that previously took 40-60 seconds are now > down to 1-2 seconds. > > I noticed that during the long compile times this VM's (2 vCPU , 2GB RAM) > CPU usage is at 100% from a single Ruby process that links to the PM Rack > process. My other PM running same exact setup with about 50 managed hosts > never has had performance issues always running 2.6.17. > > I'd like to move to 2.7.x for functionality of more modern modules but > this PM is for an HPC and I'm already struggling to convince > my predecessors that Puppetizing the HPC is not going impact performance. > Is there possibly a specific version of 2.7.x or maybe even 3.0.x that may > address the performance issues I'm seeing ? > > Thanks > - Trey > -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/cWlBauzx5hwJ. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
