[Puppet Users] puppetmaster 100%cpu usage on 2.6 (not on 0.24)

[Chris]

Hi

I recently upgraded my puppet masters (and clients) from 0.24.8 to
2.6.4

Previously, my most busy puppet master would hover around about 0.9
load  average, after the upgrade, its load hovers around 5

I am running passenger and mysql based stored configs.

Checking my running processes, ruby (puppetmasterd) shoots up to 99%
cpu load and stays there for a few seconds before dropping again.
Often there are 4 of these running simultaneously, pegging each core
at 99% cpu.

It seems that there has been a serious performance regression between
0.24 and 2.6 for my configuration

I hop the following can help work out where...

I ran puppetmasterd through a profiler to find the root cause of this
(http://boojum.homelinux.org/profile.svg).  The main problem appears
to be in /usr/lib/ruby/site_ruby/1.8/puppet/parser/ast/resource.rb, in
the evaluate function.

I added a few timing commands around various sections of that function
to find the following breakdown of times spent inside it, and the two
most intensive calls are
---
    paramobjects = parameters.collect { |param|
      param.safeevaluate(scope)
    }
---

and
---
    resource_titles.flatten.collect { |resource_title|
      exceptwrap :type => Puppet::ParseError do
        resource = Puppet::Parser::Resource.new(
          fully_qualified_type, resource_title,
          :parameters => paramobjects,
          :file => self.file,
          :line => self.line,
          :exported => self.exported,
          :virtual => virt,
          :source => scope.source,
          :scope => scope,
          :strict => true
        )

        if resource.resource_type.is_a? Puppet::Resource::Type
          resource.resource_type.instantiate_resource(scope, resource)
        end
        scope.compiler.add_resource(scope, resource)
        scope.compiler.evaluate_classes([resource_title],scope,false)
if fully_qualified_type == 'class'
        resource
      end
    }.reject { |resource| resource.nil? }
---


Unfortunately, that is about the limit of my current ruby skills.
What else can be looked at to speed 2.6 back up to the performance of
0.24?



-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.