Jira (PUP-2924) Puppet searches disk for whit classes

[Charlie Sharpsteen (JIRA)]

Title: Message Title

Charlie Sharpsteen commented on an issue   Re: Puppet searches disk for whit classes Thanks for reporting this! There is definitely some wasted effort going on here. The core of the problem is that we have a very expensive lookup operation that determines the Type associated with a resource: Puppet::Resource#resource_type. Due to the expense, some methods set the resource_type if this information is known and available when a Resource instance is created. However, when retrieving the state of whits, a Resource instance is created that does not receive available type information: debug call: load /puppetlabs/puppet/lib/puppet/util/autoload.rb:194 name = :completed_stage env = production /puppetlabs/puppet/lib/puppet/util/autoload.rb:194:in `load' /puppetlabs/puppet/lib/puppet/metatype/manager.rb:163:in `type' /puppetlabs/puppet/lib/puppet/resource.rb:242:in `resource_type' /puppetlabs/puppet/lib/puppet/resource.rb:163:in `builtin_type?' /puppetlabs/puppet/lib/puppet/resource.rb:475:in `namevar' /puppetlabs/puppet/lib/puppet/resource.rb:466:in `parameter_name' /puppetlabs/puppet/lib/puppet/resource.rb:140:in `[]=' /puppetlabs/puppet/lib/puppet/type.rb:1032:in `retrieve' /puppetlabs/puppet/lib/puppet/type.rb:1063:in `retrieve_resource' /puppetlabs/puppet/lib/puppet/transaction/resource_harness.rb:223:in `from_resource' /puppetlabs/puppet/lib/puppet/transaction/resource_harness.rb:19:in `evaluate' /puppetlabs/puppet/lib/puppet/transaction.rb:174:in `apply' /puppetlabs/puppet/lib/puppet/transaction.rb:187:in `eval_resource' /puppetlabs/puppet/lib/puppet/transaction.rb:117:in `call' /puppetlabs/puppet/lib/puppet/transaction.rb:117:in `block (2 levels) in evaluate' /puppetlabs/puppet/lib/puppet/util.rb:327:in `block in thinmark' /opt/rh/ruby193/root/usr/share/ruby/benchmark.rb:295:in `realtime' /puppetlabs/puppet/lib/puppet/util.rb:326:in `thinmark' /puppetlabs/puppet/lib/puppet/transaction.rb:117:in `block in evaluate' /puppetlabs/puppet/lib/puppet/graph/relationship_graph.rb:118:in `traverse' /puppetlabs/puppet/lib/puppet/transaction.rb:108:in `evaluate' /puppetlabs/puppet/lib/puppet/resource/catalog.rb:167:in `block in apply' /puppetlabs/puppet/lib/puppet/util/log.rb:149:in `with_destination' /puppetlabs/puppet/lib/puppet/transaction/report.rb:112:in `as_logging_destination' /puppetlabs/puppet/lib/puppet/resource/catalog.rb:166:in `apply' /puppetlabs/puppet/lib/puppet/configurer.rb:117:in `block in apply_catalog' /puppetlabs/puppet/lib/puppet/util.rb:161:in `block in benchmark' /opt/rh/ruby193/root/usr/share/ruby/benchmark.rb:295:in `realtime' /puppetlabs/puppet/lib/puppet/util.rb:160:in `benchmark' /puppetlabs/puppet/lib/puppet/configurer.rb:116:in `apply_catalog' /puppetlabs/puppet/lib/puppet/configurer.rb:191:in `run' /puppetlabs/puppet/lib/puppet/agent.rb:47:in `block (4 levels) in run' /puppetlabs/puppet/lib/puppet/agent/locker.rb:20:in `lock' /puppetlabs/puppet/lib/puppet/agent.rb:47:in `block (3 levels) in run' /puppetlabs/puppet/lib/puppet/agent.rb:117:in `with_client' /puppetlabs/puppet/lib/puppet/agent.rb:44:in `block (2 levels) in run' /puppetlabs/puppet/lib/puppet/agent.rb:82:in `run_in_fork' /puppetlabs/puppet/lib/puppet/agent.rb:43:in `block in run' /puppetlabs/puppet/lib/puppet/application.rb:179:in `call' /puppetlabs/puppet/lib/puppet/application.rb:179:in `controlled_run' /puppetlabs/puppet/lib/puppet/agent.rb:41:in `run' /puppetlabs/puppet/lib/puppet/application/agent.rb:355:in `onetime' /puppetlabs/puppet/lib/puppet/application/agent.rb:321:in `run_command' /puppetlabs/puppet/lib/puppet/application.rb:371:in `block (2 levels) in run' /puppetlabs/puppet/lib/puppet/application.rb:477:in `plugin_hook' /puppetlabs/puppet/lib/puppet/application.rb:371:in `block in run' /puppetlabs/puppet/lib/puppet/util.rb:479:in `exit_on_fail' /puppetlabs/puppet/lib/puppet/application.rb:371:in `run' /puppetlabs/puppet/lib/puppet/util/command_line.rb:137:in `run' /puppetlabs/puppet/lib/puppet/util/command_line.rb:91:in `execute' /puppetlabs/puppet/bin/puppet:28:in `<main>' Peter Bakkum: The following patch seems to solve the issue, if you would like to give it a try: type_retrieve.patch diff --git a/lib/puppet/type.rb b/lib/puppet/type.rb index f382f30..e7e2fee 100644 --- a/lib/puppet/type.rb +++ b/lib/puppet/type.rb @@ -1027,6 +1027,7 @@ class Type fail "Provider #{provider.class.name} is not functional on this host" if self.provider.is_a?(Puppet::Provider) and ! provider.class.suitable? result = Puppet::Resource.new(type, title) + result.resource_type = self.class # Provide the name, so we know we'll always refer to a real thing result[:name] = self[:name] unless self[:name] == title Add Comment   Puppet / PUP-2924 Puppet searches disk for whit classes Our Puppet instance has been pretty slow lately so we traced a puppet agent run to see what it was doing. In the trace we see a bunch of lines like this: stat("/opt/local/lib/ruby/site_ruby/2.0.0/puppet/vendor/semantic/lib/puppet/type/completed_stage.rb", 0xFFFFFD7FFFDFE4A0) Err#2 ENOENT stat("/etc/puppet/modules/admin/lib/puppet/type/completed_stage.... This message was sent by Atlassian JIRA (v6.1.4#6159-sha1:44eaede) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.