Jira (PUP-5013) Improve the code for profiling resource evaluation?

[Christopher Price (JIRA)]

Title: Message Title

Christopher Price created an issue   Puppet / PUP-5013 Improve the code for profiling resource evaluation? Issue Type: Improvement Assignee: Unassigned Created: 2015/08/10 4:55 AM Priority: Normal Reporter: Christopher Price When doing some experimental work around improving metrics in Puppet Server, I noticed that one of the more interesting metrics (about how long is spent evaluating each resource type during a catalog compilation) was not always showing up. After some poking through the code, I discovered this: https://github.com/puppetlabs/puppet/blob/2667053d0438d650c909158c79ce85cfaee8019a/lib/puppet/parser/compiler.rb#L385-L388 def evaluate_definitions exceptwrap do Puppet::Util::Profiler.profile("Evaluated definitions", [:compiler, :evaluate_definitions]) do !unevaluated_resources.each do |resource| Puppet::Util::Profiler.profile("Evaluated resource #{resource}", [:compiler, :evaluate_resource, resource]) do resource.evaluate end end.empty? end end end It turns out that this code seems to track metrics for resources that get into the catalog via some kind of classifier (PE NC or OSS ENC script), but if you use node blocks in the site.pp file directly, then the resources all get evaluated non-lazily sometime earlier in the compilation: https://github.com/puppetlabs/puppet/blob/2667053d0438d650c909158c79ce85cfaee8019a/lib/puppet/parser/compiler.rb#L226-L237 And thus, you don't get any resource evaluation metrics for those resources. My proposal to improve this would be to move that profiling code to this spot: https://github.com/puppetlabs/puppet/blob/2667053d0438d650c909158c79ce85cfaee8019a/lib/puppet/parser/resource.rb#L77-L86 where, I believe, it would happen every time a resource was evaluated, regardless of how it found its way into the catalog. The site.pp / node default approach seems useful for debugging, and a user might be really interested in looking at metrics for resource evaluation in that sort of use case. I've only looked into this for Puppet 3 so far, but if the general approach seems reasonable, I'll be happy to make sure it gets patched all the way up to Puppet 4/master (assuming that the compilation logic hasn't changed drastically in the other branches). Add Comment   This message was sent by Atlassian JIRA (v6.4.5#64020-sha1:78acd6c) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.