Nick, I just tried this patch with 2.7.13 and I didn't notice a large difference in timing overall but I also didn't notice any adverse effects.
And, yes, I'm more than happy to try out new patches for optimization. Thanks, Trevor On Thu, May 10, 2012 at 2:44 PM, Nick Lewis <[email protected]> wrote: > On Thursday, May 10, 2012 at 10:30 AM, Ashley Penney wrote: > > On Thu, May 10, 2012 at 1:42 AM, Jeff Weiss <[email protected]> > wrote: > > > Fourth, Peter, Émile, and Trevor (or anyone else experiencing the problem), > would you be willing to be pre-release testers of improvements? Our ops team > is seeing the problem too, but that's only a single real-world data point. > We need to make sure we don't self-optimize. We need your help to make > sure the performance fixes address *your* problems not just ours. > > > I just wanted to chime in to offer to test any performance patches because > I'm definitely interested in helping get to the bottom of any performance > drops. We also see the massive memory use during run issues other people > have seen which has resulted in us stopping puppet runs on certain > over-commited boxes as it can tip them over the edge. Anything that targets > either of this areas is something I'll happily hack in by hand for testing > purposes. (I have 4 puppetmasters currently so it's easy to target just a > few and not the rest for comparisons). > > > On that note: After spending a few hours hunting yesterday, we identified > the cause of a significant performance regression in compilation when using > modules with metadata. I've attached a patch for anyone who would like to > test it out. We're seeing approximately 5-10x faster compilation. > > The basic gist of the issue is that every time Puppet looks up a class (via > include/import/autoload), it constructs a list of possible modules the class > could be in (based on its name). For each module, it will create a > Puppet::Module instance, so that it can ask which manifests in that module > may contain the class. Unfortunately, creating the Puppet::Module instance > will cause the metadata.json file to be read and parsed. Given that this > happens several times *per include*, it gets expensive quickly. > > Fortunately, as we already have a list of Puppet::Module objects for all the > modules, the simple solution is to just use one of those. > > Once again, a patch is attached. Please test this and provide feedback. Note > that this patch should not be applied to Puppet 2.6, or to 2.7.3 or earlier. > Due to a bug in caching, it will cause significantly *worse* performance. > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Developers" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/puppet-dev?hl=en. > > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Developers" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/puppet-dev?hl=en. -- Trevor Vaughan Vice President, Onyx Point, Inc (410) 541-6699 [email protected] -- This account not approved for unencrypted proprietary information -- -- You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/puppet-dev?hl=en.
