Issue #18880 has been updated by Chris Spence. Status changed from Unreviewed to Needs More Information
I have tried to recreate the catalog compile process, but need more of your code for any investigation to be meaningful. Without your other haproxy, nagios and perhaps other defines/classes it's difficult to see where the time is. My suspicion is that it's the process of creating dependencies between defines and other resources/classes that's taking the time. One technique for this, and I don't know whether it will help in your case but it's my first suspicion about the root of the problem, is to declare those dependencies once in a class, so that each instance of a define doesn't have to repeat the work. Whether this is the case is impossible for me to say without seeing more Puppet code. ---------------------------------------- Bug #18880: compile with large number of defines is becoming very slow, ruby 1.9.3 is affected even more https://projects.puppetlabs.com/issues/18880#change-98248 * Author: konrad rzentarzewski * Status: Needs More Information * Priority: Normal * Assignee: * Category: compiler * Target version: * Affected Puppet version: * Keywords: * Branch: ---------------------------------------- today i've made some synthetic checks with our lb setup, which consists of 121 defines, each of them wraps around 4-7 another defines (nagios, haproxy, stunnel configs, firewall). results are catastrophic. even more, we started to use ruby 1.9.3, as 1.8.7 is becoming obsolete and it's adding a more lag. i've reduced define numbers and observed that compile time reduces significantly each time. additionally i've inlined code from each define that we source from our lb define, and it also speeds up compile a lot. i've compiled with: <pre> time /usr/local/rvm/rubies/ruby-1.9.3-p327/bin/ruby /usr/bin/puppet master --compile lb2.acme.corp </pre> and <pre> time /usr/bin/ruby /usr/bin/puppet master --compile lb2.acme.corp </pre> actual times are here: <https://github.com/asquelt/puppet-defines-wtf/blob/master/BENCHMARK> manifest is here: <https://github.com/asquelt/puppet-defines-wtf/blob/master/manifests/projects/dc1-lb.pp> catalogs are also in repo: <https://github.com/asquelt/puppet-defines-wtf> you can see that i've decorated everything with double colon to make sure this is not related to #17335 case is described in puppet-users but wasn't commented: <https://groups.google.com/forum/?fromgroups=#!topic/puppet-users/4V01Kw5cReA> we use puppetdb, but disabling it won't help (it strips 3-20 seconds depending on number of resources). cpu on test machine is 2x Intel(R) Xeon(R) CPU E5335 @ 2.00GHz 4 cores, testing with single process. -- You have received this notification because you have either subscribed to it, or are involved in it. To change your notification preferences, please click here: http://projects.puppetlabs.com/my/account -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/groups/opt_out.
