On Thursday, January 24, 2013 1:29:32 PM UTC-6, asq wrote: > > are we having problem with defines in puppet 3.0.1/3.0.2? > the more and more we use defines in our repo, the more lagish puppetmaster > becomes. > > today i've made some synthetic checks with our lb setup, which consists of > 121 defines, each of them wraps around 4 another defines (nagios, haproxy, > stunnel configs). > results are catastrophic. > > even more, we started to use ruby 1.9.3, as 1.8.7 is becoming obsolete and > it's adding a more lag. > > i've reduced define numbers and observed that compile time reduces > significantly each time. > additionally i've inlined code from each define that we source from our lb > define, and it also speeds up compile a lot. > > i've compiled with: > time /usr/local/rvm/rubies/ruby-1.9.3-p327/bin/ruby /usr/bin/puppet master > --compile lb2 > and > /usr/bin/ruby /usr/bin/puppet master --compile lb2 > > actual times are here: > https://github.com/asquelt/puppet-defines-wtf/blob/master/BENCHMARK > manifest is here: > https://github.com/asquelt/puppet-defines-wtf/blob/master/manifests/projects/dc1-lb.pp > catalogs are also in repo: https://github.com/asquelt/puppet-defines-wtf > > you can see that i've decorated everything with double colon to make sure > this is not related to #17335 > > is it just me? > do you have any workarounds? > should i file a bug? >
To be clear, it looks like you are talking about counts of defined type * instances*, not counts of distinct defined types. Is that right? I am inclined to suspect that the problem is not defined types in general, but rather the specific defined types you are using. Indeed, your data seem to support that inasmuch as inlining definitions doesn't cut your compilation times very much. I can think of several things might cause long compilation times: - evaluating many templates, especially complex ones - making many hiera lookups if your hierarchy definition uses interpolated non-global variables - repetition of anything requiring network access, such as syncing files with a version-control repository, accessing files on a remote file system, or performing name resolutions. - calling generate() a lot That's not an exhaustive list. Anyway, I think filing a bug now would be premature. It may be that there is a bona fide performance problem, but if so then I don't think it has been pinpointed yet. John -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-users?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
