Issue #9671 has been updated by Nick Lewis. Status changed from Accepted to In Topic Branch Pending Review
Some improvements for this are available at [pull request 203](https://github.com/puppetlabs/puppet/pull/203). This branch contains several improvements to the transaction performance, primarily focused around eval_generate, and finding resources to evaluate. When eval_generating and creating dependencies between generated resources and their dependencies, we now ask each generated resource for its potential ancestors, and check a hash to see if we generated any of them. Previously we would do a linear search (including a string slice per element) for each resource. We now track the number of blockers for every resource, rather than recalculate them each time a dependency completes. This lets us quickly determine which resources ought to be added to the ready list. The ready list is now represented as a sorted set (implemented as a red-black tree), rather than sorting the entire list each time we need a resource. ---------------------------------------- Bug #9671: transaction eval_generate slow on many files in recurse managed directory https://projects.puppetlabs.com/issues/9671 Author: Simon Effenberg Status: In Topic Branch Pending Review Priority: High Assignee: Josh Cooper Category: file Target version: 2.7.x Affected Puppet version: 2.7.0 Keywords: transaction file recurse Branch: Since 2.7 (jump from 2.6.2 to 2.7.3) a file resource for a directory with many files in it is really really too slow. <pre> file { "/var/www/munin": ensure => directory, owner => munin, group => munin, mode => 644, recurse => true, require => [ Class['munin::master'], ] } </pre> In the directory there are ~19000 files and in the /usr/lib/ruby/1.8/transaction.rb in lines 171-174: <pre> made.each do |res| add_conditional_directed_dependency(made.find { |r| r != res && r.name == res.name[0,r.name.length]} || resource, res) add_conditional_directed_dependency(res, sentinal, default_label) end </pre> the first add_conditional_directed_dependency needs a time between 0.2 and 1.5s which is, let's say an avg (it is growing over the time) of 0.4s and this *19000 files means, that the resource needs >2h... maybe this is not changeable but i wants to know if i have to remove the recurse => true for this or if this is a "bug" -- You have received this notification because you have either subscribed to it, or are involved in it. To change your notification preferences, please click here: http://projects.puppetlabs.com/my/account -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/puppet-bugs?hl=en.
