Issue #9671 has been updated by Brice Figureau.
As Simon discovered, it looks like this code is the culprit:
<pre>
made.each do |res|
add_conditional_directed_dependency(made.find { |r| r != res && r.name ==
res.name[0,r.name.length]} || resource, res)
add_conditional_directed_dependency(res, sentinal, default_label)
end
</pre>
This code is quadratic in nature. It might be possible to rewrite it without
the inner loop. It's also possible to optimize
add_conditional_directed_dependency for this specific use case (there are some
operations unneeded).
This code is here only for the file type. This code assumes eval_generate
returns things in complete random orders, but we know that the file type orders
its children. It might be possible to take this into account to remove the
inner loop.
----------------------------------------
Bug #9671: transaction eval_generate slow on many files in recurse managed
directory
https://projects.puppetlabs.com/issues/9671
Author: Simon Effenberg
Status: Needs More Information
Priority: Normal
Assignee: Josh Cooper
Category: file
Target version: 2.7.x
Affected Puppet version: 2.7.3
Keywords: transaction file recurse
Branch:
Since 2.7 (jump from 2.6.2 to 2.7.3) a file resource for a directory with many
files in it is really really too slow.
<pre>
file { "/var/www/munin":
ensure => directory,
owner => munin,
group => munin,
mode => 644,
recurse => true,
require => [
Class['munin::master'],
]
}
</pre>
In the directory there are ~19000 files and in the
/usr/lib/ruby/1.8/transaction.rb in lines 171-174:
<pre>
made.each do |res|
add_conditional_directed_dependency(made.find { |r| r != res && r.name ==
res.name[0,r.name.length]} || resource, res)
add_conditional_directed_dependency(res, sentinal, default_label)
end
</pre>
the first add_conditional_directed_dependency needs a time between 0.2 and 1.5s
which is, let's say an avg (it is growing over the time) of 0.4s and this
*19000 files means, that the resource needs >2h... maybe this is not changeable
but i wants to know if i have to remove the recurse => true for this or if this
is a "bug"
--
You have received this notification because you have either subscribed to it,
or are involved in it.
To change your notification preferences, please click here:
http://projects.puppetlabs.com/my/account
--
You received this message because you are subscribed to the Google Groups
"Puppet Bugs" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/puppet-bugs?hl=en.
