Issue #9671 has been updated by Josh Cooper.
We tried a few different configurations: 20000 files in one directory, 8000
files in one directory, and nested 20 directories each containing 20
directories each containing 20 files.
<table>
<tr>
<td/>
<td>2.6.2</td>
<td>2.7.6</td>
<td>now</td>
</tr>
<tr>
<td>20000</td>
<td>88.51 s</td>
<td>> 1 hr</td>
<td>109 s</td>
</tr>
<tr>
<td>8000</td>
<td>~25 s</td>
<td>106.1 s</td>
<td>~30 s</td>
</tr>
<tr>
<td>20/20/20</td>
<td>25.15 s</td>
<td>106.5 s</td>
<td>~30 s</td>
</tr>
</table>
----------------------------------------
Bug #9671: transaction eval_generate slow on many files in recurse managed
directory
https://projects.puppetlabs.com/issues/9671
Author: Simon Effenberg
Status: Merged - Pending Release
Priority: High
Assignee: Josh Cooper
Category: file
Target version: 2.7.7
Affected Puppet version: 2.7.0
Keywords: transaction file recurse
Branch:
Since 2.7 (jump from 2.6.2 to 2.7.3) a file resource for a directory with many
files in it is really really too slow.
<pre>
file { "/var/www/munin":
ensure => directory,
owner => munin,
group => munin,
mode => 644,
recurse => true,
require => [
Class['munin::master'],
]
}
</pre>
In the directory there are ~19000 files and in the
/usr/lib/ruby/1.8/transaction.rb in lines 171-174:
<pre>
made.each do |res|
add_conditional_directed_dependency(made.find { |r| r != res && r.name ==
res.name[0,r.name.length]} || resource, res)
add_conditional_directed_dependency(res, sentinal, default_label)
end
</pre>
the first add_conditional_directed_dependency needs a time between 0.2 and 1.5s
which is, let's say an avg (it is growing over the time) of 0.4s and this
*19000 files means, that the resource needs >2h... maybe this is not changeable
but i wants to know if i have to remove the recurse => true for this or if this
is a "bug"
--
You have received this notification because you have either subscribed to it,
or are involved in it.
To change your notification preferences, please click here:
http://projects.puppetlabs.com/my/account
--
You received this message because you are subscribed to the Google Groups
"Puppet Bugs" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/puppet-bugs?hl=en.
