Issue #17828 has been reported by Per Cederqvist.
----------------------------------------
Bug #17828: Warn when eval_generate creates ridiculously many files
https://projects.puppetlabs.com/issues/17828
Author: Per Cederqvist
Status: Unreviewed
Priority: Normal
Assignee:
Category: performance
Target version:
Affected Puppet version: 3.0.1
Keywords:
Branch:
I had this innocent-looking file resource:
<pre>
file { "/var/lib/puppet/reports":
owner => 'puppet',
group => 'puppet',
recurse => true,
require => Package['puppetmaster'],
}
</pre>
Over time, more and more files accumulated. I now have over 36000 files, and it
takes over an hour to process them. Issue #17820 reduces the time to 20
minutes, but by adding recurselimit => 1 I still ensure that the directories
are writeable, and "puppet agent --test" now takes only 30 seconds. Quite an
improvement over the 4000+ seconds it took yesterday.
Tracking down the issue was hard. The enclosed patch prints a warning if more
than 1024 files are used, like this:
<pre>Warning: Considering 36870 files under File[/var/lib/puppet/reports]; you
may want to specify a recurselimit</pre>
It would have saved me a couple of days of running ruby-prof and adding debug
statements.
Running
<pre>chown -R puppet:puppet /var/lib/puppet/reports</pre>
takes 0.43 seconds. Using the file resource of Puppet to do the same thing in
Puppet 3.0.1 takes 4000+ seconds, even when all files have the proper owner. I
wonder if something besides issue #17820 could be done about that...
--
You have received this notification because you have either subscribed to it,
or are involved in it.
To change your notification preferences, please click here:
http://projects.puppetlabs.com/my/account
--
You received this message because you are subscribed to the Google Groups
"Puppet Bugs" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/puppet-bugs?hl=en.
