Issue #17828 has been updated by Josh Cooper.
Status changed from Unreviewed to Needs More Information
Assignee set to Per Cederqvist
Hi Per, I'm not sure this is necessary, because you can use the evaltrace
option to see how long it takes puppet to evaluate each resource, even
resources that are eval_generated:
<pre>
puppet apply -e "file { '/tmp/foo': ensure => directory, recurse => true }"
--evaltrace --debug
...
Info: /File[/tmp/foo/file2107]: Starting to evaluate the resource
Info: /File[/tmp/foo/file2107]: Evaluated in 0.00 seconds
Info: /File[/tmp/foo/file3597]: Starting to evaluate the resource
Info: /File[/tmp/foo/file3597]: Evaluated in 0.00 seconds
</pre>
----------------------------------------
Bug #17828: Warn when eval_generate creates ridiculously many files
https://projects.puppetlabs.com/issues/17828#change-77619
Author: Per Cederqvist
Status: Needs More Information
Priority: Normal
Assignee: Per Cederqvist
Category: performance
Target version:
Affected Puppet version: 3.0.1
Keywords:
Branch:
I had this innocent-looking file resource:
<pre>
file { "/var/lib/puppet/reports":
owner => 'puppet',
group => 'puppet',
recurse => true,
require => Package['puppetmaster'],
}
</pre>
Over time, more and more files accumulated. I now have over 36000 files, and it
takes over an hour to process them. Issue #17820 reduces the time to 20
minutes, but by adding recurselimit => 1 I still ensure that the directories
are writeable, and "puppet agent --test" now takes only 30 seconds. Quite an
improvement over the 4000+ seconds it took yesterday.
Tracking down the issue was hard. The enclosed patch prints a warning if more
than 1024 files are used, like this:
<pre>Warning: Considering 36870 files under File[/var/lib/puppet/reports]; you
may want to specify a recurselimit</pre>
It would have saved me a couple of days of running ruby-prof and adding debug
statements.
Running
<pre>chown -R puppet:puppet /var/lib/puppet/reports</pre>
takes 0.43 seconds. Using the file resource of Puppet to do the same thing in
Puppet 3.0.1 takes 4000+ seconds, even when all files have the proper owner. I
wonder if something besides issue #17820 could be done about that...
--
You have received this notification because you have either subscribed to it,
or are involved in it.
To change your notification preferences, please click here:
http://projects.puppetlabs.com/my/account
--
You received this message because you are subscribed to the Google Groups
"Puppet Bugs" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/puppet-bugs?hl=en.
