Issue #8643 has been updated by Jeff Blaine.
<pre>
[root@rcf-cm-test ~]# time scp -q -r -p
rcf-cm-master:/etc/puppet/modules/rcf-tools/files/RedHatEnterpriseServer-5.6-x86_64-2.6/usr/local/rcf-tools
/tmp
real 0m53.795s
user 0m3.959s
sys 0m5.015s
[root@rcf-cm-test ~]#
</pre>
Showing the entirety of the 'rcf-tools' module/class:
<pre>
class rcf-tools {
file { '/usr/local/rcf-tools':
mode => 644, # dirs will end up as 755
recurse => true,
links => manage,
source => "puppet:///modules/rcf-tools/$systype/usr/local/rcf-tools",
backup => false,
}
}
</pre>
~14,000 files totalling 147MB, as scp'd above:
<pre>
master# ls files/
RedHatEnterpriseServer-5.6-x86_64-2.6/
master# ls files/RedHatEnterpriseServer-5.6-x86_64-2.6/
usr/
master# ls files/RedHatEnterpriseServer-5.6-x86_64-2.6/usr/local
rcf-tools/
master#
</pre>
The puppet run:
<pre>
...
debug: Service[ntpd](provider=redhat): Executing 'ps -ef'
debug: Service[ntpd](provider=redhat): PID is 14710
debug: Puppet::Type::Service::ProviderRedhat: Executing '/sbin/chkconfig ntpd'
... 2-3 minutes here ...
debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw yaml;
using pson
debug: /Stage[main]/Rcf-tools/File[/usr/local/rcf-tools]: Storing newly-audited
value for content
debug: /File[/usr/local/rcf-tools/bin]: Storing newly-audited value for content
debug: /File[/usr/local/rcf-tools/bin/passenger-make-enterprisey]: Storing
newly-audited value absent for type
debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw yaml;
using pson
notice: /File[/usr/local/rcf-tools/bin/passenger-make-enterprisey]/ensure:
defined content as '{md5}2f38353e50980e57f31e607c9fbe465b'
FIVE WALL-CLOCK MINUTES LATER:
debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw yaml;
using pson
debug: /File[/usr/local/rcf-tools/bin/passenger-memory-stats]: Storing
newly-audited value absent for type
debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw yaml;
using pson
notice: /File[/usr/local/rcf-tools/bin/passenger-memory-stats]/ensure: defined
content as '{md5}cd351c59a693fc96fb59b2d339a8b211'
SEVEN WALL-CLOCK MINUTES LATER:
debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw yaml;
using pson
debug: /File[/usr/local/rcf-tools/bin/passenger-status]: Storing newly-audited
value absent for type
debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw yaml;
using pson
notice: /File[/usr/local/rcf-tools/bin/passenger-status]/ensure: defined
content as '{md5}c8156f8d49d85fc22e8a659a6e6c9f5c'
CONTROL-FRICKIN-C
</pre>
----------------------------------------
Bug #8643: Tree of files = day+ to xfer (note in documentation)
https://projects.puppetlabs.com/issues/8643
Author: Jeff Blaine
Status: Unreviewed
Priority: Normal
Assignee:
Category:
Target version:
Affected Puppet version:
Keywords:
Branch:
I understand the CM conceptual stance on this from Puppetlabs (or so I think),
and I understand that "make a package" is the answer, but I wanted to report
this and suggest that the documentation for the 'file' resource is updated to
reflect reality so that people (like me) are able to accurately set
expectations for the product depending on their intended usage.
I've spent ~1-2 hours now with this, looking at strace output, etc.
scp -r -p host:/tree-of-14thousand_files_totalling_147MB takes 1 minute to
complete (between the same hosts used for the puppet run below, and tested
several times, even during the puppet run).
Puppet 2.6.4 takes a full day (or more, I cancelled it) to do the same with
that content, averaging 4-6 minutes per file for some unknown reason.
--
You have received this notification because you have either subscribed to it,
or are involved in it.
To change your notification preferences, please click here:
http://projects.puppetlabs.com/my/account
--
You received this message because you are subscribed to the Google Groups
"Puppet Bugs" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/puppet-bugs?hl=en.
