Nigel
Thanks a lot! Yes, I did find other options of checksum.
However, after I changed the checksum type to mtime for jetty jar,
Puppet now gave me error and can not recover itself:
info: /Stage[main]/Emb/File[/opt/jetty-distribution-7.4.2.v20110526.tar.gz]:
Filebucketed /opt/jetty-distribution-7.4.2.v20110526.tar.gz to puppet
with sum 3dc95fb231fdabbe161d8773e2a10457
err:
/Stage[main]/Emb/File[/opt/jetty-distribution-7.4.2.v20110526.tar.gz]/content:
change from {mtime}Thu Jul 07 10:44:29 -0700 2011 to
{md5}3dc95fb231fdabbe161d8773e2a10457 failed: Could not rename
temporary file /opt/jetty-distribution-7.4.2.v20110526.tar.gz.puppettmp_7191
to /opt/jetty-distribution-7.4.2.v20110526.tar.gz: File written to
disk did not match checksum; discarding changes ({mtime} vs {mtime}Mon
Jul 11 10:30:50 -0700 2011) at
/etc/puppet/modules/emb/manifests/init.pp:38 at
/etc/puppet/modules/emb/manifests/init.pp:38
Remove the file itself on the client did not help neither. How to
reset the checksum in this case?
Also, I am curious why a 2K small file also took 10s to md5 checksum:
info: /Stage[main]/Emb/File[/opt/jetty/contexts/emb-context.xml]:
Evaluated in 10.06 seconds
Thanks again
Haitao
On Mon, Jul 11, 2011 at 10:29 AM, Nigel Kersten <[email protected]> wrote:
>
>
> On Mon, Jul 11, 2011 at 10:08 AM, Haitao Jiang <[email protected]>
> wrote:
>>
>> Nigel
>>
>> Thanks for your reply. I used --evaltrace suggested by Luke and found
>> that all my time spent on File resources, these are jar files that are
>> MB in size.
>>
>> Further question:
>>
>> - If Puppet just check the file metadata before decided on whether to
>> download the file from master or not, why it took 10s to do that? I
>> did not have recursive directories nor checksum in my File resources;
>
> Because they're large files, the checksumming at both ends takes a
> reasonable amount of time.
>
>>
>> - Can we specify which file metadata to check, say, just the ctime?
>
> Absolutely:
> http://docs.puppetlabs.com/references/stable/type.html#file
>
> CHECKSUM
>
> The checksum type to use when checksumming a file.
>
> The default checksum parameter, if checksums are enabled, is md5. Valid
> values are md5, md5lite, mtime,ctime, none.
>
> depending upon your deployment, md5lite or mtime/ctime might be appropriate
> checksums.
>
>>
>> - What are the recommended way to improve the performance here? I only
>> have 3 nodes to manage so far.
>
> I generally recommend packaging large files like jars.
>
>>
>> Thanks again
>>
>> Haitao
>>
>> On Sat, Jul 9, 2011 at 9:28 AM, Nigel Kersten <[email protected]>
>> wrote:
>> >
>> >
>> > On Fri, Jul 8, 2011 at 3:00 PM, Haitao Jiang <[email protected]>
>> > wrote:
>> >>
>> >> I just wondering if anyone can point me to the documentation on the
>> >> metric section of a report. I have a slow agent run which has
>> >> following numbers:
>> >>
>> >> Config Retrieval 10.43 seconds
>> >> Exec 0.00 seconds
>> >> File 157.11 seconds
>> >> Filebucket 0.00 seconds
>> >> Package 0.07 seconds
>> >> Schedule 0.00 seconds
>> >> Total 167.62 seconds
>> >>
>> >> I would like to know meaning of above in order to find out the reasons
>> >> of slowness. My guess is it was due to the network latency, but just
>> >> want to make sure.
>> >
>> > It's telling you that it only took 10 seconds to talk to the server and
>> > retrieve the catalog, and almost all your time was spent in File
>> > resources.
>> > File resources impose a different kind of load upon the server, as when
>> > they
>> > have a remote source the client needs to request file metadata in order
>> > to
>> > compare locally and determine whether or not the file contents need to
>> > be
>> > retrieved from the server.
>> > If you have lots of File resources, multiple concurrent clients, and are
>> > still running with the webrick Puppet master, you'll see performance
>> > degradation like this.
>> > If you have deep recursive directories in a File resource, you'll run
>> > into
>> > similar issues.
>> > If you have very large files, the default checksumming will take a
>> > while.
>> > There are a few parameters that are designed to help with these cases.
>> > http://docs.puppetlabs.com/references/stable/type.html#file
>> > Have a look at recurse and checksum.
>> >
>> > --
>> > Nigel Kersten
>> > Product Manager, Puppet Labs
>> > Twitter: @nigelkersten
>> >
>> > Join us for PuppetConf <http://www.bit.ly/puppetconfsig>
>> > September 22nd and 23rd in Portland, Oregon, USA.
>> >
>> > --
>> > You received this message because you are subscribed to the Google
>> > Groups
>> > "Puppet Users" group.
>> > To post to this group, send email to [email protected].
>> > To unsubscribe from this group, send email to
>> > [email protected].
>> > For more options, visit this group at
>> > http://groups.google.com/group/puppet-users?hl=en.
>> >
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Puppet Users" group.
>> To post to this group, send email to [email protected].
>> To unsubscribe from this group, send email to
>> [email protected].
>> For more options, visit this group at
>> http://groups.google.com/group/puppet-users?hl=en.
>>
>
>
>
> --
> Nigel Kersten
> Product Manager, Puppet Labs
> Twitter: @nigelkersten
>
> Join us for PuppetConf <http://www.bit.ly/puppetconfsig>
> September 22nd and 23rd in Portland, Oregon, USA.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To post to this group, send email to [email protected].
> To unsubscribe from this group, send email to
> [email protected].
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
>
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
