Re: [Puppet Users] Puppet 2.6.8 agent slow execution
On Jun 1, 2011, at 1:30 PM, Jacob Helwig wrote: > On Wed, 01 Jun 2011 23:58:24 +0400, Pavel Shevaev wrote: >> >>> Mitigation for 4: >>> 1) If the combined size of the files are small, you can try using "content" >>> instead of "source" for the files. >> >> Wow, that did the trick Thanks a lot! Deployment time is now 10-12 >> seconds. I'm a puppet newbie and don't quite understand why it worked >> but it's great :) >> > > So, the big difference there is that by using 'content', the file > contents are shipped down to the agent as part of the catalog. > > With 'source', the agent will have to make a new HTTP request per file > to get the content when it attempts to manage the resource. > > You end up saving a potentially large number of HTTP round-trips, which > adds up. So, to expand on this, the reason why the files need to be "small" is because all the files's contents get sent every run, instead of the normal puppet module that just sends the checksum and asks for the file if the checksum doesn't match. (As a trivial example, the file's checksum won't match if the file is empty or doesn't exist.) -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Puppet 2.6.8 agent slow execution
> You end up saving a potentially large number of HTTP round-trips, which > adds up. Thanks for explanations -- Best regards, Pavel -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Puppet 2.6.8 agent slow execution
On Wed, 01 Jun 2011 23:58:24 +0400, Pavel Shevaev wrote: > > > Mitigation for 4: > > 1) If the combined size of the files are small, you can try using "content" > > instead of "source" for the files. > > Wow, that did the trick Thanks a lot! Deployment time is now 10-12 > seconds. I'm a puppet newbie and don't quite understand why it worked > but it's great :) > So, the big difference there is that by using 'content', the file contents are shipped down to the agent as part of the catalog. With 'source', the agent will have to make a new HTTP request per file to get the content when it attempts to manage the resource. You end up saving a potentially large number of HTTP round-trips, which adds up. -- Jacob Helwig signature.asc Description: Digital signature
Re: [Puppet Users] Puppet 2.6.8 agent slow execution
> Mitigation for 4: > 1) If the combined size of the files are small, you can try using "content" > instead of "source" for the files. Wow, that did the trick Thanks a lot! Deployment time is now 10-12 seconds. I'm a puppet newbie and don't quite understand why it worked but it's great :) -- Best regards, Pavel -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Puppet 2.6.8 agent slow execution
On Jun 1, 2011, at 7:04 AM, pachanga wrote: > Hi! > > I'm incrementally developing puppet rules on the master server and > applying them to the testing machine. For some reason puppet agent > applies changes from the master very slowly, about 3 minutes every > time. At this speed it's very tiresome to test the new rules... > > I'm running puppet agent as follows: > > #puppet agent --no-daemonize --verbose --onetime --summarize --debug -- > trace > > Here is what it shows in the end: > > Changes: > Events: > Resources: >Total: 46 > Time: > Filebucket: 0.00 > Exec: 0.00 > Schedule: 0.00 > Ssh authorized key: 0.00 >Group: 0.00 > Package: 0.00 > User: 0.01 > Service: 0.07 > Config retrieval: 12.25 > Last run: 1306935632 > File: 84.25 >Total: 96.58 > > During the run most of the time is spent in the following phases: > > > info: Applying configuration version '1306935546' > debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw > yaml; using pson > debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw > yaml; using pson > debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw > yaml; using pson > debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw > yaml; using pson > debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw > yaml; using pson > debug: Service[sshd](provider=redhat): Executing '/sbin/service sshd > status' > debug: Puppet::Type::Service::ProviderRedhat: Executing '/sbin/ > chkconfig sshd' > debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw > yaml; using pson > debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw > yaml; using pson > debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw > yaml; using pson > debug: Finishing transaction 23816162721700 > > Any ideas why this can be happening? Thanks. This probably means you are doing one of these (all of which are slow): 1) Using "File" for big file(s) 2) Using "File" with "recurse=true" to deploy lots of little files. 3) Using "File" with recurse=true" to deploy into a directory with lots of little files, even though you are only deploying very few. ( purge=fasle ) 4) Using "File" over a connection with high ping. Of course, combining more than one is even worse. Mitigations for 1 or 2: 1) Put your files in a package and let your package manager of choice handle it for you. 2) Use rsync with a cron job or rsync with an exec Mitigation for 3: 1) Try "recurse=remote". Mitigation for 4: 1) If the combined size of the files are small, you can try using "content" instead of "source" for the files. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
[Puppet Users] Puppet 2.6.8 agent slow execution
Hi! I'm incrementally developing puppet rules on the master server and applying them to the testing machine. For some reason puppet agent applies changes from the master very slowly, about 3 minutes every time. At this speed it's very tiresome to test the new rules... I'm running puppet agent as follows: #puppet agent --no-daemonize --verbose --onetime --summarize --debug -- trace Here is what it shows in the end: Changes: Events: Resources: Total: 46 Time: Filebucket: 0.00 Exec: 0.00 Schedule: 0.00 Ssh authorized key: 0.00 Group: 0.00 Package: 0.00 User: 0.01 Service: 0.07 Config retrieval: 12.25 Last run: 1306935632 File: 84.25 Total: 96.58 During the run most of the time is spent in the following phases: info: Applying configuration version '1306935546' debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw yaml; using pson debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw yaml; using pson debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw yaml; using pson debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw yaml; using pson debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw yaml; using pson debug: Service[sshd](provider=redhat): Executing '/sbin/service sshd status' debug: Puppet::Type::Service::ProviderRedhat: Executing '/sbin/ chkconfig sshd' debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw yaml; using pson debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw yaml; using pson debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw yaml; using pson debug: Finishing transaction 23816162721700 Any ideas why this can be happening? Thanks. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
