On Nov 14, 2012, at 8:53 AM, Brice Figureau <[email protected]> wrote:
> Hi, > > On Wed, 2012-11-14 at 13:43 +0100, David Schmitt wrote: >> On Tue, 13 Nov 2012 11:08:57 -0800, Luke Kanies <[email protected]> >> wrote: >>>> The run with 2.7.19 takes around ten minutes wallclock time. 3.0.1 >> takes >>>> around nine minutes wallclock for a complete "puppet agent --test". >> The >>>> difference is approximately the time saved on master compiling the >>>> catalog. I'm not concerned about the two minutes spent actually >> managing >>>> 3000 resources (~0.04s per resource). I'm concerned about the other >> seven >>>> minutes overhead. >>> >>> Any idea what that other seven minutes is? >>> >>> That seems completely pathalogical. >> >> I'm still scratching at the surface. I've attached the complete >> --verbose >> --noop log from master and agent. >> >> Since I've got no comparison, perhaps one of you can point out >> something. > > Let's try to answer the question. > > Looking at the log: > 07:43:28 agent starts > 07:43:30 agent has finished pluginsync > 07:43:33 master starts catalog compilation > 07:44:48 master has finished compilation (it took 75.51 seconds which is > coherent) > 07:49:28 agent starts applying the catalog > 07:51:00 agent finishes the catalog run (it took 125.23 seconds) > > Between those two traces, we have a time span of about 7 minutes 30s. > > So everything looks normal: > compilation 75.51 > catalog run 125.23 > total 3 min 34 s > > Then we have the almost 4 minutes hole between compilation is finished > and the agent starting to apply the catalog. > > This is where the agent downloads the catalog, unserialize it to ruby > objects from pson, serialize it again to pson, and stores it to disk. > > As I (and Peter) said numerous time, this part of the agent lifecycle is > one of the most time consuming. It should now be much better than when > it was serialized in yaml (can you check that your catalog are > serialized as pson on the agent?), but still we're doing an operation > that has no use just because the internal wiring doesn't allow us to > directly write what the agent receives from the catalog. > > BTW, if you put the agent in debug mode, it should show how long it took > to serialize/dump the cached catalog. This way we could compare this > time with the total time and see if the time is spent on serialization, > unserialization, download, etc… If you're using JSON instead of YAML, then you shouldn't see a lot of time in the converting to and from. I can't remember if this is a configurable option in 3.0, or if it defaults to JSON, but if it still defaults to YAML, that could be a huge win very quickly. -- Luke Kanies | http://about.me/lak | http://puppetlabs.com/ | +1-615-594-8199 -- You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/puppet-dev?hl=en.
