On Mar 18, 2009, at 2:36 PM, Larry Ludwig wrote: > > > On Mar 18, 2009, at 2:01 PM, Paul Nasrat wrote: > >> >>> I'm working with someone to add a >>> queueing service to Puppet so that some server-side operations are >>> queued and executed as cpu time is available - generally those that >>> aren't on the critical path but particularly the storeconfigs save >>> operation. >> > > What will be the determination of when CPU time is available? A > configuration option, like what's in exim when load is above X only > queue? What's considered a 'busy' puppetmaster?
The queue processor will run either on a separate node or as a niced process on the master, so the answer should be relatively straightforward from there. >>> The goal here is that the client wouldn't have to wait for the >>> server >>> to finish storing the catalog before it got its copy of the catalog; >>> instead, the server would just put the catalog into the queue, and a >>> queue reader would come behind and process as time was available. >>> The >>> only other operation I can think of where this makes sense right now >>> is reporting, but I expect we'll have others over time. >> >> Do we have a summary of current performance and some perfomance tests >> so we can objectively measure improvements? If so getting them >> visibly >> graphing from hudson builds of that branch would be awesome. >> > > More specifically what's the most expensive CPU and/or memory > operations when a node connects and process? Is there a break down in > that process? Of that process what's needed immediately by the client > and what can wait? Compiling and storage are definitely the most expensive CPU-wise, and file serving is probably the most expensive memory wise. The compiled catalog is needed immediately by the client, but the catalog storage isn't needed on a particular timeframe. Fileserving, of course, is also needed immediately. > > In the my case queueing reporting wouldn't help me that much since > we use a centralized syslog to do reporting. Yeah, reporting is generally a pretty inexpensive operation and it's probably not worth queueing in most cases; I threw it in there just to make the point that this should be a general process. -- There are three social classes in America: upper middle class, middle class, and lower middle class. --Judith Martin --------------------------------------------------------------------- Luke Kanies | http://reductivelabs.com | http://madstop.com --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To post to this group, send email to puppet-dev@googlegroups.com To unsubscribe from this group, send email to puppet-dev+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-dev?hl=en -~----------~----~----~----~------~----~------~--~---
