Hi all, As mentioned in an email yesterday, I'm working with someone to add a queueing service to Puppet so that some server-side operations are queued and executed as cpu time is available - generally those that aren't on the critical path but particularly the storeconfigs save operation.
The goal here is that the client wouldn't have to wait for the server to finish storing the catalog before it got its copy of the catalog; instead, the server would just put the catalog into the queue, and a queue reader would come behind and process as time was available. The only other operation I can think of where this makes sense right now is reporting, but I expect we'll have others over time. The big benefit here, of course, is that you get configuration storage out of the critical path for returning catalogs to clients, and this has the benefit of making the client queries shorter and thus less likely to overlap. It also means that if you get a big rush of client connections, we can queue up the more expensive operations, and even if storeconfigs eventually becomes cheap, it would still make sense to queue anything that wasn't on the critical path. So, I'm looking for input on what people think the right tools and architecture are for this, and just generally how we should go about it. I don't want to have a six week discussion on it, but I do want to make sure we get as much feedback as possible during development, and we'll be doing normal code review and publishing code as often as possible. This work is being funded by a client, but I'm trying to make the development process as open as it would normally be. I haven't yet found the right balance there, but certainly pushing to more openness is the right choice. Ethan Rowe, who is doing most of the queueing development, will be following up on this email with more detail on the architecture and tools he's picked. -- Commit suicide. A hundred thousand lemmings can't be wrong. --------------------------------------------------------------------- Luke Kanies | http://reductivelabs.com | http://madstop.com --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To post to this group, send email to puppet-dev@googlegroups.com To unsubscribe from this group, send email to puppet-dev+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-dev?hl=en -~----------~----~----~----~------~----~------~--~---
