Hi all,

As mentioned in an email yesterday, I'm working with someone to add a  
queueing service to Puppet so that some server-side operations are  
queued and executed as cpu time is available - generally those that  
aren't on the critical path but particularly the storeconfigs save  
operation.

The goal here is that the client wouldn't have to wait for the server  
to finish storing the catalog before it got its copy of the catalog;  
instead, the server would just put the catalog into the queue, and a  
queue reader would come behind and process as time was available.  The  
only other operation I can think of where this makes sense right now  
is reporting, but I expect we'll have others over time.

The big benefit here, of course, is that you get configuration storage  
out of the critical path for returning catalogs to clients, and this  
has the benefit of making the client queries shorter and thus less  
likely to overlap.  It also means that if you get a big rush of client  
connections, we can queue up the more expensive operations, and even  
if storeconfigs eventually becomes cheap, it would still make sense to  
queue anything that wasn't on the critical path.

So, I'm looking for input on what people think the right tools and  
architecture are for this, and just generally how we should go about  
it.  I don't want to have a six week discussion on it, but I do want  
to make sure we get as much feedback as possible during development,  
and we'll be doing normal code review and publishing code as often as  
possible.

This work is being funded by a client, but I'm trying to make the  
development process as open as it would normally be.  I haven't yet  
found the right balance there, but certainly pushing to more openness  
is the right choice.

Ethan Rowe, who is doing most of the queueing development, will be  
following up on this email with more detail on the architecture and  
tools he's picked.

-- 
Commit suicide. A hundred thousand lemmings can't be wrong.
---------------------------------------------------------------------
Luke Kanies | http://reductivelabs.com | http://madstop.com


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups 
"Puppet Developers" group.
To post to this group, send email to puppet-dev@googlegroups.com
To unsubscribe from this group, send email to 
puppet-dev+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/puppet-dev?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply via email to