Hi all,
I've gotten my third and hopefully final prototype done, and it's what I tried
to do in the first one. The code is published in my
feature/master/async_compiler branch.
The client/server communication is unchanged, but on the server side, the
normal master queues catalog compile requests, and one or more compiler
processes pull those requests off the queue, compile the requested catalogs,
and stick the results in a different queue. As always, there are a couple of
caveats:
* As with previous versions, no tests, incomplete, etc.
* No error-handling here - if there's a problem, the client will
never know about it (nor will the master that the client is talking
to)
* A single pair (request/response) of queues is used for all processing
right now, but that will need to be changed to a per-client queue, or
there will need to be conflict checking in some way
* This doesn't yet support the flexibility to have each compiled catalog
sent along to a database or some such
* This is a completely different code base from the other two attempts,
and as such many of the things we learned there are missing here
That being said, it's a much better run than the first one - the client doesn't
need any modification at all, but the server can roughly scale forever, at
least when it comes to compiles.
Even better, if you wanted to implement this today, you could without a huge
amount of effort. That is, most of the work is in the new compiler application
and modifications to the existing queue terminus, and you could just add a new
terminus rather than modifying the existing one. You're not all the way there,
but you're close.
Note that this doesn't work for anything other than the compiler - no facts,
eports, etc.
The next step is to unify this series and the previous one, so the indirector
is using request/response internally, and then support generic connection
between the indirector and a bus, rather than catalog-specific connection.
This work has brought up a lot of other questions, most notably how to handle
connecting the caller with the desired termini, but Paul and I have talked
through some ideas and I'm hoping to get a chance to briefly prototype some of
that.
--
Be wary of the man who urges an action in which he himself incurs no
risk. -- Joaquin Setanti
---------------------------------------------------------------------
Luke Kanies -|- http://puppetlabs.com -|- +1(615)594-8199
--
You received this message because you are subscribed to the Google Groups
"Puppet Developers" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/puppet-dev?hl=en.
