Hello Dave,
I wondered whether a better option might be to supply (within the
VM) a standard library including a message-passing interface, so
that you could simply send running jobs a query, and they'd get back
to you (when convenient) with the responses.
Both Josh and Jack Moffitt had mentioned something similar as well.
Jack had a "way forward" with that, but given that I'm not deeply
knowledgeable of all the various XMPP dials and levers out there, I
wasn't able to completely follow/understand/grock his architecture.
However, in short, Jack was saying something about jobs having JIDs
and being able to message them directly with things like "cancel",
"status", etc. (or for them the send messages on their own initiative
--- e.g. a way for Linked Process to not be so "master/slave" oriented).
--------------------------------------------------------------------------------------------------------------
AS A SIDENOTE THAT MAY BE RELATED: We initially had virtual machines
has XMPP clients with fully-qualified JIDs, but then there were
network flood issues with <presence/> stanzas as virtual machines came
online and offline. Moreover, it was not necessary for villeins to get
<presence/> information from virtual machines they didn't spawn --- if
you discovered someone elses VM cause of a <presence/> stanza, you
could interact with it and that could get crazy for the original
spawner if you messed up the state of their VM!!! (though, with Linked
Process, sharing vm_ids is cool because multiple villeins can interact
through a single VM --- utilize a VM as a "blackboard machine").
Because of <presence/> reasons, we relegated the farm to being the
only XMPP client and serving as a "stanza router" to the virtual
machines that it managed.
--------------------------------------------------------------------------------------------------------------
On the one hand, that allows you to build simple "What's the value
of this variable?" things quite easily, but it also allows you to
build arbitrary message processing things.
I've been very wary of complicating things because I want it to be
easy to "plug and play" any virtual machine (i.e. language
interpreters) into Linked Process. By requiring a virtual machine to
implement particular libraries makes it difficult for people to easily
add a new VM language to Linked Process.
Our current implementation, LoPSideD [ http://code.google.com/p/linkedprocess/
], uses Java 1.6's ScriptEngine as its generic interface to virtual
machines and it currently supports Groovy, Ruby, Python, and
JavaScript. And as you may know, there are numerous ScriptEngine's in
various languages [ https://scripting.dev.java.net/ (scroll down to
the HTML table and the "maintained elsewhere" section) ]. Thus,
implementing "message-passing" libraries in all these languages could
be a big pain in the butt. In the end, I hate code :).
However!!! --- given that virtual machines are general-purpose
computing engines, it is possible for a villein to submit_job an XMPP
library, Linked Process library, and a villein library in a virtual
machine. Think about that for a second. I have successfully done this
in Groovy, where a villein spawns a Groovy VM and then starts up a
villein in that VM (the groovy VM just uses the LoPSideD API). In this
way, its possible to do "arbitrary message processing things." (though
I felt it was a little contrived :)... In short, if the farm permits
socket connections, then you can run XMPP clients in a virtual machine
and do any other XMPP things you want... adding it to the Linked
Process spec may be cool, but again, I fear complexity/code... Do you
have any sexy/easy/nifty solutions? I would love to have a way for
virtual machines to do arbitrary message passing things...
Thanks for your thoughts Dave...
Take care,
Marko.
http://markorodriguez.com
