I haven't used it, but ZeroVM looks pretty cool, process containers in general are getting a lot of press lately. I can see how a distributed pipe system would give them a nice simple tool on top to interconnect sandboxed processes in a controlled way. Is there some issue or proposal they are working on?
I put some thought into but didn't spike any code on an evolution of the nerve idea. I've been informally calling them zerodog and zerocat. zerodog is a process watchdog and zerocat is like netcat, but works with files, tcp streams, or 0mq sockets. A "pack" of zerodogs can monitor, buffer IO for, and trace processes, essentially becoming the brokers for wrapped processes. Zerocat is just like cat, but not only works with files but also 0mq sockets and can talk to zerodogs to find processes output streams to concatenate. The cat and the dog eat each others tails when zerodogs watch zerocats that in turn are redirecting output from processes watched by zerodogs.... There's an interesting pattern here, distributed management of not only processes but also the streams of data they produce. Food for thought. -Michel On Thu, Apr 24, 2014 at 1:10 PM, Pieter Hintjens <[email protected]> wrote: > Yes, it's a nice Unixy distribution pattern. The pull is coming from > ZeroVM (no relation) which Rackspace is putting a lot of work into. > Their VMs interconnect over pipes, so distributed pipes will let them > create VM clusters. > > > > On Thu, Apr 24, 2014 at 9:23 PM, Michel Pelletier > <[email protected]> wrote: > > Distributed pipes are an interesting idea. I started work on something > > similar, but from the other direction, wrapping processes and capturing > or > > teeing their stdin/out and other metadata (memory, cpu usage) and sending > > that up to a central broker. This 'nerve' process wrapper mimics the > Python > > Popen interface and is as transparent as I could make it. > > > > https://github.com/michelp/nerve > > > > Nerves talk to centers (N dealers to a router) and centers can be > accessed > > with a command line interface to query the state of the various processes > > being managed by nerves. In a sense it's very similar to the popular > > 'supervisor' package that is commonly used to watchdog processes, but can > > manage nerves distributed across machines. > > > > I wanted to extend the notion so that a cluster of centers could exchange > > state with each other (what procs/streams they are managing) and connect > > their various nerve wrapped processes input and outputs together to make > > distributed pipes. Distributed process management was my main goal, but > > distributed pipes is a natural unixy place to go with that. I didn't get > > much farther with the idea. The prototype is in Python and uses pyzmq > and > > gevent. > > > > -Michel > > > > > > On Wed, Apr 16, 2014 at 11:35 AM, Pieter Hintjens <[email protected]> wrote: > >> > >> Hi all, > >> > >> I'm working on a distributed pipes service (in zeromq/zbroker), and > >> wonder if anyone would like to try this in their work. > >> > >> The architecture is: > >> > >> * Client applications read/write from named pipes > >> * These pipes are hosted on edge brokers, on same box as clients. > >> * Clients talk to brokers over IPC (ironic, but so it is). > >> * Brokers interconnect and chat over Zyre. > >> > >> I'm working out the semantics for N-to-N pipes and may go for a pubsub > >> model, see https://github.com/zeromq/zbroker/issues/38. > >> > >> -Pieter > >> _______________________________________________ > >> zeromq-dev mailing list > >> [email protected] > >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev > > > > > > > > _______________________________________________ > > zeromq-dev mailing list > > [email protected] > > http://lists.zeromq.org/mailman/listinfo/zeromq-dev > > > _______________________________________________ > zeromq-dev mailing list > [email protected] > http://lists.zeromq.org/mailman/listinfo/zeromq-dev >
_______________________________________________ zeromq-dev mailing list [email protected] http://lists.zeromq.org/mailman/listinfo/zeromq-dev
