Yes, it's a nice Unixy distribution pattern. The pull is coming from ZeroVM (no relation) which Rackspace is putting a lot of work into. Their VMs interconnect over pipes, so distributed pipes will let them create VM clusters.
On Thu, Apr 24, 2014 at 9:23 PM, Michel Pelletier <[email protected]> wrote: > Distributed pipes are an interesting idea. I started work on something > similar, but from the other direction, wrapping processes and capturing or > teeing their stdin/out and other metadata (memory, cpu usage) and sending > that up to a central broker. This 'nerve' process wrapper mimics the Python > Popen interface and is as transparent as I could make it. > > https://github.com/michelp/nerve > > Nerves talk to centers (N dealers to a router) and centers can be accessed > with a command line interface to query the state of the various processes > being managed by nerves. In a sense it's very similar to the popular > 'supervisor' package that is commonly used to watchdog processes, but can > manage nerves distributed across machines. > > I wanted to extend the notion so that a cluster of centers could exchange > state with each other (what procs/streams they are managing) and connect > their various nerve wrapped processes input and outputs together to make > distributed pipes. Distributed process management was my main goal, but > distributed pipes is a natural unixy place to go with that. I didn't get > much farther with the idea. The prototype is in Python and uses pyzmq and > gevent. > > -Michel > > > On Wed, Apr 16, 2014 at 11:35 AM, Pieter Hintjens <[email protected]> wrote: >> >> Hi all, >> >> I'm working on a distributed pipes service (in zeromq/zbroker), and >> wonder if anyone would like to try this in their work. >> >> The architecture is: >> >> * Client applications read/write from named pipes >> * These pipes are hosted on edge brokers, on same box as clients. >> * Clients talk to brokers over IPC (ironic, but so it is). >> * Brokers interconnect and chat over Zyre. >> >> I'm working out the semantics for N-to-N pipes and may go for a pubsub >> model, see https://github.com/zeromq/zbroker/issues/38. >> >> -Pieter >> _______________________________________________ >> zeromq-dev mailing list >> [email protected] >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev > > > > _______________________________________________ > zeromq-dev mailing list > [email protected] > http://lists.zeromq.org/mailman/listinfo/zeromq-dev > _______________________________________________ zeromq-dev mailing list [email protected] http://lists.zeromq.org/mailman/listinfo/zeromq-dev
