Yes, events != state. In fact Zyre is already a distributed event bus that can scale to about 100 peers (from experience) on WiFi, and a bit more than that on a LAN. Persistence and consistency are quite another story.
-Pieter On Tue, Feb 19, 2013 at 10:03 PM, Ondrej Kupka <[email protected]> wrote: > > On Feb 19, 2013, at 8:08 PM, Ondrej Kupka wrote: > >> >> On Feb 19, 2013, at 7:43 PM, Pieter Hintjens wrote: >> >>> On Tue, Feb 19, 2013 at 6:40 PM, Ondrej Kupka <[email protected]> wrote: >>> >>>> ... >>> >>> Basically you want to layer statefulness above ZRE, right? This is >>> something I wanted to address but haven't had time for yet. >>> >> >> My concern was more in the direction of a reliable event bus. An exemplar >> scenario could be to scatter a bunch of FSMs around the cluster and let them >> process the events inserted for example by a bunch of processes providing a >> RESTful entry point into the whole system. If you miss an event, it kinda >> breaks your FSM's internal state. That is why I was thinking about using >> this architecture of a network <-> daemon <-> PUB+PULL on every node. The >> daemon would make sure it got all the events from other nodes it knows and >> none got lost. Then there are other questions like what to do once you are >> partitioned for a long time (restart the FSMs?), but that is >> application-dependent and not that important in this discussion I guess... >> >> Then other things could be implemented on top, like what you mention, but I >> haven't though about that much yet. >> > > Yeah, after discussing a bit with Martin Sustrik, I discovered that I am > probably going a bit overboard with this and it is not really possible to do > in any case. Right now I am being enlightened by reading Martin's blog posts > :-) > > Thanks for help! > > Ondra > >>> The hardest part is ensuring consistency if you have multiple updates >>> at the same time, to the same data. If you don't care about this, just >>> hold a key=value table in each node, and when a new node joins, it can >>> ask a random node for a state dump, much like the simple Clone >>> versions. Easy to write on top of ZRE as an application. >>> >>> If you want global consistency you have to first design a small >>> protocol for electing a leader. Perhaps the oldest node. Then when a >>> new node joins, it first discovers the leader, and then it asks the >>> leader for a snapshot. Then, every node when it has a new key=value, >>> sends this to the leader, which rebroadcasts it to everyone else. >>> >>> In any case I'd build this on top of ZRE using the group pub-sub it >>> provides you. >>> >> >> Thanks, >> Ondra Kupka >> >>> -Pieter >>> _______________________________________________ >>> zeromq-dev mailing list >>> [email protected] >>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >> > > _______________________________________________ > zeromq-dev mailing list > [email protected] > http://lists.zeromq.org/mailman/listinfo/zeromq-dev _______________________________________________ zeromq-dev mailing list [email protected] http://lists.zeromq.org/mailman/listinfo/zeromq-dev
