Hello everybody, I am thinking about using an event bus in one of my projects. It should basically just broadcast all messages to all the machines connected and optionally carry some other data (key-value pairs, for resource discovery?). What I was thinking about is to run a daemon on every node and create a SUB and PULL socket as the endpoint for local processes to connect to the bus. This would effectively lead to sharding of those processes (services) across the whole clusted connected together by this bus. Now the question is what to use to connect those daemons. It would be nice to have some sort of reliability, e.g. every message will get published on all the nodes exatly once, maybe out of order. I have a few ideas how to do this. What do you think?
1) Steal ideas from ZRE, use ROUTER + DEALER (one for every peer). Somehow buffer the messages (in a database? files?) so that when a peer connects after some time, knowing the last message it received, it could send something like REPLAY_SINCE <message-id> and receive all the messages published since that id. Maybe selective NACK on top of it could also work to save some bandwidth. 2) Use regular PUB, still with the possibility of requesting missed messages. This may look a bit simpler, but I loose implicit buffering of messages for peers that are unavailable for just a short time. On top of it having an explicit connection for every peer allows some sort of authentication perhaps if necessary. So this does not look that viable… Then I have some more general question: 1) It looks fairly useful to me. How come there is no RFC for such a thing yet? ZRE is a bit similar, but not in the spirit. Is there perhaps some other technology that could be used instead of 0MQ for this? Or am I trying to implement something devious? :-) 2) If it looks useful for others, shall I try to write a RFC for a general-purpose distributed event bus? Any ideas on what it should incorporate? Thanks a lot for any comments, ideas and suggestions! Cheers, Ondra Kupka _______________________________________________ zeromq-dev mailing list [email protected] http://lists.zeromq.org/mailman/listinfo/zeromq-dev
