Thanks Luca. As mentioned in my reply to Justin, I’m kind of hoping for an “off the shelf” solution that is already integrated with 0mq pub/sub mechanism. And, while zgossip looks like something I could build on, it doesn’t quite solve the whole problem (*I think*).
Will definitely look closer if/when I end up building this myself (which I’m beginning to suspect is how things will turn out). Regards, Bill > [zeromq-dev] topic discovery using broker > > Luca Boccassi luca.boccassi at gmail.com > <mailto:zeromq-dev%40lists.zeromq.org?Subject=Re%3A%20%5Bzeromq-dev%5D%20topic%20discovery%20using%20broker&In-Reply-To=%3C1501680827.20723.3.camel%40gmail.com%3E> > Note that zyre needs multicast (in ipv6, broadcast in v4) only with > zbeacon mode, but with zgossip mode (ie: with a well-known and pre- > configured "hub") it uses just regular unicast. > > -- > Kind regards, > Luca Boccassi >> The problem of topic discovery keeps coming up, and one of the first >> approaches recommended is to use a broker. That makes sense from the point >> of view of simplicity, but it has several disadvantages in the areas of >> scalability and availabilty. >> >> An obvious (?) solution would be to use a “topic broker" to connect >> publishers and subscribers, but to have the actual messages flow directly >> from publisher to subscriber. There are several commercial solutions that >> use this approach (e.g., Wombat, LBM) and it seems to work reasonably well. >> >> I’m new to ZeroMQ and am curious if anyone in the community has taken this >> approach? Is there a simple way of “wiring up” ZeroMQ to make this work? >> Or would this require some custom development on top of core ZeroMQ >> functionality? >> >> To be clear, I’m not interested in solutions that are built on top of >> multicast — for a number of reasons that is simply not a practical approach >> in my environment. And while there are projects associated with ZeroMQ that >> seem to address topic discovery (i.e., Zyre, zbeacon), as far as I can tell >> they all (a) depend on multicast, and (b) deliver subscribe/unsubscribe >> requests over the same stream as actual data messages. >> >> I have a general idea how to build this, but it’s a fair bit of work, esp. >> when edge cases are taken into account. Plus, I don’t want to look like an >> idiot when someone says “Why didn’t you just …?”. >> >> Any tips, hints or suggestions would be much appreciated! Thanks! >> > > You may be interested in a project I've been working on for simple > service discovery: > > https://github.com/JustinAzoff/simpledisco > > I wrote it as an alternative to gossip to use with zyre for > discovering peers over the internet. The api is simple, just like > with gossip you do a > > zsimpledisco_publish(disco, key, value); > > Once a minute(currently hardcoded) the actor will request every active > key and value. There's no 'un publish' api on the server side, keys > just timeout after a minute if no one is publishing them anymore. > Though I suppose the client side could use a DELETE command... my zyre > use case didn't need that so I haven't added it. > > Redundancy is implemented by having clients connect to multiple > servers and merge the responses. The servers do not connect to each > other and are completely self contained. > > The design is extremely simple because there is almost no state that > can get out of sync. Network timeouts and server failures are a non > issue, the client will just ignore the down server and try to > reconnect. I think it's an expanded case of the "Lazy Pirate pattern" > from the guide. > > -- > - Justin > _______________________________________________ > zeromq-dev mailing list > zeromq-dev@lists.zeromq.org > https://lists.zeromq.org/mailman/listinfo/zeromq-dev
_______________________________________________ zeromq-dev mailing list zeromq-dev@lists.zeromq.org https://lists.zeromq.org/mailman/listinfo/zeromq-dev
