Thanks Doron. I'll try this way: since I can't overload the process that recieve data from the external source, I'll use it with a XSUB/XPUB proxy in the subscribers process; XPUB will be inproc with SendWatermark to zero so that the connected subscribers will receive all the messages. I can accept high memory usage on the subscriber side, but not on the publisher one.
Looks reasonable? Giacomo On Thu, Jun 27, 2013 at 11:46 AM, Doron Somech <[email protected]> wrote: > Hi Giacomo, > > I'm pretty sure subscribers cannot block the publisher, I'm not sure the > breakpoint is a good way to test that. > > Anyway regarding your other questions, NetMQ can handle slow subscribers, > you just need to set the SendWatermark to zero and NetMQ will queue all > messages, you should however know that this can cause to a memory problem. > > I don't think you need any queues at any side, ZeroMQ and NetMQ are > working with queues, when you send a message or receive a message you > actually queue and dequeue, there is another thread that actually do the > network stuff. So I think you should publish the message to NetMQ socket > directly from the thread that listen to the source and on the subscriber > side just consume the messages directly from the queue. > > Regarding the timeout, you can also do it with NetMQ using a poller or > Poll method on a Socket. > > > > > On Thu, Jun 27, 2013 at 11:45 AM, Giacomo Tesio <[email protected]> wrote: > >> Hi Doron! >> >> Thanks for your help. >> I have two (windows7) processes that communicate via netmq pub/sub (the >> publisher runs as x86, the subscriber as x64). While debugging (in visual >> studio), I've set a breakpoint just after the send() (in the publisher) and >> just after the Recieve (in the subscriber) and I noted that the publisher >> send the message but hang while the subscriber handle it. >> >> The publisher forwards data from another async source that is very fast >> and I must ensure that: >> 1) no message from the source is lost (thus I can't block the publisher >> process, because the async source requires very low latency and do not >> provide any cache: if no one is ready to read the message they send, the >> message is lost) >> 2) no message from the publisher is lost by the subscribers, even if >> they have different processing speeds. >> >> To ensure 2 I'm using an unbounded BlockingCollection<T> as a queue (that >> btw, enable me to TryTake messages with a timeout, that is useful to >> control the consuming thread when the publisher don't send nothing) >> >> In case of slow subscribers, does NetMQ alone (without the unbounded >> BlockingCollection) ensure that I won't neither loose any message nor block >> the publisher or other subscribers? >> >> >> Giacomo >> >> >> >> >> >> >> On Thu, Jun 27, 2013 at 9:53 AM, Doron Somech <[email protected]> wrote: >> >>> Hi Giacomo, >>> >>> I'm happy to hear you choose netmq, answers to your questions: >>> >>> 1. There are no functionality differences between NetMQ and >>> CLRZMQ/ZeroMQ, there are some small API differences between NetMQ and >>> CLRZMQ. >>> 2. Same as ZeroMQ, so publisher side (it's based on ZeroMQ 3.2.2) >>> 3. What to do you mean blocked? how did you noted that? I don't think >>> you should queue because NetMQ / ZeroMQ is already doing exactly that. Just >>> consume the messages from the socket. >>> >>> Doron >>> >>> >>> On Thu, Jun 27, 2013 at 1:04 AM, Giacomo Tesio <[email protected]> wrote: >>> >>>> Hello everybody, I'm working to my first ZeroMQ based real-world >>>> application and I have a few questions: >>>> >>>> 1) I'm working on .NET and I choosed NetMQ instead of CLRZMQ just on >>>> the base of github activity. I've noted there are some differences between >>>> the two API (for example I had to Subscribe("") instead of SubscribeAll()) >>>> but they seem minor differences at a first look. Is there any difference in >>>> functionality that I should be aware of? >>>> 2) I have to use a topic based PUB tcp socket: is PUB/SUB filtering >>>> done at publisher side? >>>> 3) I noted (in debug) that the publisher blocks while the subscribers >>>> handle each message: I've tried to minimize the impact of this with a queue >>>> of messages of subscriber side, with the subscriber that fills the queue >>>> and another thread consuming it. Is there a better approach? >>>> >>>> Thanks a lot! >>>> >>>> >>>> Giacomo >>>> >>>> _______________________________________________ >>>> zeromq-dev mailing list >>>> [email protected] >>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >>>> >>>> >>> >>> _______________________________________________ >>> zeromq-dev mailing list >>> [email protected] >>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >>> >>> >> >> _______________________________________________ >> zeromq-dev mailing list >> [email protected] >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >> >> > > _______________________________________________ > zeromq-dev mailing list > [email protected] > http://lists.zeromq.org/mailman/listinfo/zeromq-dev > >
_______________________________________________ zeromq-dev mailing list [email protected] http://lists.zeromq.org/mailman/listinfo/zeromq-dev
