On Tue, 2012-02-07 at 23:04 +0100, Martin Sustrik wrote: > Hi Staffan, > > > I'm taking a look at fixing https://zeromq.jira.com/browse/LIBZMQ-299, > > and I'm not quite sure how to tackle it. > > > > There are two problems concerning HWMs and (X)SUB as I see it: > > > > 1. Subscriptions are silently dropped if the SNDHWM is reached. I don't > > know if that's the intended behaviour. If it is, it should probably be > > more clearly documented that subscriptions might be dropped. > > It definitely doesn't look like good behaviour. We should decide what > the right behaviour is though :) > > > I know there was talk about making setsockopt(ZMQ_SUBSCRIBE, ...) > > blocking [1], which would solve the "problem" of new subscriptions being > > silently dropped. > > Agreed. However, imagine the case where there are multiple publishers > connected to a single subscriber. In such case we may run into the > problem of stuck publisher. Ie. if one of the publishers doesn't process > subscriptions the buffers will ultimately get full and the subscriber > won't be able to send more subscriptions/unsubscriptions even to other > well-behaving publishers!
Good point. > A solution in a short run may be to disconnect the publisher when it > refuses to process subscriptions (consider it either hung or malevolent). But how would you discriminate between stuck and just slow? I think that'll introduce problems of its own. > Solution in the long run is probably separation of "message > distribution" pattern (one source publishes to multiple destinations) > and "message aggregation" pattern (many sources publishing to a single > destination). > > > 2. If you have a SNDHWM set on a (X)SUB socket you're quite likely to > > get an assertion at xsub.cpp:218 whenever a pipe is attached or > > hiccuped. > > Ack. The cheap and dirty fix to this problem would be to automatically > set SNDHWM on xsub/sub sockets to infinity (in xsub_t constructor). Yep that's what we do. > > If loosing subscriptions upon reaching the SNDHWM is the expected > > behaviour then 2 is easily fixable by just removing the assertion. If > > not... then I'm not sure how to tackle it. > > > > Bypassing the HWM for subscription messages is one option, but has its > > own caveats. > > Right. It can cause process to run out of memory. Still better than the > assertion probably. > > > Keeping track of which subscriptions has been sent is another > > possibility. > > What's the idea here? If you for each pipe kept track of which subscriptions had been sent you could try to re-send unsent subscriptions later. It would solve the problem but it's probably tricky to implement, will cost you memory and so on. /S _______________________________________________ zeromq-dev mailing list [email protected] http://lists.zeromq.org/mailman/listinfo/zeromq-dev
