Hi Edwin, On Mon, Jul 9, 2012 at 11:24 PM, Edwin Amsler <[email protected]> wrote: > So here I am, publishing messages through ZeroMQ's send() function at about > 300MB/s, and my network's set to only send at 10MB/s. > > This is kind of a big problem because, while I don't care if the clients > loose data on their end, the server is either using its memory until it > crashes, or I'm setting its high water mark and loosing about 29 of every > thirty messages I produce because I don't know that ZeroMQ can't keep up. > > Ideally, when a HWM condition happened, send() would return false, then I'd > test EAGAIN so I could decide for myself whether I should drop the message, > or retry later. With that kind of functionality, I could throttle back my > producer algorithm so that I exactly meet the demand of ZeroMQ instead of > overwhelming/starving it out. > > I'm willing to do the work if this sort of addition makes sense to the rest > of the project. I'd rather contribute here instead of forking it off in some > forgotten repository. > > Can/should this be done? Is there someone out there willing to mentor me? >
The behavior is intentional for pub/sub sockets. If you'd have only one subscriber you could use push/pull. Push sockets block when reach high water mark, so are Req sockets. The pub/sub sockets can't reliably block in general case because there could be multiple subscribers, only one of which reaches high water mark. Partially this comes from implementation: when you do zmq_send() zeromq starts to push message to the pipe for each connected subscribers, if one of the pipes full, there is no way to rollback messages already put into pipes, if it would it's not clear whether single slow subscriber should stop the publisher. Also pgm sockets can't have backpressure AFAIU. You can try to implement the behavior as a socket option (it can't be default behavior), but you have to be aware of the problems above (And I don't know if core developers are willing to accept the patch). -- Paul _______________________________________________ zeromq-dev mailing list [email protected] http://lists.zeromq.org/mailman/listinfo/zeromq-dev
