Re: [zeromq-dev] Notify send()er that they've hit the high water mark

2012-07-10 Thread Paul Colomiets
Hi Edwin, On Mon, Jul 9, 2012 at 11:24 PM, Edwin Amsler edwinams...@thinkboxsoftware.com wrote: So here I am, publishing messages through ZeroMQ's send() function at about 300MB/s, and my network's set to only send at 10MB/s. This is kind of a big problem because, while I don't care if the

Re: [zeromq-dev] Notify send()er that they've hit the high water mark

2012-07-10 Thread Edwin Amsler
On 10/07/2012 1:59 AM, Paul Colomiets wrote: Hi Edwin, On Mon, Jul 9, 2012 at 11:24 PM, Edwin Amsler edwinams...@thinkboxsoftware.com wrote: So here I am, publishing messages through ZeroMQ's send() function at about 300MB/s, and my network's set to only send at 10MB/s. This is kind of a big

Re: [zeromq-dev] Notify send()er that they've hit the high water mark

2012-07-10 Thread Chuck Remes
On Jul 10, 2012, at 10:31 AM, Edwin Amsler wrote: On 10/07/2012 1:59 AM, Paul Colomiets wrote: Hi Edwin, The behavior is intentional for pub/sub sockets. If you'd have only one subscriber you could use push/pull. Push sockets block when reach high water mark, so are Req sockets. The

Re: [zeromq-dev] Notify send()er that they've hit the high water mark

2012-07-10 Thread Edwin Amsler
On 10/07/2012 11:00 AM, Chuck Remes wrote: On Jul 10, 2012, at 10:31 AM, Edwin Amsler wrote: On 10/07/2012 1:59 AM, Paul Colomiets wrote: Hi Edwin, The behavior is intentional for pub/sub sockets. If you'd have only one subscriber you could use push/pull. Push sockets block when reach high

[zeromq-dev] Notify send()er that they've hit the high water mark

2012-07-09 Thread Edwin Amsler
So here I am, publishing messages through ZeroMQ's send() function at about 300MB/s, and my network's set to only send at 10MB/s. This is kind of a big problem because, while I don't care if the clients loose data on their end, the server is either using its memory until it crashes, or I'm

Re: [zeromq-dev] Notify send()er that they've hit the high water mark

2012-07-09 Thread Ian Barber
On Mon, Jul 9, 2012 at 9:24 PM, Edwin Amsler edwinams...@thinkboxsoftware.com wrote: Ideally, when a HWM condition happened, send() would return false, then I'd test EAGAIN so I could decide for myself whether I should drop the message, or retry later. With that kind of functionality, I

Re: [zeromq-dev] Notify send()er that they've hit the high water mark

2012-07-09 Thread Edwin Amsler
Guh? Reeeaally? Through my testing the publisher never errored on send, so I didn't test EAGAIN. A large part of the project is in metaphorical pieces on the floor while I redesign something. When I get it back together, I'll test this. Can anyone else confirm that that's what happens even on

Re: [zeromq-dev] Notify send()er that they've hit the high water mark

2012-07-09 Thread Ian Barber
On Mon, Jul 9, 2012 at 9:29 PM, Ian Barber ian.bar...@gmail.com wrote: This is pretty much how it works with most socket types - if you use the ZMQ_NOBLOCK flag, you'll get the EAGAIN if you've hit HWM. Ian You can see all of the HWM responses here: http://api.zeromq.org/2-1:zmq-socket

Re: [zeromq-dev] Notify send()er that they've hit the high water mark

2012-07-09 Thread Michel Pelletier
Have you looked at the suicidal snail pattern? http://zguide.zeromq.org/page:all#Slow-Subscriber-Detection-Suicidal-Snail-Pattern -Michel On Mon, Jul 9, 2012 at 1:24 PM, Edwin Amsler edwinams...@thinkboxsoftware.com wrote: So here I am, publishing messages through ZeroMQ's send() function at

Re: [zeromq-dev] Notify send()er that they've hit the high water mark

2012-07-09 Thread Edwin Amsler
There doesn't seem to be anything about how it signals that the high water mark happened. Unless I didn't read it thoroughly enough. When a/ZMQ_PUB/socket enters an exceptional state due to having reached the high water mark for a/subscriber/, then any messages that would be sent to

Re: [zeromq-dev] Notify send()er that they've hit the high water mark

2012-07-09 Thread Chuck Remes
On Jul 9, 2012, at 3:46 PM, Edwin Amsler wrote: There doesn't seem to be anything about how it signals that the high water mark happened. Unless I didn't read it thoroughly enough. When a ZMQ_PUB socket enters an exceptional state due to having reached the high water mark for a

Re: [zeromq-dev] Notify send()er that they've hit the high water mark

2012-07-09 Thread Edwin Amsler
I prefer Edwin. I'll make a test case, then report back. On 09/07/2012 3:52 PM, Chuck Remes wrote: On Jul 9, 2012, at 3:46 PM, Edwin Amsler wrote: There doesn't seem to be anything about how it signals that the high water mark happened. Unless I didn't read it thoroughly enough. When

Re: [zeromq-dev] Notify send()er that they've hit the high water mark

2012-07-09 Thread Michel Pelletier
On Mon, Jul 9, 2012 at 1:44 PM, Edwin Amsler edwinams...@thinkboxsoftware.com wrote: No need. I don't even care about subscribers. It's all about feeding the sending state machine efficiently. The SSP is about never getting to the point where the PUB socket drops messages, because your

Re: [zeromq-dev] Notify send()er that they've hit the high water mark

2012-07-09 Thread Chuck Remes
On Jul 9, 2012, at 3:54 PM, Edwin Amsler wrote: I prefer Edwin. I'll make a test case, then report back. Noted for future posts. We look forward to hearing from you. cr ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org