I don’t know. Perhaps someone else has an idea. On Jan 16, 2014, at 1:45 PM, Lindley French <[email protected]> wrote:
> > This is a common issue. If you can’t recover from dropped messages, PUB/SUB > > is not the correct pattern. > > In many cases, this is correct. I do not believe inproc is one of those > cases, however. With inproc, you should have full control of who is > subscribing and when they come up relative to the publisher. If your > subscribers aren't running when you expect them to be running, that's a bug. > If they aren't running fast enough, dropping messages *might* be a solution, > or it might not. I don't feel that's a decision that can be made in the > general case. > > Let me put it this way: If I need one-to-many semantics with backpressure and > filtering, what should I use? PUB is the only one-to-many socket type. I can > write my own filtering code, keep a vector of push sockets, etc. but that > seems to defeat the point of ZMQ patterns. PUB is exactly what I want in > every way *except* the HWM behavior. > > > On Thu, Jan 16, 2014 at 2:11 PM, Charles Remes <[email protected]> wrote: > This is a common issue. If you can’t recover from dropped messages, PUB/SUB > is not the correct pattern. > > This of PUB as a radio antenna. It broadcasts regardless of whether or not > anyone is listening. If there are no listeners, every packet gets dropped. If > a listener is slow, then packets will get dropped. If you need further > guarantees about delivery, then you need to build some kind of protocol (ack, > nak, ack window, etc) on top of DEALER/ROUTER. > > Also, as of libzmq 3.3, I believe the default HWM is 1000 (to prevent memory > exhaustion in the default configuration). If you want “infinite” then > setsockopt to -1. > > As for the dropped messages on inproc, you need to be careful to confirm that > a listener (SUB) is actually up, running and *connected* before you start > PUB’ing otherwise the PUB socket will drop messages. Synchronization for this > is discussed in the guide. Alternately, just have your PUB “sleep” for a > second after the SUB bind/connects and you should be okay. > > > On Jan 16, 2014, at 1:03 PM, Lindley French <[email protected]> wrote: > >> Well, aside from the router issue I do like the arrangement for easily >> handling different messages in different places. However, there may be a >> fatal flaw at the moment: PUB's desire to drop messages at the HWM. While >> making "drop" a default behavior for PUB is fine, I really don't like that >> it's the *only* behavior possible. >> >> Then again, that may or may not be the issue here. I haven't touched the >> HWM, so it should still be 0 which is theoretically infinite. Nonetheless, a >> bunch of my messages in a row vanished into the ether somewhere between PUB >> and SUB inproc sockets. >> >> >> On Thu, Jan 16, 2014 at 1:02 PM, Lindley French <[email protected]> wrote: >> I tend to stuff in as many different features as I can when I'm first >> learning something new, it helps me get a feel for it. >> >> You should have seen my first major python program..... >> >> >> On Thu, Jan 16, 2014 at 12:46 PM, Charles Remes <[email protected]> wrote: >> Create a socket for each worker thread and have your main thread resend the >> message down the appropriate socket. Sometimes it isn’t a good idea to try >> and shoe-horn every zeromq socket pattern into your app. :) >> >> On Jan 16, 2014, at 9:54 AM, Lindley French <[email protected]> wrote: >> >> > A problem I was wrestling with was, how do I deal with a TCP connection >> > where messages of different types may arrive, and may need to be dealt >> > with in different threads? The TCP socket can't be touched directly by >> > multiple threads, of course. The obvious solution was to immediately >> > forward messages arriving on the TCP socket to an inproc socket. >> > >> > I then took it one step further: why not make that inproc socket a PUB >> > socket and make the first part of each message be a topic identifier, so >> > that whichever thread knows how to deal with a particular message can just >> > subscribe to it and ignore the rest? >> > >> > That's a great design, right up until I try to do it with the TCP socket >> > being a ROUTER. Now, no matter what the first part of the sent message is, >> > the identity will end up being the first part on the receiving end. The >> > PUB/SUB won't work without some tweaking. >> > >> > I don't want to just drop the identity; that's useful information. I could >> > swap the first two parts; that will work, but it's unintuitive and could >> > cause confusion down the road. >> > >> > Any other ideas? >> > _______________________________________________ >> > zeromq-dev mailing list >> > [email protected] >> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev >> >> _______________________________________________ >> zeromq-dev mailing list >> [email protected] >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >> >> >> _______________________________________________ >> zeromq-dev mailing list >> [email protected] >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev > > > _______________________________________________ > zeromq-dev mailing list > [email protected] > http://lists.zeromq.org/mailman/listinfo/zeromq-dev > > > _______________________________________________ > zeromq-dev mailing list > [email protected] > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
_______________________________________________ zeromq-dev mailing list [email protected] http://lists.zeromq.org/mailman/listinfo/zeromq-dev
