[zeromq-dev] ROUTER/PULL
Inspired by the PUSH/ROUTER question a moment ago, I wonder if it ought to be possible to match ROUTER and PULL, for one-way directed communication? Currently I am using ROUTER-ROUTER for this, and the receiver just ignores the envelope. Being able to make the receiver PULL seems like it would be more natural. Justin ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] ROUTER/PULL
No, do not do this. If it works at all right now, it will break in a future library release when the wire protocol enforces peer socket type checking. ROUTER - ROUTER is fine as is ROUTER - DEALER. Alternately, do PUB - SUB. As of libzmq 3.2, the publisher filters out the outgoing messages so only those SUBs that have subscribed to the message will receive it. This will require you to add a subscription string as the first message part for all outgoing messages so the socket can filter it. Please read the guide if this doesn't make sense to you yet. There are lots of great examples with code. cr On Nov 13, 2012, at 12:00 PM, Justin Karneges jus...@affinix.com wrote: Inspired by the PUSH/ROUTER question a moment ago, I wonder if it ought to be possible to match ROUTER and PULL, for one-way directed communication? Currently I am using ROUTER-ROUTER for this, and the receiver just ignores the envelope. Being able to make the receiver PULL seems like it would be more natural. Justin ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] ROUTER/PULL
Why not make it allowed in a future version? It doesn't seem any more unusual than other mixings. On Tuesday, November 13, 2012 12:32:33 PM Chuck Remes wrote: No, do not do this. If it works at all right now, it will break in a future library release when the wire protocol enforces peer socket type checking. ROUTER - ROUTER is fine as is ROUTER - DEALER. Alternately, do PUB - SUB. As of libzmq 3.2, the publisher filters out the outgoing messages so only those SUBs that have subscribed to the message will receive it. This will require you to add a subscription string as the first message part for all outgoing messages so the socket can filter it. Please read the guide if this doesn't make sense to you yet. There are lots of great examples with code. cr On Nov 13, 2012, at 12:00 PM, Justin Karneges jus...@affinix.com wrote: Inspired by the PUSH/ROUTER question a moment ago, I wonder if it ought to be possible to match ROUTER and PULL, for one-way directed communication? Currently I am using ROUTER-ROUTER for this, and the receiver just ignores the envelope. Being able to make the receiver PULL seems like it would be more natural. Justin ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] ROUTER/PULL
There are reasons to not mix ROUTER and PULL, mainly that they are from different patterns, so if we decide to improve PUSH/PULL as a package, we would be unable, if people were mixing them with other patterns. As an example, see how we improved PUB/SUB to do pub-side filtering. DEALER does just the same as PUSH + PULL, so ROUTER/DEALER gives you what you want, and is better too, since you can do things like send heartbeats in both directions. -Pieter On Wed, Nov 14, 2012 at 4:02 AM, Justin Karneges jus...@affinix.com wrote: Why not make it allowed in a future version? It doesn't seem any more unusual than other mixings. On Tuesday, November 13, 2012 12:32:33 PM Chuck Remes wrote: No, do not do this. If it works at all right now, it will break in a future library release when the wire protocol enforces peer socket type checking. ROUTER - ROUTER is fine as is ROUTER - DEALER. Alternately, do PUB - SUB. As of libzmq 3.2, the publisher filters out the outgoing messages so only those SUBs that have subscribed to the message will receive it. This will require you to add a subscription string as the first message part for all outgoing messages so the socket can filter it. Please read the guide if this doesn't make sense to you yet. There are lots of great examples with code. cr On Nov 13, 2012, at 12:00 PM, Justin Karneges jus...@affinix.com wrote: Inspired by the PUSH/ROUTER question a moment ago, I wonder if it ought to be possible to match ROUTER and PULL, for one-way directed communication? Currently I am using ROUTER-ROUTER for this, and the receiver just ignores the envelope. Being able to make the receiver PULL seems like it would be more natural. Justin ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] ROUTER/PULL
Then how do you justify, for example, the ability to mix REQ and ROUTER? Is that just an exception to the rule? In my opinion, from a design point of view, zmq should either have very strict patterns (e.g. no more REQ+ROUTER), or it should allow mixing where it makes sense (e.g. ROUTER+PULL). Doesn't this seem reasonable? Right now the allowed mixing seems kind of arbitrary to me. On Wednesday, November 14, 2012 07:28:47 AM Pieter Hintjens wrote: There are reasons to not mix ROUTER and PULL, mainly that they are from different patterns, so if we decide to improve PUSH/PULL as a package, we would be unable, if people were mixing them with other patterns. As an example, see how we improved PUB/SUB to do pub-side filtering. DEALER does just the same as PUSH + PULL, so ROUTER/DEALER gives you what you want, and is better too, since you can do things like send heartbeats in both directions. -Pieter On Wed, Nov 14, 2012 at 4:02 AM, Justin Karneges jus...@affinix.com wrote: Why not make it allowed in a future version? It doesn't seem any more unusual than other mixings. On Tuesday, November 13, 2012 12:32:33 PM Chuck Remes wrote: No, do not do this. If it works at all right now, it will break in a future library release when the wire protocol enforces peer socket type checking. ROUTER - ROUTER is fine as is ROUTER - DEALER. Alternately, do PUB - SUB. As of libzmq 3.2, the publisher filters out the outgoing messages so only those SUBs that have subscribed to the message will receive it. This will require you to add a subscription string as the first message part for all outgoing messages so the socket can filter it. Please read the guide if this doesn't make sense to you yet. There are lots of great examples with code. cr On Nov 13, 2012, at 12:00 PM, Justin Karneges jus...@affinix.com wrote: Inspired by the PUSH/ROUTER question a moment ago, I wonder if it ought to be possible to match ROUTER and PULL, for one-way directed communication? Currently I am using ROUTER-ROUTER for this, and the receiver just ignores the envelope. Being able to make the receiver PULL seems like it would be more natural. Justin ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] ROUTER/PULL
I'm surprised that the docs aren't more clear about this detail. Honestly, the guide will teach you this but first you must read it. :) The names imply the proper pairings. In what world does ROUTER / PULL make sense from a semantic standpoint? REQ is a subclass of DEALER. REP is a subclass of ROUTER. This is why we allow REQ / ROUTER and other pairings outside of the usual REQ / REP. The allowed pairings are: PUSH / PULL PUB / SUB PAIR / PAIR REQ / REP, REQ / ROUTER, DEALER / REP, DEALER / ROUTER, DEALER / DEALER and ROUTER / ROUTER cr On Nov 13, 2012, at 4:54 PM, Justin Karneges jus...@affinix.com wrote: Then how do you justify, for example, the ability to mix REQ and ROUTER? Is that just an exception to the rule? In my opinion, from a design point of view, zmq should either have very strict patterns (e.g. no more REQ+ROUTER), or it should allow mixing where it makes sense (e.g. ROUTER+PULL). Doesn't this seem reasonable? Right now the allowed mixing seems kind of arbitrary to me. On Wednesday, November 14, 2012 07:28:47 AM Pieter Hintjens wrote: There are reasons to not mix ROUTER and PULL, mainly that they are from different patterns, so if we decide to improve PUSH/PULL as a package, we would be unable, if people were mixing them with other patterns. As an example, see how we improved PUB/SUB to do pub-side filtering. DEALER does just the same as PUSH + PULL, so ROUTER/DEALER gives you what you want, and is better too, since you can do things like send heartbeats in both directions. -Pieter On Wed, Nov 14, 2012 at 4:02 AM, Justin Karneges jus...@affinix.com wrote: Why not make it allowed in a future version? It doesn't seem any more unusual than other mixings. On Tuesday, November 13, 2012 12:32:33 PM Chuck Remes wrote: No, do not do this. If it works at all right now, it will break in a future library release when the wire protocol enforces peer socket type checking. ROUTER - ROUTER is fine as is ROUTER - DEALER. Alternately, do PUB - SUB. As of libzmq 3.2, the publisher filters out the outgoing messages so only those SUBs that have subscribed to the message will receive it. This will require you to add a subscription string as the first message part for all outgoing messages so the socket can filter it. Please read the guide if this doesn't make sense to you yet. There are lots of great examples with code. cr On Nov 13, 2012, at 12:00 PM, Justin Karneges jus...@affinix.com wrote: Inspired by the PUSH/ROUTER question a moment ago, I wonder if it ought to be possible to match ROUTER and PULL, for one-way directed communication? Currently I am using ROUTER-ROUTER for this, and the receiver just ignores the envelope. Being able to make the receiver PULL seems like it would be more natural. Justin ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] ROUTER/PULL
Justin, The combinations are not arbitrary. There's an explanation here: http://zguide.zeromq.org/page:all#Request-Reply-Combinations. Firstly, you can replace REQ with DEALER, and REP with ROUTER, to turn a synchronous REQ-REP into a partially or fully asynchronous flow. Secondly, you can combine DEALER to DEALER and ROUTER to ROUTER to get two new patterns. The legal combinations are formally defined in the zmq_socket man page. The original design ideology, which was accurate IMO, was that each low-level pattern was a separate package. Thus, PUB+SUB, PUSH+PULL (pipeline), PAIR, REQ+REP+DEALER+ROUTER. We've been extending the protocol to announce each peer's socket type, to let us catch invalid combinations. The public spec in zmq_socket[3] defines what kind of future compatibility we promise. Since it states that ROUTER+REQ is valid, this will always work. But since it does not allow ROUTER+PUSH, we're free to break that in the future. Hope this helps to understand the reasoning here. -Pieter On Wed, Nov 14, 2012 at 7:54 AM, Justin Karneges jus...@affinix.com wrote: Then how do you justify, for example, the ability to mix REQ and ROUTER? Is that just an exception to the rule? In my opinion, from a design point of view, zmq should either have very strict patterns (e.g. no more REQ+ROUTER), or it should allow mixing where it makes sense (e.g. ROUTER+PULL). Doesn't this seem reasonable? Right now the allowed mixing seems kind of arbitrary to me. On Wednesday, November 14, 2012 07:28:47 AM Pieter Hintjens wrote: There are reasons to not mix ROUTER and PULL, mainly that they are from different patterns, so if we decide to improve PUSH/PULL as a package, we would be unable, if people were mixing them with other patterns. As an example, see how we improved PUB/SUB to do pub-side filtering. DEALER does just the same as PUSH + PULL, so ROUTER/DEALER gives you what you want, and is better too, since you can do things like send heartbeats in both directions. -Pieter On Wed, Nov 14, 2012 at 4:02 AM, Justin Karneges jus...@affinix.com wrote: Why not make it allowed in a future version? It doesn't seem any more unusual than other mixings. On Tuesday, November 13, 2012 12:32:33 PM Chuck Remes wrote: No, do not do this. If it works at all right now, it will break in a future library release when the wire protocol enforces peer socket type checking. ROUTER - ROUTER is fine as is ROUTER - DEALER. Alternately, do PUB - SUB. As of libzmq 3.2, the publisher filters out the outgoing messages so only those SUBs that have subscribed to the message will receive it. This will require you to add a subscription string as the first message part for all outgoing messages so the socket can filter it. Please read the guide if this doesn't make sense to you yet. There are lots of great examples with code. cr On Nov 13, 2012, at 12:00 PM, Justin Karneges jus...@affinix.com wrote: Inspired by the PUSH/ROUTER question a moment ago, I wonder if it ought to be possible to match ROUTER and PULL, for one-way directed communication? Currently I am using ROUTER-ROUTER for this, and the receiver just ignores the envelope. Being able to make the receiver PULL seems like it would be more natural. Justin ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] ROUTER/PULL
On Wed, Nov 14, 2012 at 8:47 AM, Charles Remes ch...@chuckremes.com wrote: I'm surprised that the docs aren't more clear about this detail. Honestly, the guide will teach you this but first you must read it. :) The zmq_socket man page is pretty clear about the valid combinations I think. The man page was wrong, not allowing DEALER-DEALER and ROUTER-ROUTER, which I recently fixed. http://api.zeromq.org/3-2:zmq-socket#toc3 seems pretty clear IMO. The names imply the proper pairings. In what world does ROUTER / PULL make sense from a semantic standpoint? How does a PULL socket even send a message to a ROUTER socket? How can a ROUTER talk to a PULL socket that hasn't sent it a message? You'd have to cheat with hard-coded identities. -Pieter ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] ROUTER/PULL
Thanks for the explanation. The original grouping by pattern makes it more clear why things work they way they do. On Wednesday, November 14, 2012 09:57:54 AM Pieter Hintjens wrote: Justin, The combinations are not arbitrary. There's an explanation here: http://zguide.zeromq.org/page:all#Request-Reply-Combinations. Firstly, you can replace REQ with DEALER, and REP with ROUTER, to turn a synchronous REQ-REP into a partially or fully asynchronous flow. Secondly, you can combine DEALER to DEALER and ROUTER to ROUTER to get two new patterns. The legal combinations are formally defined in the zmq_socket man page. The original design ideology, which was accurate IMO, was that each low-level pattern was a separate package. Thus, PUB+SUB, PUSH+PULL (pipeline), PAIR, REQ+REP+DEALER+ROUTER. We've been extending the protocol to announce each peer's socket type, to let us catch invalid combinations. The public spec in zmq_socket[3] defines what kind of future compatibility we promise. Since it states that ROUTER+REQ is valid, this will always work. But since it does not allow ROUTER+PUSH, we're free to break that in the future. Hope this helps to understand the reasoning here. -Pieter On Wed, Nov 14, 2012 at 7:54 AM, Justin Karneges jus...@affinix.com wrote: Then how do you justify, for example, the ability to mix REQ and ROUTER? Is that just an exception to the rule? In my opinion, from a design point of view, zmq should either have very strict patterns (e.g. no more REQ+ROUTER), or it should allow mixing where it makes sense (e.g. ROUTER+PULL). Doesn't this seem reasonable? Right now the allowed mixing seems kind of arbitrary to me. On Wednesday, November 14, 2012 07:28:47 AM Pieter Hintjens wrote: There are reasons to not mix ROUTER and PULL, mainly that they are from different patterns, so if we decide to improve PUSH/PULL as a package, we would be unable, if people were mixing them with other patterns. As an example, see how we improved PUB/SUB to do pub-side filtering. DEALER does just the same as PUSH + PULL, so ROUTER/DEALER gives you what you want, and is better too, since you can do things like send heartbeats in both directions. -Pieter On Wed, Nov 14, 2012 at 4:02 AM, Justin Karneges jus...@affinix.com wrote: Why not make it allowed in a future version? It doesn't seem any more unusual than other mixings. On Tuesday, November 13, 2012 12:32:33 PM Chuck Remes wrote: No, do not do this. If it works at all right now, it will break in a future library release when the wire protocol enforces peer socket type checking. ROUTER - ROUTER is fine as is ROUTER - DEALER. Alternately, do PUB - SUB. As of libzmq 3.2, the publisher filters out the outgoing messages so only those SUBs that have subscribed to the message will receive it. This will require you to add a subscription string as the first message part for all outgoing messages so the socket can filter it. Please read the guide if this doesn't make sense to you yet. There are lots of great examples with code. cr On Nov 13, 2012, at 12:00 PM, Justin Karneges jus...@affinix.com wrote: Inspired by the PUSH/ROUTER question a moment ago, I wonder if it ought to be possible to match ROUTER and PULL, for one-way directed communication? Currently I am using ROUTER-ROUTER for this, and the receiver just ignores the envelope. Being able to make the receiver PULL seems like it would be more natural. Justin ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] ROUTER/PULL
On Wednesday, November 14, 2012 10:08:31 AM Pieter Hintjens wrote: On Wed, Nov 14, 2012 at 8:47 AM, Charles Remes ch...@chuckremes.com wrote: The names imply the proper pairings. In what world does ROUTER / PULL make sense from a semantic standpoint? How does a PULL socket even send a message to a ROUTER socket? How can a ROUTER talk to a PULL socket that hasn't sent it a message? You'd have to cheat with hard-coded identities. Well, or use a separate socket for inbound communication. My application receives from one socket and sends to another. Justin ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] ROUTER/PULL
On Wed, Nov 14, 2012 at 10:50 AM, Justin Karneges jus...@affinix.com wrote: Well, or use a separate socket for inbound communication. My application receives from one socket and sends to another. How do you know the identity of the PULL socket you're sending to? The ROUTER socket does not work well with purely outbound communications. There are several aspects: * You need to invent and manage your own identities * You have no way of knowing when the connection is actually live * When sending, and connection is not live, messages are silently dropped * You have no way to heartbeat the connection, so cannot detect failures You've not identified any problem with using DEALER except it's not PULL, which seems arbitrary. You can literally replace ZMQ_PULL with ZMQ_DEALER in your code and it will work the same. The best way to use ROUTER sockets, from experience, is as response sockets: initiate the dialog from the peer. -Pieter ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] ROUTER/PULL
On Wednesday, November 14, 2012 11:15:09 AM Pieter Hintjens wrote: You've not identified any problem with using DEALER except it's not PULL, which seems arbitrary. You can literally replace ZMQ_PULL with ZMQ_DEALER in your code and it will work the same. Ah, okay, I did not think of this. Currently my receiving socket uses ROUTER and ignores the header, but I like the idea of using DEALER instead as that aligns slightly closer to the intended usage (read: of not really caring about envelopes or return addresses). Note that I never was using PULL in this context, I only wanted to start a discussion about the possibility. Anyway, the only real difference between DEALER and PULL then is that, at a glance, the PULL socket has a more obvious contract. This may help when familiarizing oneself to a zmq-based project, or for a network administrator that is configuring socket connections. But this is just a nitpick. How do you know the identity of the PULL socket you're sending to? The ROUTER socket does not work well with purely outbound communications. There are several aspects: * You need to invent and manage your own identities * You have no way of knowing when the connection is actually live * When sending, and connection is not live, messages are silently dropped * You have no way to heartbeat the connection, so cannot detect failures Of course there is some extra work when doing this. In my case, the use of credits-based flow control and timeouts (in both directions, but on separate sockets) ensures that failures are noticed. Justin ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
