Re: [zeromq-dev] ZMQ_PAIR over TCP
OK I think I finally get the ZMQ_PAIR concept. From Wikipedia: Monogamy is a form of relationship in which an individual has only one partner during their lifetime — alternately, only one partner at any one time (serial monogamy) A ZMQ_PAIR concept is *monogamy* I originally thought it was *serial monogamy* I will let it go now :) -Original Message- From: zeromq-dev On Behalf Of Luca Boccassi Sent: 20 June 2018 16:33 To: ZeroMQ development list Subject: Re: [zeromq-dev] ZMQ_PAIR over TCP As I mentioned, only one pipe is accepted at any given time. Just because externally it seems like the connection has dropped, it doesn't mean the TCP socket is gone yet and thus the pipe will still be there, and further connection will be refused. Your best bet is to try and use the heartbeat options with aggressive timings, which should cause the binding side to drop the dead pipe much faster. But in the end, PAIR is meant to be used between two endpoints, hence the name - if one disappears and another arrives, it's no longer a pair, and as they say, three's a crowd. On Wed, 2018-06-20 at 14:33 +, James Harvey wrote: > Yes it does work over TCP and it does attempt to reconnect when you > restart the bind end. > > In the background the zmq::tcp_connector attempts to make the > reconnection. It does all the right things and ends up in > zmq::tcp_connecter_t::out_event and calls the last line _socket- > >event_connected (_endpoint, fd); > > After this all looks good in terms of TCP, the connection is > established BUT no messages flow. > > I have been looking through the code trying to work out why ZMQ_PAIR > never supported reconnection. Is it related to the fact it was > originally designed for inproc which doesn’t support reconnect? https > ://github.com/zeromq/libzmq/blob/6bfa91f13f73b67be47ded18986b76d154d5 > 923b/src/socket_base.cpp#L685 > > It seems odd though as I use PUB/SUB/PUSH/PULL over inproc and TCP but > those patterns are not excluded from reconnect just because they can > be used over inproc. > > > > From: zeromq-dev On Behalf Of > vincent freedom > Sent: 20 June 2018 04:48 > To: zeromq-dev@lists.zeromq.org > Subject: Re: [zeromq-dev] ZMQ_PAIR over TCP > > Nope, it doesn't reconnect. It looks like it does work over tcp > though. > > https://pastebin.com/e42ysUAy > https://pastebin.com/jbi1KA3n > > Run the client for the second time and it won't receive any messages. > > -vf > > ___ > zeromq-dev mailing list > zeromq-dev@lists.zeromq.org > https://lists.zeromq.org/mailman/listinfo/zeromq-dev -- Kind regards, Luca Boccassi ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org https://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] ZMQ_PAIR over TCP
As I mentioned, only one pipe is accepted at any given time. Just because externally it seems like the connection has dropped, it doesn't mean the TCP socket is gone yet and thus the pipe will still be there, and further connection will be refused. Your best bet is to try and use the heartbeat options with aggressive timings, which should cause the binding side to drop the dead pipe much faster. But in the end, PAIR is meant to be used between two endpoints, hence the name - if one disappears and another arrives, it's no longer a pair, and as they say, three's a crowd. On Wed, 2018-06-20 at 14:33 +, James Harvey wrote: > Yes it does work over TCP and it does attempt to reconnect when you > restart the bind end. > > In the background the zmq::tcp_connector attempts to make the > reconnection. It does all the right things and ends up in > zmq::tcp_connecter_t::out_event and calls the last line _socket- > >event_connected (_endpoint, fd); > > After this all looks good in terms of TCP, the connection is > established BUT no messages flow. > > I have been looking through the code trying to work out why ZMQ_PAIR > never supported reconnection. Is it related to the fact it was > originally designed for inproc which doesn’t support reconnect? https > ://github.com/zeromq/libzmq/blob/6bfa91f13f73b67be47ded18986b76d154d5 > 923b/src/socket_base.cpp#L685 > > It seems odd though as I use PUB/SUB/PUSH/PULL over inproc and TCP > but those patterns are not excluded from reconnect just because they > can be used over inproc. > > > > From: zeromq-dev On Behalf Of > vincent freedom > Sent: 20 June 2018 04:48 > To: zeromq-dev@lists.zeromq.org > Subject: Re: [zeromq-dev] ZMQ_PAIR over TCP > > Nope, it doesn't reconnect. It looks like it does work over tcp > though. > > https://pastebin.com/e42ysUAy > https://pastebin.com/jbi1KA3n > > Run the client for the second time and it won't receive any messages. > > -vf > > ___ > zeromq-dev mailing list > zeromq-dev@lists.zeromq.org > https://lists.zeromq.org/mailman/listinfo/zeromq-dev -- Kind regards, Luca Boccassi signature.asc Description: This is a digitally signed message part ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org https://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] ZMQ_PAIR over TCP
Yes it does work over TCP and it does attempt to reconnect when you restart the bind end. In the background the zmq::tcp_connector attempts to make the reconnection. It does all the right things and ends up in zmq::tcp_connecter_t::out_event and calls the last line _socket->event_connected (_endpoint, fd); After this all looks good in terms of TCP, the connection is established BUT no messages flow. I have been looking through the code trying to work out why ZMQ_PAIR never supported reconnection. Is it related to the fact it was originally designed for inproc which doesn’t support reconnect? https://github.com/zeromq/libzmq/blob/6bfa91f13f73b67be47ded18986b76d154d5923b/src/socket_base.cpp#L685 It seems odd though as I use PUB/SUB/PUSH/PULL over inproc and TCP but those patterns are not excluded from reconnect just because they can be used over inproc. From: zeromq-dev On Behalf Of vincent freedom Sent: 20 June 2018 04:48 To: zeromq-dev@lists.zeromq.org Subject: Re: [zeromq-dev] ZMQ_PAIR over TCP Nope, it doesn't reconnect. It looks like it does work over tcp though. https://pastebin.com/e42ysUAy https://pastebin.com/jbi1KA3n Run the client for the second time and it won't receive any messages. -vf ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org https://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] ZMQ_PAIR over TCP
Nope, it doesn't reconnect. It looks like it does work over tcp though. https://pastebin.com/e42ysUAy https://pastebin.com/jbi1KA3n Run the client for the second time and it won't receive any messages. -vf ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org https://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] ZMQ_PAIR over TCP
I have checked the git repository of the zguide, this line belongs to 6 years ago: https://github.com/booksbyus/zguide/blame/b9ac51e2ef48712d3416628db626a97ff4006846/chapter2.txt#L948 If you are sure (make sure to test other binding and implementation like JeroMQ) change the documentation, so others will understand this limitation is not valid anymore On Mon, Jun 18, 2018 at 3:40 PM, Attila Magyari wrote: > @Esa, > > are these documents recent? I am currently using ZMQ_PAIR sockets and they > do reconnect quite well. > > On Mon, Jun 18, 2018 at 1:05 PM Esa HekmatiZadeh > wrote: > >> Hi James >> According to zguide: >> >>> When you want to coordinate a set of nodes on a network, PAIR sockets >>> won't work well any more. This is one of the few areas where the strategies >>> for threads and nodes are different. Principally, nodes come and go whereas >>> threads are usually static. PAIR sockets do not automatically reconnect if >>> the remote node goes away and comes back. >>> >> As I understand from the above quote; Thanks to these characteristics >> PAIR socket is more suitable for inproc environment where threads are >> statics however you can use PAIR sockets over TCP as long as you aware and >> accept the behavior of PAIR which does not reconnect. >> >> On Mon, Jun 18, 2018 at 3:31 PM, Attila Magyari wrote: >> >>> As I understood, the documents are outdated, and ZMQ_PAIR is supposed to >>> work not only for inproc protocols. I am definitely not sure about this, >>> and I as well use it for TCP, so would be interested in the answers. >>> >>> On Mon, Jun 18, 2018 at 12:40 PM James Harvey >>> wrote: >>> Hi, I have been using ZMQ_PAIR recently for a one to one data feed over TCP. I know the documentation doesn’t explicitly say this should work (over TCP) but for the most part it does. Last week I came across a situation where the bind end of the connection had reconnection issues if no data is flowing during the bounce of the TCP connection. Before I go to the trouble of recreating/documenting the behavior (I won’t call it a bug at this point) I want to know if there is support in the zeromq community for ZMQ_PAIR to be supported over TCP? As far as I can tell the documentation doesn’t explicitly say either way. I know there are other patterns (PUB/SUB, PUSH/PULL) that support TCP but they are not as simple as the ZMQ_PAIR where you want a strict bi directional one to one relationship. If it’s the case that ZMQ_PAIR is only to be used over inproc then maybe the connect/bind should fail for anything other than inproc so people don’t make mistakes and assume it’s going to work. Thanks James ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org https://lists.zeromq.org/mailman/listinfo/zeromq-dev >>> >>> ___ >>> zeromq-dev mailing list >>> zeromq-dev@lists.zeromq.org >>> https://lists.zeromq.org/mailman/listinfo/zeromq-dev >>> >>> >> ___ >> zeromq-dev mailing list >> zeromq-dev@lists.zeromq.org >> https://lists.zeromq.org/mailman/listinfo/zeromq-dev >> > > ___ > zeromq-dev mailing list > zeromq-dev@lists.zeromq.org > https://lists.zeromq.org/mailman/listinfo/zeromq-dev > > ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org https://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] ZMQ_PAIR over TCP
It depends. A pair socket will only ever have one peer at a time, as the doc says. So if the binding end gets a new peer trying to join before the other one was removed, it will ignore it. https://github.com/zeromq/libzmq/blob/master/src/pair.cpp#L56 On Mon, 2018-06-18 at 13:10 +0200, Attila Magyari wrote: > @Esa, > > are these documents recent? I am currently using ZMQ_PAIR sockets and > they > do reconnect quite well. > > On Mon, Jun 18, 2018 at 1:05 PM Esa HekmatiZadeh m> > wrote: > > > Hi James > > According to zguide: > > > > > When you want to coordinate a set of nodes on a network, PAIR > > > sockets > > > won't work well any more. This is one of the few areas where the > > > strategies > > > for threads and nodes are different. Principally, nodes come and > > > go whereas > > > threads are usually static. PAIR sockets do not automatically > > > reconnect if > > > the remote node goes away and comes back. > > > > > > > As I understand from the above quote; Thanks to these > > characteristics PAIR > > socket is more suitable for inproc environment where threads are > > statics > > however you can use PAIR sockets over TCP as long as you aware and > > accept > > the behavior of PAIR which does not reconnect. > > > > On Mon, Jun 18, 2018 at 3:31 PM, Attila Magyari > > wrote: > > > > > As I understood, the documents are outdated, and ZMQ_PAIR is > > > supposed to > > > work not only for inproc protocols. I am definitely not sure > > > about this, > > > and I as well use it for TCP, so would be interested in the > > > answers. > > > > > > On Mon, Jun 18, 2018 at 12:40 PM James Harvey > > m> > > > wrote: > > > > > > > Hi, > > > > > > > > > > > > > > > > I have been using ZMQ_PAIR recently for a one to one data feed > > > > over > > > > TCP. I know the documentation doesn’t explicitly say this > > > > should work > > > > (over TCP) but for the most part it does. Last week I came > > > > across a > > > > situation where the bind end of the connection had reconnection > > > > issues if > > > > no data is flowing during the bounce of the TCP connection. > > > > > > > > > > > > > > > > Before I go to the trouble of recreating/documenting the > > > > behavior (I > > > > won’t call it a bug at this point) I want to know if there is > > > > support in > > > > the zeromq community for ZMQ_PAIR to be supported over TCP? As > > > > far as I can > > > > tell the documentation doesn’t explicitly say either way. > > > > > > > > > > > > > > > > I know there are other patterns (PUB/SUB, PUSH/PULL) that > > > > support TCP > > > > but they are not as simple as the ZMQ_PAIR where you want a > > > > strict bi > > > > directional one to one relationship. > > > > > > > > > > > > > > > > If it’s the case that ZMQ_PAIR is only to be used over inproc > > > > then maybe > > > > the connect/bind should fail for anything other than inproc so > > > > people don’t > > > > make mistakes and assume it’s going to work. > > > > > > > > > > > > > > > > Thanks > > > > > > > > > > > > > > > > James > > > > ___ > > > > zeromq-dev mailing list > > > > zeromq-dev@lists.zeromq.org > > > > https://lists.zeromq.org/mailman/listinfo/zeromq-dev > > > > > > > > > > ___ > > > zeromq-dev mailing list > > > zeromq-dev@lists.zeromq.org > > > https://lists.zeromq.org/mailman/listinfo/zeromq-dev > > > > > > > > > > ___ > > zeromq-dev mailing list > > zeromq-dev@lists.zeromq.org > > https://lists.zeromq.org/mailman/listinfo/zeromq-dev > > > > ___ > zeromq-dev mailing list > zeromq-dev@lists.zeromq.org > https://lists.zeromq.org/mailman/listinfo/zeromq-dev -- Kind regards, Luca Boccassi signature.asc Description: This is a digitally signed message part ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org https://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] ZMQ_PAIR over TCP
@Esa, are these documents recent? I am currently using ZMQ_PAIR sockets and they do reconnect quite well. On Mon, Jun 18, 2018 at 1:05 PM Esa HekmatiZadeh wrote: > Hi James > According to zguide: > >> When you want to coordinate a set of nodes on a network, PAIR sockets >> won't work well any more. This is one of the few areas where the strategies >> for threads and nodes are different. Principally, nodes come and go whereas >> threads are usually static. PAIR sockets do not automatically reconnect if >> the remote node goes away and comes back. >> > As I understand from the above quote; Thanks to these characteristics PAIR > socket is more suitable for inproc environment where threads are statics > however you can use PAIR sockets over TCP as long as you aware and accept > the behavior of PAIR which does not reconnect. > > On Mon, Jun 18, 2018 at 3:31 PM, Attila Magyari wrote: > >> As I understood, the documents are outdated, and ZMQ_PAIR is supposed to >> work not only for inproc protocols. I am definitely not sure about this, >> and I as well use it for TCP, so would be interested in the answers. >> >> On Mon, Jun 18, 2018 at 12:40 PM James Harvey >> wrote: >> >>> Hi, >>> >>> >>> >>> I have been using ZMQ_PAIR recently for a one to one data feed over >>> TCP. I know the documentation doesn’t explicitly say this should work >>> (over TCP) but for the most part it does. Last week I came across a >>> situation where the bind end of the connection had reconnection issues if >>> no data is flowing during the bounce of the TCP connection. >>> >>> >>> >>> Before I go to the trouble of recreating/documenting the behavior (I >>> won’t call it a bug at this point) I want to know if there is support in >>> the zeromq community for ZMQ_PAIR to be supported over TCP? As far as I can >>> tell the documentation doesn’t explicitly say either way. >>> >>> >>> >>> I know there are other patterns (PUB/SUB, PUSH/PULL) that support TCP >>> but they are not as simple as the ZMQ_PAIR where you want a strict bi >>> directional one to one relationship. >>> >>> >>> >>> If it’s the case that ZMQ_PAIR is only to be used over inproc then maybe >>> the connect/bind should fail for anything other than inproc so people don’t >>> make mistakes and assume it’s going to work. >>> >>> >>> >>> Thanks >>> >>> >>> >>> James >>> ___ >>> zeromq-dev mailing list >>> zeromq-dev@lists.zeromq.org >>> https://lists.zeromq.org/mailman/listinfo/zeromq-dev >>> >> >> ___ >> zeromq-dev mailing list >> zeromq-dev@lists.zeromq.org >> https://lists.zeromq.org/mailman/listinfo/zeromq-dev >> >> > ___ > zeromq-dev mailing list > zeromq-dev@lists.zeromq.org > https://lists.zeromq.org/mailman/listinfo/zeromq-dev > ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org https://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] ZMQ_PAIR over TCP
Hi James According to zguide: > When you want to coordinate a set of nodes on a network, PAIR sockets > won't work well any more. This is one of the few areas where the strategies > for threads and nodes are different. Principally, nodes come and go whereas > threads are usually static. PAIR sockets do not automatically reconnect if > the remote node goes away and comes back. > As I understand from the above quote; Thanks to these characteristics PAIR socket is more suitable for inproc environment where threads are statics however you can use PAIR sockets over TCP as long as you aware and accept the behavior of PAIR which does not reconnect. On Mon, Jun 18, 2018 at 3:31 PM, Attila Magyari wrote: > As I understood, the documents are outdated, and ZMQ_PAIR is supposed to > work not only for inproc protocols. I am definitely not sure about this, > and I as well use it for TCP, so would be interested in the answers. > > On Mon, Jun 18, 2018 at 12:40 PM James Harvey wrote: > >> Hi, >> >> >> >> I have been using ZMQ_PAIR recently for a one to one data feed over TCP. >> I know the documentation doesn’t explicitly say this should work (over TCP) >> but for the most part it does. Last week I came across a situation where >> the bind end of the connection had reconnection issues if no data is >> flowing during the bounce of the TCP connection. >> >> >> >> Before I go to the trouble of recreating/documenting the behavior (I >> won’t call it a bug at this point) I want to know if there is support in >> the zeromq community for ZMQ_PAIR to be supported over TCP? As far as I can >> tell the documentation doesn’t explicitly say either way. >> >> >> >> I know there are other patterns (PUB/SUB, PUSH/PULL) that support TCP but >> they are not as simple as the ZMQ_PAIR where you want a strict bi >> directional one to one relationship. >> >> >> >> If it’s the case that ZMQ_PAIR is only to be used over inproc then maybe >> the connect/bind should fail for anything other than inproc so people don’t >> make mistakes and assume it’s going to work. >> >> >> >> Thanks >> >> >> >> James >> ___ >> zeromq-dev mailing list >> zeromq-dev@lists.zeromq.org >> https://lists.zeromq.org/mailman/listinfo/zeromq-dev >> > > ___ > zeromq-dev mailing list > zeromq-dev@lists.zeromq.org > https://lists.zeromq.org/mailman/listinfo/zeromq-dev > > ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org https://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] ZMQ_PAIR over TCP
As I understood, the documents are outdated, and ZMQ_PAIR is supposed to work not only for inproc protocols. I am definitely not sure about this, and I as well use it for TCP, so would be interested in the answers. On Mon, Jun 18, 2018 at 12:40 PM James Harvey wrote: > Hi, > > > > I have been using ZMQ_PAIR recently for a one to one data feed over TCP. > I know the documentation doesn’t explicitly say this should work (over TCP) > but for the most part it does. Last week I came across a situation where > the bind end of the connection had reconnection issues if no data is > flowing during the bounce of the TCP connection. > > > > Before I go to the trouble of recreating/documenting the behavior (I won’t > call it a bug at this point) I want to know if there is support in the > zeromq community for ZMQ_PAIR to be supported over TCP? As far as I can > tell the documentation doesn’t explicitly say either way. > > > > I know there are other patterns (PUB/SUB, PUSH/PULL) that support TCP but > they are not as simple as the ZMQ_PAIR where you want a strict bi > directional one to one relationship. > > > > If it’s the case that ZMQ_PAIR is only to be used over inproc then maybe > the connect/bind should fail for anything other than inproc so people don’t > make mistakes and assume it’s going to work. > > > > Thanks > > > > James > ___ > zeromq-dev mailing list > zeromq-dev@lists.zeromq.org > https://lists.zeromq.org/mailman/listinfo/zeromq-dev > ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org https://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] ZMQ_PAIR over TCP
Hi, I have been using ZMQ_PAIR recently for a one to one data feed over TCP. I know the documentation doesn't explicitly say this should work (over TCP) but for the most part it does. Last week I came across a situation where the bind end of the connection had reconnection issues if no data is flowing during the bounce of the TCP connection. Before I go to the trouble of recreating/documenting the behavior (I won't call it a bug at this point) I want to know if there is support in the zeromq community for ZMQ_PAIR to be supported over TCP? As far as I can tell the documentation doesn't explicitly say either way. I know there are other patterns (PUB/SUB, PUSH/PULL) that support TCP but they are not as simple as the ZMQ_PAIR where you want a strict bi directional one to one relationship. If it's the case that ZMQ_PAIR is only to be used over inproc then maybe the connect/bind should fail for anything other than inproc so people don't make mistakes and assume it's going to work. Thanks James ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org https://lists.zeromq.org/mailman/listinfo/zeromq-dev
