Re: [zeromq-dev] Reconnect failed after iptables tricks
hi Pieter, Interface didn't change I believe. I conducted on server host (which is VirtualBox instace) following: iptables -I INPUT -s windows-client-ip -j DROP and after some time re-enabled client by flushing all iptable rules: iptables -F That's it.. 2013/12/13 Pieter Hintjens p...@imatix.com Is it possible that the interface is changed, so the bind is no longer active? On Fri, Dec 13, 2013 at 1:54 PM, artemv zmq artemv@gmail.com wrote: hi I was testing simple connection lose scenario on a simple DEALER-ROUTER architecture. D was a client on Windows7, R was a server on VirtualBox. What happened: I started them both, saw chatting between them in logs; after that I disabled a client ip on server host with iptables , ensured that there is no more chatting between peers; after that I had re-enabled IP address of the client on server host; then went to logs expecting to see that chatting began, but .. there was silence. The question is: is it eligible way to test reconnection? or this is a bug in ZMQ core? BR -artemv PS block script: iptables -I INPUT -s IP-of-client -j DROP unlock script: iptables -F ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Reconnect failed after iptables tricks
Could you please pastebin the code so we can have a look? -- Justin Cook +44 7500 960 000 +1 682 738 5380 Sent from a mobile device On 14 Dec 2013 16:11, artemv zmq artemv@gmail.com wrote: hi Pieter, Interface didn't change I believe. I conducted on server host (which is VirtualBox instace) following: iptables -I INPUT -s windows-client-ip -j DROP and after some time re-enabled client by flushing all iptable rules: iptables -F That's it.. 2013/12/13 Pieter Hintjens p...@imatix.com Is it possible that the interface is changed, so the bind is no longer active? On Fri, Dec 13, 2013 at 1:54 PM, artemv zmq artemv@gmail.com wrote: hi I was testing simple connection lose scenario on a simple DEALER-ROUTER architecture. D was a client on Windows7, R was a server on VirtualBox. What happened: I started them both, saw chatting between them in logs; after that I disabled a client ip on server host with iptables , ensured that there is no more chatting between peers; after that I had re-enabled IP address of the client on server host; then went to logs expecting to see that chatting began, but .. there was silence. The question is: is it eligible way to test reconnection? or this is a bug in ZMQ core? BR -artemv PS block script: iptables -I INPUT -s IP-of-client -j DROP unlock script: iptables -F ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Reconnect failed after iptables tricks
hi Justin, There's no sense to paste code. Client is - DEALER. Server is ROUTER. Inside infinite loop client sends a message and await for reply. By turn server -- inside infinite loop receives message and sends answer back. All socket settings -- by default. That's it. 2013/12/14 Justin Cook jhc...@gmail.com Could you please pastebin the code so we can have a look? -- Justin Cook +44 7500 960 000 +1 682 738 5380 Sent from a mobile device On 14 Dec 2013 16:11, artemv zmq artemv@gmail.com wrote: hi Pieter, Interface didn't change I believe. I conducted on server host (which is VirtualBox instace) following: iptables -I INPUT -s windows-client-ip -j DROP and after some time re-enabled client by flushing all iptable rules: iptables -F That's it.. 2013/12/13 Pieter Hintjens p...@imatix.com Is it possible that the interface is changed, so the bind is no longer active? On Fri, Dec 13, 2013 at 1:54 PM, artemv zmq artemv@gmail.com wrote: hi I was testing simple connection lose scenario on a simple DEALER-ROUTER architecture. D was a client on Windows7, R was a server on VirtualBox. What happened: I started them both, saw chatting between them in logs; after that I disabled a client ip on server host with iptables , ensured that there is no more chatting between peers; after that I had re-enabled IP address of the client on server host; then went to logs expecting to see that chatting began, but .. there was silence. The question is: is it eligible way to test reconnection? or this is a bug in ZMQ core? BR -artemv PS block script: iptables -I INPUT -s IP-of-client -j DROP unlock script: iptables -F ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Does ZMQ handle tcp-RST?
Ok. I set HWM to 0. Launched DEALER (my client) , and ROUTER (my server). Client sends hello , server replies with world. Laucnhed them in separate processes, looked at logs, seen some chatting, hello-world-hello-world , and so on. And then I decided to kill server process (on windows in cmdline: taskkill /f /pid PID). I expected that I would see the warnings produced by my application (since appl. logic is checking the result of .send(byte[]) function). But .send() is always good. So, with HWM=0 on socket and gotten RST, .send() function still tells me that send was successfull. Isn't this is a bug ? 2013/12/13 Pieter Hintjens p...@imatix.com On Fri, Dec 13, 2013 at 9:14 PM, Justin Karneges jus...@affinix.com wrote: If you want to prevent queuing in all cases, set HWM to 0. This will not actually prevent all queuing, just remove buffering in ZeroMQ. You'll still get buffering in TCP and on the network itself. If you want to remove all queuing completely, you have to switch to a synchronous REQ/REP model, which is nasty. Better, use a credit based flow control system to manage precisely the total amount of buffering. ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Calling connect more than once on the same endpoint
Good ol' pedantry. I'll take a stab at submitting a pull request. -Trev On Fri, Dec 13, 2013 at 3:14 PM, Pieter Hintjens p...@imatix.com wrote: I'll pedantically ask that we record problems rather than features. Problem: SUB accepts multiple connects to same PUB endpoint and result is nonsensical Solution: SUB socket should not allow multiple connects to same PUB endpoint. An issue will just rot, so you may want to find a way to send a patch to fix this problem. On Thu, Dec 12, 2013 at 8:09 PM, Trevor Bernard trevor.bern...@gmail.com wrote: Should I submit an issue for this feature? I would find it extremely useful. -Trev On Wed, Oct 16, 2013 at 11:39 AM, Pieter Hintjens p...@imatix.com wrote: Having said that, it is undocumented behavior, and there is no valid use for this in pub-sub (nor in dealer-router, nor in req-rep) and we might want to change the behavior. We do know what endpoints are already used, and could reject duplicate connects to the same endpoint. On Wed, Oct 16, 2013 at 4:26 PM, Pieter Hintjens p...@imatix.com wrote: Yes, this is expected behavior. Each connect is independent. In pub-sub this produces nonsense results but in other patterns it can be used to do things like prioritize one node over others (e.g. a PULL that connects twice to a PUSH). On Wed, Oct 16, 2013 at 4:17 PM, Trevor Bernard trevor.bern...@gmail.com wrote: I have a PUB/SUB topology and I accidently called connect twice to the same PUB endpoint and received duplicate messages. This holds true for N connects as well. Is this the correct behaviour? Simple test that recreates it: https://github.com/trevorbernard/double-trouble/blob/master/src/double_trouble/core.clj -Trev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev -- - Pieter Hintjens CEO of iMatix.com Founder of ZeroMQ community blog: http://hintjens.com -- - Pieter Hintjens CEO of iMatix.com Founder of ZeroMQ community blog: http://hintjens.com ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] will receive fail if TCP goes down in REQ socket
Thanks all for your comments. I should clarify my use case better. I am
using the lbbroker3 pattern, where the worker uses a REQ socket to connect
to a ROUTER socket on the broker. It sends I'm READY to let the broker
know that it can serve a request. After the first request, the response
serves as the I'm READY indication to the broker. Now, if the broker
restarts it will wait for an I'm READY message but worker won't send any.
How can worker know that broker restarted?
On Fri, Dec 13, 2013 at 10:24 PM, Pieter Hintjens p...@imatix.com wrote:
The ZMQ_REQ_RELAXED option on ZeroMQ v4, lets you resend requests.
On Sat, Dec 14, 2013 at 3:46 AM, Justin Karneges jus...@affinix.com
wrote:
ZeroMQ does not resend messages, so while the reconnect/queuing logic
will protect you to some degree, you still need to account for message
loss.
If you're using REQ then you'll need to timeout the request, otherwise
if a request or response message is lost then you'll never be able to
make a request on the socket ever again. So don't just indefinitely
block on a send or receive. Further, ZeroMQ historically hasn't had a
way to get a REQ socket out of the waiting for response state in the
event of a timeout other than by closing the socket and starting over.
This means the REQ type is not really usable in production. Better to
use DEALER and format a REQ-compatible message yourself. REP does not
suffer from these problems, so you can keep on using that and have
DEALER talk to REP.
Note: it is possible that very recent versions of ZeroMQ allow REQ
sockets to revert state on error but I haven't been following this
closely.
On 12/13/2013 04:17 PM, Sean Robertson wrote:
I believe the REQ will simply wait for the REP to come back up,
re-bind and send something.
On Fri, Dec 13, 2013 at 2:53 PM, Mohit Jaggi mohitja...@gmail.com
wrote:
Hi,
In most ZMQ sockets, the underlying TCP socket can change
transparently.
Does that apply to REQ-REP sockets as well? Or will the receive call
to ZMQ
socket fail?
sock = new REQ socket;
connect(sock);
while(1) {
request = receive(...);
...
send(response);
}
For example in the above code, let us say that the server with REP
socket on
the other side crashed and restarted. What will happen?
Thanks,
Mohit.
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] will receive fail if TCP goes down in REQ socket
Hi Mohit,
I went through the same thing recently. Not only does the worker need to know
if the broker has gone, the broker also needs to know the worker has gone:
1. worker connects to broker (based on lbbroker3 example)
2. worker sends “ready” command and waits for work.
3. broker sends idle commands every X seconds when there is no work to do.
4. worker replies to idle commands immediately.
In this way both the worker and broker know that they are still connected to
each other.
The broker removes the worker from being available until it replies from the
idle command.
If the worker doesn’t receive an idle command, it closes the socket and makes a
new connection.
This allows the worker to use a simple REQ socket. There is always one request
to one response over the connection.
If the broker can exit gracefully (deliberate shutdown, onto a crash), it could
also send a message to the workers to tell them to close their sockets and make
a new connection with another ready command.
Regards,
Matt.
On 15 Dec 2013, at 9:20 am, Mohit Jaggi mohitja...@gmail.com wrote:
Thanks all for your comments. I should clarify my use case better. I am using
the lbbroker3 pattern, where the worker uses a REQ socket to connect to a
ROUTER socket on the broker. It sends I'm READY to let the broker know that
it can serve a request. After the first request, the response serves as the
I'm READY indication to the broker. Now, if the broker restarts it will
wait for an I'm READY message but worker won't send any. How can worker
know that broker restarted?
On Fri, Dec 13, 2013 at 10:24 PM, Pieter Hintjens p...@imatix.com wrote:
The ZMQ_REQ_RELAXED option on ZeroMQ v4, lets you resend requests.
On Sat, Dec 14, 2013 at 3:46 AM, Justin Karneges jus...@affinix.com wrote:
ZeroMQ does not resend messages, so while the reconnect/queuing logic
will protect you to some degree, you still need to account for message loss.
If you're using REQ then you'll need to timeout the request, otherwise
if a request or response message is lost then you'll never be able to
make a request on the socket ever again. So don't just indefinitely
block on a send or receive. Further, ZeroMQ historically hasn't had a
way to get a REQ socket out of the waiting for response state in the
event of a timeout other than by closing the socket and starting over.
This means the REQ type is not really usable in production. Better to
use DEALER and format a REQ-compatible message yourself. REP does not
suffer from these problems, so you can keep on using that and have
DEALER talk to REP.
Note: it is possible that very recent versions of ZeroMQ allow REQ
sockets to revert state on error but I haven't been following this closely.
On 12/13/2013 04:17 PM, Sean Robertson wrote:
I believe the REQ will simply wait for the REP to come back up,
re-bind and send something.
On Fri, Dec 13, 2013 at 2:53 PM, Mohit Jaggi mohitja...@gmail.com wrote:
Hi,
In most ZMQ sockets, the underlying TCP socket can change transparently.
Does that apply to REQ-REP sockets as well? Or will the receive call to
ZMQ
socket fail?
sock = new REQ socket;
connect(sock);
while(1) {
request = receive(...);
...
send(response);
}
For example in the above code, let us say that the server with REP socket
on
the other side crashed and restarted. What will happen?
Thanks,
Mohit.
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
