Re: [zeromq-dev] Sending via ROUTER socket to ROUTER socket fails initially with No route to host
Thanks Pieter. Does the hand shaking happen at connect? Because suppose there is a delay after the connect, before the initial send, the send succeeds. If the handshaking does happen at connect, and if it is possible to make sure the connect returns only after the handshaking is done, that would fix the weird behaviour. Of course it is weird only when you expect a successful connect to indicate 'pipe' established and the ability to send successfully after. But I am guessing since the connect should succeed even if the peer hasn't bound, it would be hard to wait till the handshake happens, as it could potentially take forever. Anoop Pieter Hintjens wrote: OK, we found it. There is some internal handshaking before a message can be routed, even if the ROUTER socket connects out. There's no way for libzmq to know the identity of the peer until it receives the identity from the peer. ROUTER pipes are created only for known identities, i.e. for a peer that has sent us its identity, no matter who connected to whom. We could force pipe creation in some way but it would require changes. Hope this helps. -Pieter On Fri, Apr 5, 2013 at 11:13 PM, Anoop Karollil anoop.karol...@gmail.com wrote: Pieter Hintjens wrote: On Mon, Apr 1, 2013 at 6:23 PM, Anoop Karollil anoop.karol...@gmail.com wrote: Pieter, any clue as to why the send with the connected (not bound) ROUTER socket might be failing initially? No clue, but if you can make a minimal test case in C, we can investigate. Afaik it should work and this would be considered a bug. Attached are two C programs that reproduce the problem. The broker binds and receives. The client connects and then sends. The send fails initially and then succeeds. ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Sending via ROUTER socket to ROUTER socket fails initially with No route to host
OK, we found it. There is some internal handshaking before a message can be routed, even if the ROUTER socket connects out. There's no way for libzmq to know the identity of the peer until it receives the identity from the peer. ROUTER pipes are created only for known identities, i.e. for a peer that has sent us its identity, no matter who connected to whom. We could force pipe creation in some way but it would require changes. Hope this helps. -Pieter On Fri, Apr 5, 2013 at 11:13 PM, Anoop Karollil anoop.karol...@gmail.com wrote: Pieter Hintjens wrote: On Mon, Apr 1, 2013 at 6:23 PM, Anoop Karollil anoop.karol...@gmail.com wrote: Pieter, any clue as to why the send with the connected (not bound) ROUTER socket might be failing initially? No clue, but if you can make a minimal test case in C, we can investigate. Afaik it should work and this would be considered a bug. Attached are two C programs that reproduce the problem. The broker binds and receives. The client connects and then sends. The send fails initially and then succeeds. Anoop ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Sending via ROUTER socket to ROUTER socket fails initially with No route to host
Pieter Hintjens wrote:
On Mon, Apr 1, 2013 at 6:23 PM, Anoop Karollil anoop.karol...@gmail.com wrote:
Pieter, any clue as to why the send with the connected (not bound)
ROUTER socket might be failing initially?
No clue, but if you can make a minimal test case in C, we can
investigate. Afaik it should work and this would be considered a bug.
Attached are two C programs that reproduce the problem. The broker binds
and receives. The client connects and then sends. The send fails
initially and then succeeds.
Anoop
#include zmq.h
#include stdio.h
#include string.h
#include assert.h
#define URI tcp://127.0.0.1:2
void main()
{
void *ctx = zmq_ctx_new();
assert(ctx);
void *client = zmq_socket(ctx, ZMQ_ROUTER);
assert(client);
char *client_id = CLIENT;
int rc = zmq_setsockopt(client, ZMQ_IDENTITY, client_id, strlen(client_id));
assert(!rc);
int option = 1;
rc = zmq_setsockopt(client, ZMQ_ROUTER_MANDATORY, option, sizeof(option));
assert(!rc);
rc = zmq_connect(client, URI);
char *broker_id = BROKER;
char *msg = Hello;
int sent_addr = 0;
while(1) {
if (!sent_addr) {
printf(Sending address...\n);
rc = zmq_send(client, broker_id, strlen(broker_id), ZMQ_SNDMORE);
} else {
printf(Sending %s...\n, msg);
rc = zmq_send(client, msg, strlen(msg), 0);
}
if (rc 0) {
printf(Success!\n);
if (sent_addr)
break;
else
sent_addr = 1;
} else {
printf(Error: %s\n, zmq_strerror(errno));
}
}
zmq_close(client);
zmq_ctx_destroy(ctx);
}
#include zmq.h
#include stdio.h
#include string.h
#include assert.h
#define URI tcp://127.0.0.1:2
void main()
{
void *ctx = zmq_ctx_new();
assert(ctx);
void *broker = zmq_socket(ctx, ZMQ_ROUTER);
assert(broker);
char *broker_id = BROKER;
int rc = zmq_setsockopt(broker, ZMQ_IDENTITY, broker_id, strlen(broker_id));
assert(!rc);
int option = 1;
rc = zmq_setsockopt(broker, ZMQ_ROUTER_MANDATORY, option, sizeof(option));
assert(!rc);
rc = zmq_bind(broker, URI);
assert(!rc);
zmq_msg_t message;
while (1) {
zmq_msg_init(message);
zmq_msg_recv(message, broker, 0);
int more = zmq_msg_more(message);
zmq_msg_close(message);
if (!more)
break;
}
zmq_close(broker);
zmq_ctx_destroy(ctx);
}
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Sending via ROUTER socket to ROUTER socket fails initially with No route to host
On Mon, Apr 1, 2013 at 6:23 PM, Anoop Karollil anoop.karol...@gmail.com wrote: Pieter, any clue as to why the send with the connected (not bound) ROUTER socket might be failing initially? Please see my previous email on this thread for more details about what I am seeing. No clue, but if you can make a minimal test case in C, we can investigate. Afaik it should work and this would be considered a bug. -Pieter ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Sending via ROUTER socket to ROUTER socket fails initially with No route to host
On Tue, Mar 26, 2013 at 12:53 AM, Anoop Karollil anoop.karol...@gmail.com wrote: When I try sending messages via a ROUTER socket to a ROUTER socket (both having their identities set), with ZMQ_ROUTER_MANDATORY set, the initial send of the destination sock identity (sent before actual message for routing) fails once, but succeeds after. From the zmq_bind man page: following a zmq_bind(), the socket enters a mute state unless or until at least one incoming or outgoing connection is made, at which point the socket enters a ready state. In the mute state, the socket blocks or drops messages according to the socket type, as defined in zmq_socket(3). By contrast, following a libzmq:zmq_connect[3], the socket enters the ready state. -Pieter ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Sending via ROUTER socket to ROUTER socket fails initially with No route to host
Hi Mark, ROUTER sockets can be used to do sends and receives in any order. See Send/receive pattern for it here: http://api.zeromq.org/3-2:zmq-socket Anoop A. Mark wrote: Hi Anoop! As I understand ZMQ_ROUTER sockets are like ZMQ_REP socket in that they must receive a msg first. You cannot send a msg off the ZMQ_ROUTER socket without it already had received a msg first. In your example you send to the broker first. Also in your loop there should be REQ-REP pattern eventually somewhere. -Mark On Mon, Mar 25, 2013 at 4:53 PM, Anoop Karollil anoop.karol...@gmail.com mailto:anoop.karol...@gmail.com wrote: Hello, I am using ZeroMQ 3.2.2 and PyZMQ 13.0.0. When I try sending messages via a ROUTER socket to a ROUTER socket (both having their identities set), with ZMQ_ROUTER_MANDATORY set, the initial send of the destination sock identity (sent before actual message for routing) fails once, but succeeds after. Please see attached C or Python file which reproduces the behavior. It seems to happen only if the SNDMORE option is set - which if my understanding is correct is what should be done while sending the destination socket identity, before the actual message is sent with the SNDMORE flag unset. Is this intended behavior or am I doing something wrong? The C program gives the following output: Sending address... Error: No route to host Sending address... Success! Sending Hello... Success! Thanks, Anoop ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org mailto:zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Sending via ROUTER socket to ROUTER socket fails initially with No route to host
On 13-03-25 11:14 PM Pieter Hintjens p...@imatix.com wrote to ZeroMQ development list zeromq-dev@lists.zeromq.org : On Tue, Mar 26, 2013 at 12:53 AM, Anoop Karollil anoop.karol...@gmail.com wrote: When I try sending messages via a ROUTER socket to a ROUTER socket (both having their identities set), with ZMQ_ROUTER_MANDATORY set, the initial send of the destination sock identity (sent before actual message for routing) fails once, but succeeds after. From the zmq_bind man page: following a zmq_bind(), the socket enters a mute state unless or until at least one incoming or outgoing connection is made, at which point the socket enters a ready state. In the mute state, the socket blocks or drops messages according to the socket type, as defined in zmq_socket(3). By contrast, following a libzmq:zmq_connect[3], the socket enters the ready state. Pieter, the second socket does make a connection to the URI bound to by the first socket before it sends the message. Or by connection does the manual mean the actual underlying TCP connection? And that it gets made only when an initial message is sent to the bound socket and not when the connect call is done? If that is the case, there is no getting around having to resend the first message that you send to a bound socket because it will always be in a mute state initially? Thanks, Anoop ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Sending via ROUTER socket to ROUTER socket fails initially with No route to host
On 13-03-26 11:07 AM Pieter Hintjens p...@imatix.com wrote to ZeroMQ development list zeromq-dev@lists.zeromq.org : * 0MQ sockets manage a set of pipes to connected peers * A pipe is created either on a connect (at once) or when an incoming connection arrives * No pipe means messages can't be sent... (the mute state) So if you bind a socket and then connect to it, there will be an arbitrary period when you can't send. Using ZMQ_MANDATORY means you get an error (otherwise the message is simply dropped at once). Thank you for clearing that up. Once the pipe is there, you can send messages. The standard pattern for a bound socket is to wait for an incoming message, which guarantees the pipe is there. But you can also use other strategies, like pausing and retrying. I am actually following the standard pattern for a bound socket but not in the previous example code. Attached are the broker and client split up into two separate programs that can be run independently and lets the bound broker socket receive first and the connected client socket send after. And yes, like you said, the send after connect succeeds if there is a pause in between. Would it be possible to make the connect call block till the pipe is set up? The failure would actually get handled by higher level reliability mechanisms, but I found the behaviour weird. Thanks, Anoop import zmq broker_uri = tcp://127.0.0.1:2 context = zmq.Context() broker = context.socket(zmq.ROUTER) broker.setsockopt(zmq.IDENTITY, BROKER) broker.setsockopt(zmq.ROUTER_BEHAVIOR, 1) broker.bind(broker_uri) parts = broker.recv_multipart() print(parts) import zmq import time broker_uri = tcp://127.0.0.1:2 context = zmq.Context() client = context.socket(zmq.ROUTER) client.setsockopt(zmq.IDENTITY, CLIENT) client.setsockopt(zmq.ROUTER_BEHAVIOR, 1) client.connect(broker_uri) time.sleep(1) try: client.send_multipart([BROKER, Hello]) except zmq.error.ZMQError, e: print(Error: %s % e) else: print(Success!) ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Sending via ROUTER socket to ROUTER socket fails initially with No route to host
On Tue, Mar 26, 2013 at 10:09 PM, Anoop Karollil anoop.karol...@gmail.com wrote: And yes, like you said, the send after connect succeeds if there is a pause in between. The send should work immediately after connect, which creates a pipe at once. It's the bind that's troublesome. -Pieter ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Sending via ROUTER socket to ROUTER socket fails initially with No route to host
Pieter Hintjens wrote: On Tue, Mar 26, 2013 at 10:09 PM, Anoop Karollil anoop.karol...@gmail.com wrote: And yes, like you said, the send after connect succeeds if there is a pause in between. The send should work immediately after connect, which creates a pipe at once. It's the bind that's troublesome. I do see the send after connect failing if it is done immediately after, and it succeeding if there is a pause in between. Please see the example python programs. Starting the broker first (which binds and does a receive), and then the client after (which connects and sends), results in the client send failing, unless there is a pause after the connect. If you uncomment the sleep in client.py, it works without failure. If you don't, it will fail for a while and then succeed. Thanks, Anoop import zmq broker_uri = tcp://127.0.0.1:2 context = zmq.Context() broker = context.socket(zmq.ROUTER) broker.setsockopt(zmq.IDENTITY, BROKER) broker.setsockopt(zmq.ROUTER_BEHAVIOR, 1) broker.bind(broker_uri) parts = broker.recv_multipart() print(parts) import zmq import time broker_uri = tcp://127.0.0.1:2 context = zmq.Context() client = context.socket(zmq.ROUTER) client.setsockopt(zmq.IDENTITY, CLIENT) client.setsockopt(zmq.ROUTER_BEHAVIOR, 1) client.connect(broker_uri) #time.sleep(1) while True: try: client.send_multipart([BROKER, Hello]) except zmq.error.ZMQError, e: print(Error: %s % e) else: print(Success!) break ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Sending via ROUTER socket to ROUTER socket fails initially with No route to host
I see, so since it's unrestricted it's fine with ROUTER to ROUTER, but with REQ-ROUTER is not going to work. On Tue, Mar 26, 2013 at 2:55 PM, Anoop Karollil anoop.karol...@gmail.comwrote: Pieter Hintjens wrote: On Tue, Mar 26, 2013 at 10:09 PM, Anoop Karollil anoop.karol...@gmail.com wrote: And yes, like you said, the send after connect succeeds if there is a pause in between. The send should work immediately after connect, which creates a pipe at once. It's the bind that's troublesome. I do see the send after connect failing if it is done immediately after, and it succeeding if there is a pause in between. Please see the example python programs. Starting the broker first (which binds and does a receive), and then the client after (which connects and sends), results in the client send failing, unless there is a pause after the connect. If you uncomment the sleep in client.py, it works without failure. If you don't, it will fail for a while and then succeed. Thanks, Anoop ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] Sending via ROUTER socket to ROUTER socket fails initially with No route to host
Hello,
I am using ZeroMQ 3.2.2 and PyZMQ 13.0.0.
When I try sending messages via a ROUTER socket to a ROUTER socket (both
having their identities set), with ZMQ_ROUTER_MANDATORY set, the initial
send of the destination sock identity (sent before actual message for
routing) fails once, but succeeds after.
Please see attached C or Python file which reproduces the behavior. It
seems to happen only if the SNDMORE option is set - which if my
understanding is correct is what should be done while sending the
destination socket identity, before the actual message is sent with the
SNDMORE flag unset.
Is this intended behavior or am I doing something wrong?
The C program gives the following output:
Sending address...
Error: No route to host
Sending address...
Success!
Sending Hello...
Success!
Thanks,
Anoop
#include zmq.h
#include stdio.h
#include string.h
#include assert.h
#define URI tcp://127.0.0.1:2
void main()
{
void *ctx = zmq_ctx_new();
assert(ctx);
void *broker = zmq_socket(ctx, ZMQ_ROUTER);
assert(broker);
char *broker_id = BROKER;
int rc = zmq_setsockopt(broker, ZMQ_IDENTITY, broker_id, strlen(broker_id));
assert(!rc);
int option = 1;
rc = zmq_setsockopt(broker, ZMQ_ROUTER_MANDATORY, option, sizeof(option));
assert(!rc);
rc = zmq_bind(broker, URI);
assert(!rc);
void *client = zmq_socket(ctx, ZMQ_ROUTER);
assert(client);
char *client_id = CLIENT;
rc = zmq_setsockopt(client, ZMQ_IDENTITY, client_id, strlen(client_id));
assert(!rc);
option = 1;
rc = zmq_setsockopt(client, ZMQ_ROUTER_MANDATORY, option, sizeof(option));
assert(!rc);
rc = zmq_connect(client, URI);
char *msg = Hello;
int sent_addr = 0;
while(1) {
if (!sent_addr) {
printf(Sending address...\n);
rc = zmq_send(client, broker_id, strlen(broker_id), ZMQ_SNDMORE);
} else {
printf(Sending %s...\n, msg);
rc = zmq_send(client, msg, strlen(msg), 0);
}
if (rc 0) {
printf(Success!\n);
if (sent_addr)
break;
else
sent_addr = 1;
} else {
printf(Error: %s\n, zmq_strerror(errno));
sleep(1);
}
}
}
import zmq
import time
import sys
broker_uri = tcp://127.0.0.1:2
context = zmq.Context()
broker = context.socket(zmq.ROUTER)
broker.setsockopt(zmq.IDENTITY, BROKER)
broker.setsockopt(zmq.ROUTER_BEHAVIOR, 1)
broker.bind(broker_uri)
client = context.socket(zmq.ROUTER)
client.setsockopt(zmq.IDENTITY, CLIENT)
client.setsockopt(zmq.ROUTER_BEHAVIOR, 1)
client.connect(broker_uri)
while True:
try:
client.send_multipart([BROKER, Hello])
except zmq.error.ZMQError, e:
print(Error: %s % e)
else:
print(Success!)
break
time.sleep(1)
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Sending via ROUTER socket to ROUTER socket fails initially with No route to host
Hi Anoop! As I understand ZMQ_ROUTER sockets are like ZMQ_REP socket in that they must receive a msg first. You cannot send a msg off the ZMQ_ROUTER socket without it already had received a msg first. In your example you send to the broker first. Also in your loop there should be REQ-REP pattern eventually somewhere. -Mark On Mon, Mar 25, 2013 at 4:53 PM, Anoop Karollil anoop.karol...@gmail.comwrote: Hello, I am using ZeroMQ 3.2.2 and PyZMQ 13.0.0. When I try sending messages via a ROUTER socket to a ROUTER socket (both having their identities set), with ZMQ_ROUTER_MANDATORY set, the initial send of the destination sock identity (sent before actual message for routing) fails once, but succeeds after. Please see attached C or Python file which reproduces the behavior. It seems to happen only if the SNDMORE option is set - which if my understanding is correct is what should be done while sending the destination socket identity, before the actual message is sent with the SNDMORE flag unset. Is this intended behavior or am I doing something wrong? The C program gives the following output: Sending address... Error: No route to host Sending address... Success! Sending Hello... Success! Thanks, Anoop ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
