Re: [zeromq-dev] FW: Help With Regard To the ZMQ Forwarder
Hello Cornelius,
Thanks for pointing that out. I Used ProcessDevice and it worked the way i
want, but was not able to understand, some questions:
1) Does ProcessDevice takes care of doing the clean up work ??
2) Is there way i can close the processdevice or i can check if its already
running ??
I tried terminating the python interpreter , but as that is a different
process i dont this thats going to killed with my old way.
Thanks,
Ronald
Date: Fri, 23 Mar 2012 10:35:02 -0500
From: cornelius.to...@gmail.com
To: zeromq-dev@lists.zeromq.org
Subject: Re: [zeromq-dev] FW: Help With Regard To the ZMQ Forwarder
Check out the documentation on devices and pyzmq as well as the
zguide
... whenever i start a zmq.device(zmq.Forwarder), this blocks my interpreter,
so is there any way of starting a forwarder always as a different process or
any ways of making it non-blocking.
http://zeromq.github.com/pyzmq/devices.html
http://zguide.zeromq.org/page:all#Intermediates-and-Devices--
Cornelius Toole
Sent with Sparrow
On Friday, March 23, 2012 at 12:19 AM, Ronald Swain wrote:
Hello All,
I had one more small question, whenever i start a zmq.device(zmq.Forwarder),
this blocks my interpreter, so is there any way of starting a forwarder always
as a different process or any ways of making it non-blocking.
Hope my question is clear.
Thanks Regards,
Ronald
From: proj_symb...@live.com
To: zeromq-dev@lists.zeromq.org
Date: Tue, 20 Mar 2012 12:27:24 +0530
Subject: Re: [zeromq-dev] FW: Help With Regard To the ZMQ Forwarder
Ok the first question is i am back to zero, so now when my scripts are working
i am able to send messages, the question of cleaning comes.
i am using the following code for the forwarder and i think thats correct, but
when i kill the process which is running the forwarder the SIGTERM signal is
not fired i guess and incoming, outgoing and zmq context is not cleaned
properly.
code:
import zmq
import signal
# this method handles the termination of the app
# this handler is responsible for properly cleaning of the ports
def termSignalHandler(signum,frame):
incoming.close()
outgoing.close()
context.term()
print(termSignal Handler Called)
def startForwarder():
context = zmq.Context(1)
incoming = context.socket(zmq.SUB)
outgoing = context.socket(zmq.PUB)
try:
incoming.connect(tcp://127.0.0.1:5559);
incoming.setsockopt(zmq.SUBSCRIBE, )
except:
print(incoming socket is already open)
try:
outgoing.bind('tcp://127.0.0.1:4449')
except:
print(outgoing socket is open)
zmq.device(zmq.FORWARDER, incoming, outgoing)
signal.signal(signal.SIGTERM,termSignalHandler)
startForwarder()
a proper copy can be found at https://gist.github.com/2132163, can you guys
tell me what wrong i am doing here.
From: proj_symb...@live.com
To: zeromq-dev@lists.zeromq.org
Date: Tue, 20 Mar 2012 11:23:44 +0530
Subject: Re: [zeromq-dev] FW: Help With Regard To the ZMQ Forwarder
Hello Cornelius and Justin,
I was at last able to get my forwarder to work and it was my mistake.
My sender code which was like:
def sendMessage():
context = zmq.Context()
sender = context.socket(zmq.PUB)
sender.bind('tcp://127.0.0.1:5558')
sender.send(This is the sender)
was the actual problem, i got to know from my reading that call to socket.bind
should happen only once, so i changed my sender code to
something like:
socket = context.socket(zmq.PUB)
socket.bind('tcp://127.0.0.1:4443')
def sendMessage(socket,message):
try:
socket.send(message)
except:
print Unable To Send Message
and than i was able to send messages but this gets me into two three more
problems, i hope you guys will help me out is giving me some
ideas about this.
Date: Mon, 19 Mar 2012 13:18:59 -0500
From: cornto...@cct.lsu.edu
To: zeromq-dev@lists.zeromq.org
Subject: Re: [zeromq-dev] FW: Help With Regard To the ZMQ Forwarder
From the looks of your code https://gist.github.com/2119078,
you define functions for sending and receiving messages, but you do not call
them, like you do with Forwarder.py.
Also for the sake of easier testing, you may want to have your sender script to
send multiple messages to allow you to see what's going on.--
Cornelius Toole
Sent with Sparrow
On Monday, March 19, 2012 at 11:59 AM, Symbian Projects wrote:
Thanks for your reply cornelius.
I am now using netstat for checking the blocked ports.
Meanwhile i have uploaded the code here https://gist.github.com/2119078
and there is no proprietary code at the moment its the basic code where i am
trying to get the forwarder working.
and your last point is very valid, i start first the
Re: [zeromq-dev] Duplicate identities - #325
From what I understand it may take more time to fix it by closing the offender's connection and may cause other problems, if this is the case I'd rather have a working fix right now. I am working on a project where zeromq (or crosroads btw) would be of great help but I need to have the server publicly accessible and this bug is preventing me to even think of doing it currently (or at least was). I still fear there are remaining assertions which may explode in my face but at least that's one less ! On 24 March 2012 16:59, Chuck Remes li...@chuckremes.com wrote: On Mar 23, 2012, at 8:24 PM, Pieter Hintjens wrote: This is for routing only, we don't do durable sockets any longer in 3.1. So it should be OK. FYI, this patch was rejected by crossroads-io. They prefer it be handled by closing the offending connection rather than silently allowing it to connect but with an auto identity. cr ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] Event-driven publish-subscribe model
Hello, I am working on a requirement whereby a process (say producer) needs to send out one-way messages to a variable number of processes (say consumers). The publish-subscribe model seemed good for this because the consumers will subscribe to messages from the producer. I tried using ZeroMQ to achieve this. However, I have a few problems with it: 1. The consumers have to continuously poll for messages. I would have the consumers to be notified when there is a new message. 2. There is a possibility of the producer queue being filled up. I would have liked the producer to remove messages from the queue based on some condition (say remove messages older than 5 seconds, or remove messages that have been read 5 times). I see ZMQ recommends using a HWM, but I would like more selective removal of elements from the buffer. 3. Since the consumers are polling and the messages are not removed from the queue, the consumers see duplicate messages till a new message comes in. I want the consumer to be notified only once per new message. I understand I may be using a wrong model (publish-subscribe may not be suitable). I have thought about using request-reply, but that doesn't work since the producer does not want to keep track of the number of consumers. Can anyone suggest a good alternative? Thanks in advance! Raj -- Twitter: @Blismobile http://twitter.com/#!/blismobile BlisMobile Media 32 Percy Street, London W1T 2DE www.blismobile.com [image: BlisMobile] http://www.blismobile.com/[image: Follow on Twitter]http://twitter.com/#!/blismobile[image: Blis Website] http://www.blismobile.com/ This communication is from BlisMobile Media, which is a trading name of Breeze Tech (UK) Ltd, a company registered in England and Wales with registered number 06455773. Its registered office is 32 Percy Street, London W1T 2DE, United Kingdom. This communication contains information that is confidential and may also be privileged. It is for the exclusive use of the intended recipient(s). If you are not the intended recipient(s), please (1) notify i...@blismobile.com by forwarding this email and delete all copies from your system and (2) note that disclosure, distribution, copying or use of this communication is strictly prohibited. Email communications cannot be guaranteed to be secure or free from error or viruses. All emails sent to or from a Blismobile email account are securely archived and stored by an external supplier. This email does not constitute a contractual agreement; such agreements are in specified contractual or Insertion Order (IO) form only and exclusively contain all the terms to which Breeze Tech )UK) Ltd will be bound. To the extent permitted by law, Breeze Tech (UK) Ltd does not accept any liability for use of or reliance on the contents of this email by any person save by the intended recipient(s) to the extent agreed in a contract or Insertion Order. Opinions, conclusions and other information in this email which have not been delivered by way of the business of Breeze Tech (UK) Ltd are neither given nor endorsed by it. ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Event-driven publish-subscribe model
Hello, I understand I have to use zmq_poll which is essentially event-driven. So that solves problem (1) and (3). However, I still need some comments on point (2) below. Thanks! On 26 March 2012 12:52, Rajalakshmi Iyer r...@blismobile.com wrote: Hello, I am working on a requirement whereby a process (say producer) needs to send out one-way messages to a variable number of processes (say consumers). The publish-subscribe model seemed good for this because the consumers will subscribe to messages from the producer. I tried using ZeroMQ to achieve this. However, I have a few problems with it: 1. The consumers have to continuously poll for messages. I would have the consumers to be notified when there is a new message. 2. There is a possibility of the producer queue being filled up. I would have liked the producer to remove messages from the queue based on some condition (say remove messages older than 5 seconds, or remove messages that have been read 5 times). I see ZMQ recommends using a HWM, but I would like more selective removal of elements from the buffer. 3. Since the consumers are polling and the messages are not removed from the queue, the consumers see duplicate messages till a new message comes in. I want the consumer to be notified only once per new message. I understand I may be using a wrong model (publish-subscribe may not be suitable). I have thought about using request-reply, but that doesn't work since the producer does not want to keep track of the number of consumers. Can anyone suggest a good alternative? Thanks in advance! Raj -- Twitter: @Blismobile http://twitter.com/#!/blismobile BlisMobile Media 32 Percy Street, London W1T 2DE www.blismobile.com [image: BlisMobile] http://www.blismobile.com/[image: Follow on Twitter]http://twitter.com/#!/blismobile[image: Blis Website] http://www.blismobile.com/ This communication is from BlisMobile Media, which is a trading name of Breeze Tech (UK) Ltd, a company registered in England and Wales with registered number 06455773. Its registered office is 32 Percy Street, London W1T 2DE, United Kingdom. This communication contains information that is confidential and may also be privileged. It is for the exclusive use of the intended recipient(s). If you are not the intended recipient(s), please (1) notify i...@blismobile.com by forwarding this email and delete all copies from your system and (2) note that disclosure, distribution, copying or use of this communication is strictly prohibited. Email communications cannot be guaranteed to be secure or free from error or viruses. All emails sent to or from a Blismobile email account are securely archived and stored by an external supplier. This email does not constitute a contractual agreement; such agreements are in specified contractual or Insertion Order (IO) form only and exclusively contain all the terms to which Breeze Tech )UK) Ltd will be bound. To the extent permitted by law, Breeze Tech (UK) Ltd does not accept any liability for use of or reliance on the contents of this email by any person save by the intended recipient(s) to the extent agreed in a contract or Insertion Order. Opinions, conclusions and other information in this email which have not been delivered by way of the business of Breeze Tech (UK) Ltd are neither given nor endorsed by it. ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Event-driven publish-subscribe model
On Mar 26, 2012, at 10:24 AM, Rajalakshmi Iyer wrote: Hello, I understand I have to use zmq_poll which is essentially event-driven. So that solves problem (1) and (3). However, I still need some comments on point (2) below. Thanks! On 26 March 2012 12:52, Rajalakshmi Iyer r...@blismobile.com wrote: Hello, I am working on a requirement whereby a process (say producer) needs to send out one-way messages to a variable number of processes (say consumers). The publish-subscribe model seemed good for this because the consumers will subscribe to messages from the producer. I tried using ZeroMQ to achieve this. However, I have a few problems with it: The consumers have to continuously poll for messages. I would have the consumers to be notified when there is a new message. There is a possibility of the producer queue being filled up. I would have liked the producer to remove messages from the queue based on some condition (say remove messages older than 5 seconds, or remove messages that have been read 5 times). I see ZMQ recommends using a HWM, but I would like more selective removal of elements from the buffer. Since the consumers are polling and the messages are not removed from the queue, the consumers see duplicate messages till a new message comes in. I want the consumer to be notified only once per new message. I understand I may be using a wrong model (publish-subscribe may not be suitable). I have thought about using request-reply, but that doesn't work since the producer does not want to keep track of the number of consumers. Can anyone suggest a good alternative? Raj, I recommend that you read this posting: http://unprotocols.org/blog:15 That is a far better mechanism than the built-in HWM mechanism. Since it isn't built in to the library, you would have to add this logic at the application level. Since it *requires* two-way communication between the client and server, you cannot use the PUB/SUB sockets. You would need to use DEALER/ROUTER sockets to handle this communication. Yes, this will be a little more work but it is necessary for the kind of flexibility that your #2 issue requires. I hope this helps. cr PS - we would *love* it if someone made a patch to libzmq that added credit-based flow control as an option for all socket types. ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Event-driven publish-subscribe model
On Mon, Mar 26, 2012 at 4:42 PM, Chuck Remes li...@chuckremes.com wrote: That is a far better mechanism than the built-in HWM mechanism. Since it isn't built in to the library, you would have to add this logic at the application level. Since it *requires* two-way communication between the client and server, you cannot use the PUB/SUB sockets. You would need to use DEALER/ROUTER sockets to handle this communication. Yes, this will be a little more work but it is necessary for the kind of flexibility that your #2 issue requires. I think you can still use PUB/SUB here, if you're doing at-least-one kind of flow control you just need credit from *someone*, not everyone, and you could do that with a simple pull or rep socket - as soon as anyone adds credit, then you push the message out the pub. This is actually pretty much just a send-on-ack sliding window, but with multiple clients. Doesn't need to be a library level, that's an easy bit of code in application land. By doing the checking in app land you also get the opportunity to be clever - for example, rolling up messages if they supersede each other, that kind of thing. Ian ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] ZeroMQ 2.1.11 - poll returns errno 11 on ZMQ_SUB sockets
Hello,
I am trying a simple PUBLISH/SUBSCRIBE setup with ZeroMQ 2.1.11.
Here is the server code:
#include zmq.hpp
#include iostream
using namespace std;
int main(void)
{
try
{
zmq::context_t context(1);
zmq::socket_t publisher(context, ZMQ_PUB);
publisher.bind(tcp://*:5556);
sleep(5);
while(1)
{
char* command = Command;
zmq::message_t msg(command, strlen(command), NULL, NULL);
if (!publisher.send(msg))
{
cout Unable to send message endl;
}
sleep(2);
}
}
catch (zmq::error_t e)
{
cout ZMQ Exception : e.what() endl;
}
return 0;
}
And this is the client code:
#include zmq.hpp
#include iostream
using namespace std;
int main()
{
try
{
zmq::context_t context(1);
cout Serving commands ... endl;
zmq::socket_t subscriber(context, ZMQ_SUB);
subscriber.connect(tcp://localhost:5556);
sleep(2);
while(1)
{
zmq::pollitem_t items[] = {
{(void*)subscriber, 0, ZMQ_POLLIN, 0}
};
int rc = zmq::poll(items, 1, 1);
cout zmq_poll(rc= errno ) endl;
if (items[0].revents ZMQ_POLLIN)
{
zmq::message_t command;
if (!subscriber.recv(command, ZMQ_NOBLOCK))
{
cout Failed to receive command endl;
}
cout Received command: (char*)command.data()
endl;
}
}
}
catch (zmq::error_t e)
{
cout ZMQ Exception : e.what() endl;
}
return 0;
}
The server is sending commands every 2 seconds over the ZMQ_PUB socket. The
client does zmq_poll on the ZMQ_SUB socket in a loop. However, the client
always sees errno 11 on a zmq_poll (which seems to indicate Resource
Temporarily Unavailable).
I have tried to figure out the cause from other posts in the mailing lists.
They seemed to indicate adding a sleep after the bind/connect phase before
starting to send/recv data. Even that has not helped.
Apologies, if this has been answered earlier, but it would be really great
if someone can help me out here.
Thanks,
Raj.
--
Twitter: @Blismobile http://twitter.com/#!/blismobile
BlisMobile Media
32 Percy Street,
London W1T 2DE
www.blismobile.com
[image: BlisMobile] http://www.blismobile.com/[image: Follow on
Twitter]http://twitter.com/#!/blismobile[image:
Blis Website] http://www.blismobile.com/
This communication is from BlisMobile Media, which is a trading name of
Breeze Tech (UK) Ltd, a company registered in England and Wales with
registered number 06455773. Its registered office is 32 Percy Street,
London W1T 2DE, United Kingdom.
This communication contains information that is confidential and may also
be privileged. It is for the exclusive use of the intended recipient(s). If
you are not the intended recipient(s), please (1) notify i...@blismobile.com by
forwarding this email and delete all copies from your system and (2) note
that disclosure, distribution, copying or use of this communication is
strictly prohibited. Email communications cannot be guaranteed to be secure
or free from error or viruses. All emails sent to or from a Blismobile
email account are securely archived and stored by an external supplier. This
email does not constitute a contractual agreement; such agreements are in
specified contractual or Insertion Order (IO) form only
and exclusively contain all the terms to which Breeze Tech )UK) Ltd will be
bound. To the extent permitted by law, Breeze Tech (UK) Ltd does not accept
any liability for use of or reliance on the contents of this email by any
person save by the intended recipient(s) to the extent agreed in a contract
or Insertion Order.
Opinions, conclusions and other information in this email which have not
been delivered by way of the business of Breeze Tech (UK) Ltd are neither
given nor endorsed by it.
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Duplicate identities - #325
On Sat, Mar 24, 2012 at 10:59 AM, Chuck Remes li...@chuckremes.com wrote: FYI, this patch was rejected by crossroads-io. They prefer it be handled by closing the offending connection rather than silently allowing it to connect but with an auto identity. The problem here is that the protocol is unable to report errors back. (We're running on an amateurish protocol, sadly.) So if you're facing a buggy client, you can't help it in any way. Closing the connection won't help (it'll just retry). Asserting in the server is pathological. Switching to an automatic identity isn't great either. If you're facing a hostile client, switching to an automatic identity is a decent strategy since it'll keep the client busy. -Pieter ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] ZeroMQ 2.1.11 - poll returns errno 11 on ZMQ_SUB sockets
Just an update,
I have also tried adding:
subscriber.setsockopt(ZMQ_SUBSCRIBE, , 0);
so the subscriber will subscribe to all messages.
But this did not help either.
Thanks,
Raj
On 26 March 2012 17:14, Rajalakshmi Iyer r...@blismobile.com wrote:
Hello,
I am trying a simple PUBLISH/SUBSCRIBE setup with ZeroMQ 2.1.11.
Here is the server code:
#include zmq.hpp
#include iostream
using namespace std;
int main(void)
{
try
{
zmq::context_t context(1);
zmq::socket_t publisher(context, ZMQ_PUB);
publisher.bind(tcp://*:5556);
sleep(5);
while(1)
{
char* command = Command;
zmq::message_t msg(command, strlen(command), NULL, NULL);
if (!publisher.send(msg))
{
cout Unable to send message endl;
}
sleep(2);
}
}
catch (zmq::error_t e)
{
cout ZMQ Exception : e.what() endl;
}
return 0;
}
And this is the client code:
#include zmq.hpp
#include iostream
using namespace std;
int main()
{
try
{
zmq::context_t context(1);
cout Serving commands ... endl;
zmq::socket_t subscriber(context, ZMQ_SUB);
subscriber.connect(tcp://localhost:5556);
sleep(2);
while(1)
{
zmq::pollitem_t items[] = {
{(void*)subscriber, 0, ZMQ_POLLIN, 0}
};
int rc = zmq::poll(items, 1, 1);
cout zmq_poll(rc= errno ) endl;
if (items[0].revents ZMQ_POLLIN)
{
zmq::message_t command;
if (!subscriber.recv(command, ZMQ_NOBLOCK))
{
cout Failed to receive command endl;
}
cout Received command: (char*)command.data()
endl;
}
}
}
catch (zmq::error_t e)
{
cout ZMQ Exception : e.what() endl;
}
return 0;
}
The server is sending commands every 2 seconds over the ZMQ_PUB socket.
The client does zmq_poll on the ZMQ_SUB socket in a loop. However, the
client always sees errno 11 on a zmq_poll (which seems to indicate Resource
Temporarily Unavailable).
I have tried to figure out the cause from other posts in the mailing
lists. They seemed to indicate adding a sleep after the bind/connect phase
before starting to send/recv data. Even that has not helped.
Apologies, if this has been answered earlier, but it would be really great
if someone can help me out here.
Thanks,
Raj.
--
Twitter: @Blismobile http://twitter.com/#!/blismobile
BlisMobile Media
32 Percy Street,
London W1T 2DE
www.blismobile.com
[image: BlisMobile] http://www.blismobile.com/[image: Follow on
Twitter]http://twitter.com/#!/blismobile[image:
Blis Website] http://www.blismobile.com/
This communication is from BlisMobile Media, which is a trading name of
Breeze Tech (UK) Ltd, a company registered in England and Wales with
registered number 06455773. Its registered office is 32 Percy Street,
London W1T 2DE, United Kingdom.
This communication contains information that is confidential and may also
be privileged. It is for the exclusive use of the intended recipient(s). If
you are not the intended recipient(s), please (1) notify i...@blismobile.com by
forwarding this email and delete all copies from your system and (2) note
that disclosure, distribution, copying or use of this communication is
strictly prohibited. Email communications cannot be guaranteed to be secure
or free from error or viruses. All emails sent to or from a Blismobile
email account are securely archived and stored by an external supplier. This
email does not constitute a contractual agreement; such agreements are in
specified contractual or Insertion Order (IO) form only
and exclusively contain all the terms to which Breeze Tech )UK) Ltd will be
bound. To the extent permitted by law, Breeze Tech (UK) Ltd does not accept
any liability for use of or reliance on the contents of this email by any
person save by the intended recipient(s) to the extent agreed in a contract
or Insertion Order.
Opinions, conclusions and other information in this email which have not
been delivered by way of the business of Breeze Tech (UK) Ltd are neither
given nor endorsed by it.
___
zeromq-dev mailing list
[zeromq-dev] polling for messages
Hi list, I'm considering using czmq in my single threaded application. However, the application is already using an event driven framework in which a file descriptor has to be registered along with a callback that is invoked when it becomes readable. My question is: can I use the file descriptor returned by zsockopt_fd() in this way, or would this break things when I issue zframe_recv_nowait()? Thanks in advance, Alex ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Event-driven publish-subscribe model
On Mar 26, 2012, at 11:08 AM, Ian Barber wrote: On Mon, Mar 26, 2012 at 4:42 PM, Chuck Remes li...@chuckremes.com wrote: That is a far better mechanism than the built-in HWM mechanism. Since it isn't built in to the library, you would have to add this logic at the application level. Since it *requires* two-way communication between the client and server, you cannot use the PUB/SUB sockets. You would need to use DEALER/ROUTER sockets to handle this communication. Yes, this will be a little more work but it is necessary for the kind of flexibility that your #2 issue requires. I think you can still use PUB/SUB here, if you're doing at-least-one kind of flow control you just need credit from *someone*, not everyone, and you could do that with a simple pull or rep socket - as soon as anyone adds credit, then you push the message out the pub. This is actually pretty much just a send-on-ack sliding window, but with multiple clients. Doesn't need to be a library level, that's an easy bit of code in application land. By doing the checking in app land you also get the opportunity to be clever - for example, rolling up messages if they supersede each other, that kind of thing. Yes, this is true. I answered assuming that the OP wanted to work with just a single set of sockets rather than having a side channel for communicating the credit flows. Either way will work, so it's up to the OP to figure out which way makes more sense to him. cr ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Duplicate identities - #325
Is it possible to provide both behaviors (reject, autoid) switched with a socket option? -Michel On Mon, Mar 26, 2012 at 9:15 AM, Pieter Hintjens p...@imatix.com wrote: On Sat, Mar 24, 2012 at 10:59 AM, Chuck Remes li...@chuckremes.com wrote: FYI, this patch was rejected by crossroads-io. They prefer it be handled by closing the offending connection rather than silently allowing it to connect but with an auto identity. The problem here is that the protocol is unable to report errors back. (We're running on an amateurish protocol, sadly.) So if you're facing a buggy client, you can't help it in any way. Closing the connection won't help (it'll just retry). Asserting in the server is pathological. Switching to an automatic identity isn't great either. If you're facing a hostile client, switching to an automatic identity is a decent strategy since it'll keep the client busy. -Pieter ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Event-driven publish-subscribe model
On Mon, Mar 26, 2012 at 5:53 PM, Chuck Remes li...@chuckremes.com wrote: Yes, this is true. I answered assuming that the OP wanted to work with just a single set of sockets rather than having a side channel for communicating the credit flows. Either way will work, so it's up to the OP to figure out which way makes more sense to him. cr Ack. I've found that I tend to lean to separating message flows over different topologies as I've done more stuff with ZMQ - it seems to fall out of a lot of different requirements. Definitely does cause more config management though to have to deal with extra sockets though, so a fair point. If 3.1 was an option and you wanted a tasty one socket set hack, you can communicate credit via subscribe messages back up stream :) Ian ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Event-driven publish-subscribe model
On Mar 26, 2012, at 11:58 AM, Ian Barber wrote: If 3.1 was an option and you wanted a tasty one socket set hack, you can communicate credit via subscribe messages back up stream :) Absolutely. I would love to add this kind of back channel communication so that we could have credit-based flow control built-in directly to libzmq for all socket types. I'm hoping that some enterprising individual runs with this idea as my C C++ skills are probably not up to the task of such an endeavor. cr ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] zeromq and libuv
On Sunday 25, rektide wrote: Robert, thanks for the reply. I confess to not understanding the issue fully. Are there any workarounds or possibilities you'd want to explore, to test out, as a possible way forwards? I'm just looking to pull whatever constructive guidanc eI can from you (or any others on the ml) on this issue. If you don't have to support Windows, then you can by-pass lib-uv and create an ev_io watcher for zmq socket's FD with lib-ev's default loop. I can't think of away to make zmq work with Window's IOCP without changing zmq's API. On Wed, Mar 21, 2012 at 11:49 PM, Robert G. Jakabosky bo...@sharedrealm.com wrote: On Monday 19, John D. Mitchell wrote: Hmm... Something is very confused in this discussion... The zeromq.node project and the performance problems there have to do that projects's Node.js binding with the zeromq libraries. I.e. it doesn't seem to have much, if anything to do with libuv itself. libuv doesn't provide a way to register a FD for io events. It only allows starting an async read/write, with notification of when the read/write finishes. The FD for a zmq socket can't be read from by the application, so it can't be used with libev. When using zmq sockets in an event loop, one has to register for notification of read events on the FD and then check for the true events with the ZMQ_EVENTS socket option. I think the main reason libuv is designed like that is because that is how Window's IOCP works. As far as I can tell zeromq.node only works on *nix systems (where libuv uses libev as the event loop backend). zeromq.node has to bypass libuv and use a libev IO watcher directly to work. The Luvit [1] (LuaJIT + libuv, like nodejs but with Lua instead of Javascript) has the same problem with trying to use the Lua bindings [2] for zmq. 1. https://github.com/luvit/luvit 2. https://github.com/Neopallium/lua-zmq -- Robert G. Jakabosky ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev -- Robert G. Jakabosky ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] polling for messages
Yes, it should work fine. On Mar 26, 2012 11:44 AM, Alex Zeffertt azeffe...@cambridgesys.com wrote: Hi list, I'm considering using czmq in my single threaded application. However, the application is already using an event driven framework in which a file descriptor has to be registered along with a callback that is invoked when it becomes readable. My question is: can I use the file descriptor returned by zsockopt_fd() in this way, or would this break things when I issue zframe_recv_nowait()? Thanks in advance, Alex ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Duplicate identities - #325
Sure, but we need a patch for the reject behavior then. On Mar 26, 2012 11:55 AM, Michel Pelletier pelletier.mic...@gmail.com wrote: Is it possible to provide both behaviors (reject, autoid) switched with a socket option? -Michel On Mon, Mar 26, 2012 at 9:15 AM, Pieter Hintjens p...@imatix.com wrote: On Sat, Mar 24, 2012 at 10:59 AM, Chuck Remes li...@chuckremes.com wrote: FYI, this patch was rejected by crossroads-io. They prefer it be handled by closing the offending connection rather than silently allowing it to connect but with an auto identity. The problem here is that the protocol is unable to report errors back. (We're running on an amateurish protocol, sadly.) So if you're facing a buggy client, you can't help it in any way. Closing the connection won't help (it'll just retry). Asserting in the server is pathological. Switching to an automatic identity isn't great either. If you're facing a hostile client, switching to an automatic identity is a decent strategy since it'll keep the client busy. -Pieter ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] polling for messages
Hi Alex, That'll be fine - the fd events are edge-triggered and would work with any reactor implementation. Just remember to read until completion as the notification can be for more than one message (zmq protocol is framed) as well. Failure to do so will result in further notifications not being triggered. - Lourens On Monday, March 26, 2012, Pieter Hintjens piet...@gmail.com wrote: Yes, it should work fine. On Mar 26, 2012 11:44 AM, Alex Zeffertt azeffe...@cambridgesys.com wrote: Hi list, I'm considering using czmq in my single threaded application. However, the application is already using an event driven framework in which a file descriptor has to be registered along with a callback that is invoked when it becomes readable. My question is: can I use the file descriptor returned by zsockopt_fd() in this way, or would this break things when I issue zframe_recv_nowait()? Thanks in advance, Alex ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] FW: Help With Regard To the ZMQ Forwarder
On Sun, Mar 25, 2012 at 23:05, Ronald Swain proj_symb...@live.com wrote:
Hello Cornelius,
Thanks for pointing that out. I Used ProcessDevice and it worked the way i
want, but was not able to understand, some questions:
1) Does ProcessDevice takes care of doing the clean up work ??
2) Is there way i can close the processdevice or i can check if its
already running ??
I tried terminating the python interpreter , but as that is a different
process i dont this thats going to killed with my old way.
ProcessDevice simply sets up sockets and calls zmq.device() via
multiprocessing, which in turn uses fork (or a filthy broken mess on
Windows). The underlying Process (or Thread) is available as
Device.launcher. You can query that with dev.launcher.is_alive(),
dev.launcher.pid, etc. The appropriate references for these objects are
the stdlib docs for
multiprocessing.Processhttp://docs.python.org/library/multiprocessing.html#multiprocessing.Processand
threading.Threadhttp://docs.python.org/library/threading.html#threading.Threadrespectively.
The devices default to using `daemon=True`, which means that Python will
try to terminate them on a clean exit (forcefully terminating the parent
doesn't give it the chance to cleanup, and ProcessDevices will continue to
run as orphans in this case).
-MinRK
Thanks,
Ronald
--
Date: Fri, 23 Mar 2012 10:35:02 -0500
From: cornelius.to...@gmail.com
To: zeromq-dev@lists.zeromq.org
Subject: Re: [zeromq-dev] FW: Help With Regard To the ZMQ Forwarder
Check out the documentation on devices and pyzmq as well as the zguide
... whenever i start a zmq.device(zmq.Forwarder), this blocks my
interpreter, so is there any way of starting a forwarder always as a
different process or any ways of making it non-blocking.
http://zeromq.github.com/pyzmq/devices.html
http://zguide.zeromq.org/page:all#Intermediates-and-Devices
--
Cornelius Toole
Sent with Sparrow http://www.sparrowmailapp.com
On Friday, March 23, 2012 at 12:19 AM, Ronald Swain wrote:
Hello All,
I had one more small question, whenever i start a
zmq.device(zmq.Forwarder), this blocks my interpreter, so is there any way
of starting a forwarder always as a different process or any ways of making
it non-blocking.
Hope my question is clear.
Thanks Regards,
Ronald
--
From: proj_symb...@live.com
To: zeromq-dev@lists.zeromq.org
Date: Tue, 20 Mar 2012 12:27:24 +0530
Subject: Re: [zeromq-dev] FW: Help With Regard To the ZMQ Forwarder
Ok the first question is i am back to zero, so now when my scripts are
working i am able to send messages, the question of cleaning comes.
i am using the following code for the forwarder and i think thats correct,
but when i kill the process which is running the forwarder the SIGTERM
signal is not fired i guess and incoming, outgoing and zmq context is not
cleaned properly.
code:
import zmq
import signal
# this method handles the termination of the app
# this handler is responsible for properly cleaning of the ports
def termSignalHandler(signum,frame):
incoming.close()
outgoing.close()
context.term()
print(termSignal Handler Called)
def startForwarder():
context = zmq.Context(1)
incoming = context.socket(zmq.SUB)
outgoing = context.socket(zmq.PUB)
try:
incoming.connect(tcp://127.0.0.1:5559);
incoming.setsockopt(zmq.SUBSCRIBE, )
except:
print(incoming socket is already open)
try:
outgoing.bind('tcp://127.0.0.1:4449')
except:
print(outgoing socket is open)
zmq.device(zmq.FORWARDER, incoming, outgoing)
signal.signal(signal.SIGTERM,termSignalHandler)
startForwarder()
a proper copy can be found at https://gist.github.com/2132163, can you
guys tell me what wrong i am doing here.
--
From: proj_symb...@live.com
To: zeromq-dev@lists.zeromq.org
Date: Tue, 20 Mar 2012 11:23:44 +0530
Subject: Re: [zeromq-dev] FW: Help With Regard To the ZMQ Forwarder
Hello Cornelius and Justin,
I was at last able to get my forwarder to work and it was my mistake.
My sender code which was like:
def sendMessage():
context = zmq.Context()
sender = context.socket(zmq.PUB)
sender.bind('tcp://127.0.0.1:5558')
sender.send(This is the sender)
was the actual problem, i got to know from my reading that call to
socket.bind should happen only once, so i changed my sender code to
something like:
socket = context.socket(zmq.PUB)
socket.bind('tcp://127.0.0.1:4443')
def sendMessage(socket,message):
try:
socket.send(message)
except:
print Unable To Send Message
and than i was able to send messages but this gets me into two three more
problems, i hope you guys will help me out is giving me some
ideas about this.
--
Date: Mon, 19 Mar 2012 13:18:59 -0500
From:
[zeromq-dev] REQ/REP for possibly async comm?
I have two processes that need to exchange data. Process A has to perform time-critical work, but also needs to obtain some configuration data from process B. Initially, I considered using REQ/REP sockets since it's a roundtrip query, doing something like on A: initialize work unit if (previously sent request) check response from B if (still no response after 30 seconds) send config request else send config request finish work unit The problem I see with REQ/REP is that they work in lockstep and that B may not reply in a reliable fashion. Thus, I wouldn't be able to send a new config request until I heard back from B. Is it better to switch to PUB/SUB or DEALER/REP scheme here? -Andrei ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] REQ/REP for possibly async comm?
I wondered about this as well. I want to make a worker that does remote method calls, and it would expose a REQ/REP interface that other workers would use whenever they want to make a remote call. Since the work bottleneck is time (waiting for remote server responses), I'd prefer the worker use asynchronous sockets rather than waste many threads on synchronous sockets that spend the majority of their time idle. However, I also want to be able to respond to requests out of sequence (e.g. receive requests A, B, but then respond to B before A), and I think this breaks the REQ/REP pattern. On Monday, March 26, 2012 03:21:08 PM Andrei Zmievski wrote: I have two processes that need to exchange data. Process A has to perform time-critical work, but also needs to obtain some configuration data from process B. Initially, I considered using REQ/REP sockets since it's a roundtrip query, doing something like on A: initialize work unit if (previously sent request) check response from B if (still no response after 30 seconds) send config request else send config request finish work unit The problem I see with REQ/REP is that they work in lockstep and that B may not reply in a reliable fashion. Thus, I wouldn't be able to send a new config request until I heard back from B. Is it better to switch to PUB/SUB or DEALER/REP scheme here? -Andrei ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] REQ/REP for possibly async comm?
You can use non-blocking ROUTER on a poll loop and REQ or REP on the other end, that way you can handle requests and their responses as they come and as you compute them, respectively. Whether to use REQ or REP only matters for whichever end starts the conversation. The LRU pattern documented in the guide has a good implementation of this pattern with REQ-ROUTER-REP. It doesn't sound like you need the actual LRU part of the pattern, just the ROUTER-REQ part. See the code for an example how to use a ROUTER socket in non-blocking mode in a poll loop to handle out of order patterns like A then B requests but you respond B then A. Start here and read it a few times. :) http://zguide.zeromq.org/page:all#Least-Recently-Used-Routing-LRU-Pattern -Michel On Mon, Mar 26, 2012 at 4:08 PM, Justin Karneges jus...@affinix.com wrote: I wondered about this as well. I want to make a worker that does remote method calls, and it would expose a REQ/REP interface that other workers would use whenever they want to make a remote call. Since the work bottleneck is time (waiting for remote server responses), I'd prefer the worker use asynchronous sockets rather than waste many threads on synchronous sockets that spend the majority of their time idle. However, I also want to be able to respond to requests out of sequence (e.g. receive requests A, B, but then respond to B before A), and I think this breaks the REQ/REP pattern. On Monday, March 26, 2012 03:21:08 PM Andrei Zmievski wrote: I have two processes that need to exchange data. Process A has to perform time-critical work, but also needs to obtain some configuration data from process B. Initially, I considered using REQ/REP sockets since it's a roundtrip query, doing something like on A: initialize work unit if (previously sent request) check response from B if (still no response after 30 seconds) send config request else send config request finish work unit The problem I see with REQ/REP is that they work in lockstep and that B may not reply in a reliable fashion. Thus, I wouldn't be able to send a new config request until I heard back from B. Is it better to switch to PUB/SUB or DEALER/REP scheme here? -Andrei ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
