Re: [zeromq-dev] Passing a python object (PyZMQ)
On Jun 8, 2015, at 01:35, Arnaud Loonstra arn...@sphaero.org wrote:
On 06/06/2015 01:52 AM, MinRK wrote:
Without using ctypes, you could pass the objects through a namespace:
|# shared namespace
ns = {}
# sender
ns[id(obj)] = obj
pipe_out.send(struct.pack(b'Q',1))
# receiver
id_bytes = pipe_in.recv()
obj_id = struct.unpack(b'Q', id_bytes)[0]
obj = ns.pop(obj_id)
|
The ctypes cast approach doesn’t hold a reference to the object while
it’s in transit, so it’s possible for the restoration to fail if the
object has been garbage collected in between send/recv.
You're right I ended up doing something very similar to workaround early
garbage collection but this is much nicer. However it only works between
a producer and consumer thread, not more. So every producer thread needs
its own namespace.
I'm not sure why. You should only need one namespace per process.
-MinRK
Rg,
Arnaud
--
w: http://www.sphaero.org
t: http://twitter.com/sphaero
g: http://github.com/sphaero
i: freenode: sphaero_z25
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Z/J-yre+Android
When I tested Jyre last time it was able to communicate between multiple devices both with Jeromq and JZMQ. As both libraries are interchangeable, you can start with any of them you're familiar with. Jeromq is a bit easier to deploy if you want to use Jyre on Android. Thanks Min On Feb 16, 2015, at 9:04 AM, Utsav Drolia utsavdro...@gmail.com wrote: Hi, So if I start down the path of using Jyre, is it functional enough for two devices to talk to each other using the current version of Jyre? If I want to develop/use it further, is JeroMQ a better choice or JZMQ? Thanks! Utsav On Feb 16, 2015 10:20 AM, Min mini...@gmail.com wrote: Hello, Jyre was designed to support JeroMQ or JZMQ seamlessly. As far as I know Zyre are not much depends on zeromq advanced features so there would be very little or zero effort to support Jyre with JeroMQ and JZMQ. So 1, 2 are basically the same. Based on active development in Zyre these days, Jyre looks too far from the current state of Zyre and might not communicate each other at all. The third option is possible but it doesn't look like quite good structure. In my opinion, make Jyre compatible with Zyre could be better approach as the third option might all require similar amount of efforts. Unfortunately I'm not certain when either of Jyre (on JeroMQ or JZMQ) and JNI Zyre is available as there're not many contributors on this project. Thanks Min On Wed, Feb 11, 2015 at 8:52 AM, Utsav Drolia utsavdro...@gmail.com wrote: Hi Min, Thanks for the response! When you say out of date, could you quantify or qualitatively tell me how out-of-date it is? I’d like to use Zyre from Java (most likely on Android). There are a couple of ways to do that : 1. Jyre+JeroMQ. Use whatever is available+build Jyre upto Zyre. However, this requires that JeroMQ is also up to date. 2. Jyre+JZMQ. Use whatever is available+build Jyre upto Zyre. This requires that the JNI interface (JZMQ) is also up to date. 3. JNI(Zyre)+JZMQ. This looks expensive in the sense that I will have to access both zeroMQ and Zyre through JNI. However, given that Zyre and JZMQ depend on libzmq, would the software structure be like this? Application / \ JNI(Zyre) JZMQ \ / libzmq Could you please tell me the pros and cons of each, especially with respect to the maturity/updated-ness of projects? On that note, is there a JNI-wrappered Zyre available? Thanks! Utsav Date: Tue, 10 Feb 2015 10:40:02 -0800 From: Yu Dongmin mini...@gmail.com Subject: Re: [zeromq-dev] Z/J-yre+Android To: ZeroMQ development list zeromq-dev@lists.zeromq.org Message-ID: 7790b6c0-0a28-4d56-a365-638b4af44...@gmail.com Content-Type: text/plain; charset=us-ascii Jyre was intended to make the same level of Zyre. But it is out-of-date and the community might not have enough resource to catch up with Zyre. We still need more volunteers for this project. Zyre+Jzmq(JNI) might be good start and could be migrated later. Thanks Min On Feb 10, 2015, at 8:43 AM, Utsav Drolia utsavdro...@gmail.com wrote: Hello Everyone, Newbie here wrt 0MQ in general, with some questions on status of JeroMQ+Jyre. Is Jyre on JeroMQ at the same level of progress as Zyre? And are there any pointers to have JeroMQ+Jyre work on Android devices? Or is it better to go the Zyre+JNI way? I always hesitate to use JNI given the lore about it eating up too many CPU cycles. I also took a look at the edgenets github page - Hydra, drops etc. Any Java implementations to be expected in the edgenet project? Would make using it with Android a lot easier. Thanks! Utsav ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Z/J-yre+Android
Hello, Jyre was designed to support JeroMQ or JZMQ seamlessly. As far as I know Zyre are not much depends on zeromq advanced features so there would be very little or zero effort to support Jyre with JeroMQ and JZMQ. So 1, 2 are basically the same. Based on active development in Zyre these days, Jyre looks too far from the current state of Zyre and might not communicate each other at all. The third option is possible but it doesn't look like quite good structure. In my opinion, make Jyre compatible with Zyre could be better approach as the third option might all require similar amount of efforts. Unfortunately I'm not certain when either of Jyre (on JeroMQ or JZMQ) and JNI Zyre is available as there're not many contributors on this project. Thanks Min On Wed, Feb 11, 2015 at 8:52 AM, Utsav Drolia utsavdro...@gmail.com wrote: Hi Min, Thanks for the response! When you say out of date, could you quantify or qualitatively tell me how out-of-date it is? I’d like to use Zyre from Java (most likely on Android). There are a couple of ways to do that : 1. Jyre+JeroMQ. Use whatever is available+build Jyre upto Zyre. However, this requires that JeroMQ is also up to date. 2. Jyre+JZMQ. Use whatever is available+build Jyre upto Zyre. This requires that the JNI interface (JZMQ) is also up to date. 3. JNI(Zyre)+JZMQ. This looks expensive in the sense that I will have to access both zeroMQ and Zyre through JNI. However, given that Zyre and JZMQ depend on libzmq, would the software structure be like this? Application / \ JNI(Zyre) JZMQ \ / libzmq Could you please tell me the pros and cons of each, especially with respect to the maturity/updated-ness of projects? On that note, is there a JNI-wrappered Zyre available? Thanks! Utsav Date: Tue, 10 Feb 2015 10:40:02 -0800 From: Yu Dongmin mini...@gmail.com Subject: Re: [zeromq-dev] Z/J-yre+Android To: ZeroMQ development list zeromq-dev@lists.zeromq.org Message-ID: 7790b6c0-0a28-4d56-a365-638b4af44...@gmail.com Content-Type: text/plain; charset=us-ascii Jyre was intended to make the same level of Zyre. But it is out-of-date and the community might not have enough resource to catch up with Zyre. We still need more volunteers for this project. Zyre+Jzmq(JNI) might be good start and could be migrated later. Thanks Min On Feb 10, 2015, at 8:43 AM, Utsav Drolia utsavdro...@gmail.com wrote: Hello Everyone, Newbie here wrt 0MQ in general, with some questions on status of JeroMQ+Jyre. Is Jyre on JeroMQ at the same level of progress as Zyre? And are there any pointers to have JeroMQ+Jyre work on Android devices? Or is it better to go the Zyre+JNI way? I always hesitate to use JNI given the lore about it eating up too many CPU cycles. I also took a look at the edgenets github page - Hydra, drops etc. Any Java implementations to be expected in the edgenet project? Would make using it with Android a lot easier. Thanks! Utsav ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Serializing messages
Hi, My personal preference is MessagePack with gzip, but there would more than dozens of other options. Try picking up several libraries and compare the performances. Thanks. Min On Dec 3, 2014, at 1:04 AM, George ggi...@ilabafrica.co.ke wrote: Hi, It seems like sending multi-part messages via zstr_sendx has severely affected my through put. When using zstr_send the throughput is about 300,000 messages per second but with zstr_sendx with about 6 messages that brings it down to about 30,000 messages per second. I want to send all the multi-part messages all at once as one message with zstr_send so that I can improve performance. Which is the best way to serialize these messages. I could append a semi-colon at the end of each message but this could easily be messed up by a message string that already has a semi-colon in it. ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] ZMQ Java - Best practice to clean when exiting
context.destroy might be ok if you only work with jeromq. But still the best practice is calling close and term explicitly at the end of your logic for future reference and same style on other platforms. Thanks Min On Nov 18, 2014, at 1:26 AM, rails stammail...@gmail.com wrote: Thanks. But if I look at the jeromq code - I see inside ZContext.destroy a call for destroySocket (to each socket ) which s.setLinger(linger); s.close(); sockets.remove(s); And then It even calls sockets.clear(); and context.term(); So - Am i safe by calling the java (jeroMQ ) implementation of context.destroy On Tue, Nov 18, 2014 at 10:25 AM, Pieter Hintjens p...@imatix.com wrote: 1. set linger = 0 on all sockets 2. close all sockets 3. terminate context On Tue, Nov 18, 2014 at 9:10 AM, rails stammail...@gmail.com wrote: In ZMQ zguide they advice to always clean up when task ends. I am a bit confused regarding the best practice. First ... even if you close all sockets, zmq_ctx_destroy() will by default wait forever ... Finally, destroy the context ... This will cause any blocking receives or polls or sends in attached threads to return with an error. Catch that error, and then set linger on, and close sockets in that thread, So do I need to close? destroy? all sockets before I destroy the context? Looking at the context.destroy() I see it calls linger and close on each socket. How should I terminate my ZMQ resources? ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] ZeroMQ multiple sockets connection in Java
If you want use PUB-SUB, please refer Getting the Message Out http://zguide.zeromq.org/page:all#Getting-the-Message-Out http://zguide.zeromq.org/java:wuserver (run single instance) http://zguide.zeromq.org/java:wuclient (run multiple instances) Or If you want to use PUSH-PULL Divide and Conquer http://zguide.zeromq.org/page:all#Divide-and-Conquer http://zguide.zeromq.org/java:taskvent (run single instance) http://zguide.zeromq.org/java:taskwork (run multiple instance) http://zguide.zeromq.org/java:tasksink (run single instance) On Sat, Nov 15, 2014 at 1:13 PM, Gregg Irwin gr...@pointillistic.com wrote: CP Yes I have read the guide and in my case I have multiple ports CP opened on a socket and I need send the data on all those four ports. CP Can we not do this? If not then why we cannot do this? sorry for CP my ignorance I don't use Jero, but you should just be able to call connect multiple times for your socket, once for each endpoint. -- Gregg ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] PyZMQ with Libsodium on Windows?
For pyzmq, it must work with VC9 (VS2008), for Python 2.7. -MinRK On Oct 9, 2014, at 18:08, Steven McCoy steven.mc...@miru.hk wrote: I think it is easier now as they have more support than just MSVC2013 (C99 compat) when crypto was added to ZeroMQ. On 9 October 2014 17:47, MinRK benjami...@gmail.com wrote: It's not bundled simply because I couldn't build it on Windows. If you can come up with a simple fix for building bundled libsodium, then I would bundle libsodium on Windows. -MinRK On Thu, Oct 9, 2014 at 1:48 PM, André Caron andre.l.ca...@gmail.com wrote: Hi there! I'm trying to secure some PyZMQ communications with the curve security features. The code seems straightforward, but when I call `zmq.auth.create_certificates()` to generate some keys, I get the following exception: zmq.error.ZMQError: Not supported This seems to be due to the absence of libsodium. In the release notes for PyZMQ 14.1.0, I can see that libsodium is not bundled with PyZMQ on Windows. First, I'm curious to know why it's not bundled on Windows. If it's simply a matter of finding a volunteer, maybe I can pitch in? Second, I'd like to know if there's any known procedure to get it running on Windows? Thanks! André ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Zper
Hello, I'm the only maintainer and user for the project, LOL. I've used it to collect server metrics and seems to be working fine until now for about one hundred servers. Thanks Min On Sep 24, 2014, at 3:59 PM, Mohit Anchlia mohitanch...@gmail.com wrote: I came across ZPer project in github and it looks interesting as it seems to provide much of the functionalities. I was wondering if other people have worked with ZPer and what the experience has been with ZPer? https://github.com/miniway/zper ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Zper
ZPer is a kind of replica of Kafka. Technically you can use request-reply pattern with ZPer but publish-subscribe pattern is much fit for its purpose. Publishers can send message to ZPer and forget it. Then subscribers fetch the messages asynchronously. One of the biggest strength of Kafka structure is subscribers can consume messages from at any point and multiple times. For both publisher and subscribers, DEALER is the best socket type for it. But you might also use PUSH socket for publisher. Thanks Min From: Mohit Anchlia mohitanch...@gmail.com Reply: ZeroMQ development list zeromq-dev@lists.zeromq.org Date: September 24, 2014 at 5:06:50 PM To: ZeroMQ development list zeromq-dev@lists.zeromq.org Subject: Re: [zeromq-dev] Zper I am new to ZeroMQ and still trying to understand the concepts. I looked at the ZPer code and it sounds like Writer thread has 2 sockets and Reader thread has 2 ROUTER sockets. What I don't understand is what I as a client need to do to 1) request-repy pattern 2) publish subscribe pattern? Do I need to connect as a DEALER over tcp to the frontend port of writer thread and then connect another socket of type REQ to DEALER? Also, can you share what are the main challenges you've faced so far. Thanks! On Wed, Sep 24, 2014 at 4:19 PM, Min mini...@gmail.com wrote: Hello, I'm the only maintainer and user for the project, LOL. I've used it to collect server metrics and seems to be working fine until now for about one hundred servers. Thanks Min On Sep 24, 2014, at 3:59 PM, Mohit Anchlia mohitanch...@gmail.com wrote: I came across ZPer project in github and it looks interesting as it seems to provide much of the functionalities. I was wondering if other people have worked with ZPer and what the experience has been with ZPer? https://github.com/miniway/zper ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] about inproc mode of jeromq
No, jeromq is not an in-memery reference passing queue like the disruptor. If you’re building a message based application, zeromq would be perfect solution for you, and you could choose or mix your prefer languages. But if you need a fast java only in-memory queue, the disruptor might be a better solution. Thanks Min From: 夏巍巍 duchipore...@gmail.com Reply: ZeroMQ development list zeromq-dev@lists.zeromq.org Date: June 9, 2014 at 2:15:13 PM To: zeromq-dev@lists.zeromq.org zeromq-dev@lists.zeromq.org Subject: [zeromq-dev] about inproc mode of jeromq I am using jeromq recently. And I find that I still need to serialize and deserialize data to make a msg when using the inproc mode of jeromq. Could I add the Object of data to a msg directly ? So that I don't need to serialize and deserialize the Object of data like using disruptor. If posible, how to implement it? Thank you very much. ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Jyre Zyre
ZMsg and Frame don’t involve JNI instances. So it should be GCed when they’re get out of scope. ‘destroy' was insert to have common semantics between czmq and jzmq. At the future, it could be removed or obsoleted to reduce misunderstanding. Thanks Min From: techbird techb...@birdsoft.co.uk Reply: ZeroMQ development list zeromq-dev@lists.zeromq.org Date: April 21, 2014 at 11:37:17 AM To: ZeroMQ development list zeromq-dev@lists.zeromq.org Subject: Re: [zeromq-dev] Jyre Zyre On 21/04/14 18:38, Pieter Hintjens wrote: I don't think so. On Mon, Apr 21, 2014 at 4:40 PM, techbird techb...@birdsoft.co.uk wrote: Is there any reason not to change Jyre's pom.xml to use: dependency groupIdorg.zeromq/groupId artifactIdjzmq4/artifactId version4.0.0-SNAPSHOT/version /dependency rather than, the current, version 3? On 21/04/14 09:28, techbird wrote: I'll update the Java code base to use a java.util.UUID for the identity - thanks Pieter. On 20/04/14 23:05, Pieter Hintjens wrote: The correct representation is binary. On Sun, Apr 20, 2014 at 8:52 PM, techbird techb...@birdsoft.co.uk wrote: Hello Pieter. Could you provide some direction. After making the protocol changes - Jyre and Zyre clients time out. I've work this through and discovered that... Jyre uses a string for the sender identity UUID, and Zyre uses a binary representation. This prevents the UUID from being un-marshalled correctly. And the peers timeout. Which is preferred binary or string representation for the peer's UUID? On 19/04/14 20:24, techbird wrote: Hiya Pieter. thank you - I've started taking a look, made a little progress updating the protocol. Problem is I've had to stop to make supper, fillet a large salmon, and now I have an hour back on the computer (with a glass of wine). Got the Jyre talking to Zyre now... sorted out the differences String vs lString etc. Zyre is timing out now, so I'll read further and see what's what! :) Thank you, by the way I do code 'C', Python and Java ... but it's nolonger my day job so I am not as quick as some. On 19/04/14 19:25, Pieter Hintjens wrote: techbird, if you make a start and come back with issues you hit, we'll help you. On Sat, Apr 19, 2014 at 4:10 PM, techbird techb...@birdsoft.co.uk wrote: Hiya Pieter. Of course you're right. - better to work in the present. The other option is to write my consumer using Pyre, but my other components live in Camel/Karaf. So maybe it makes sense for me to stay in the world of Java. I guess a lack of knowledge and time makes me apprehensive. Pushing that aside - I'll start looking at the code. Thank you Pieter :) Like everyone else my problems are a lo On 19/04/14 13:18, Pieter Hintjens wrote: My advice would be to take the latest ZRE protocol spec (RFC 36) and upgrade Jyre to work with that. It'll teach you quite a lot about the details and it's easy to test against Zyre. Otherwise you check out an older version of Zyre from github, from the moment we forked off Jyre. The stacks are fun... I'm using Zyre in some projects and it does make life simple. -Pieter On Sat, Apr 19, 2014 at 1:08 PM, techbird techb...@birdsoft.co.uk wrote: Thank you kindly for your support yesterday. Jyre should talk to Zyre and implement the same functionality (it will need some updates since Zyre has evolved a little) I gave it a go. Zyre protocols have evolved away from the trunk Jyre - but independently both C and Java implementations work. As the protocols, beacon ports etc are different Jyre clients do not communicate with trunk zyre and visa-vera. Is there a earlier version of Zyre I could use which is compatible with trunk Jyre? I would like to write a 'producer' in C, and competing consumer(s) in Java. As an aside - I am only 2 days into 0MQ, and - my jaw is dropping - this project is amazing. Thank you so much for so much wisdom translated into an incredible set of integration stacks. Kind regards. ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev
Re: [zeromq-dev] Is the JeroMQ available via Maven have the issue #34 fix?
https://github.com/zeromq/jeromq/commits/master has the commit history. I hope it might help. Thanks Min On March 2, 2014 at 4:09:35 PM, Mark Wright (markscottwri...@gmail.com) wrote: I'm using gradle to build a jeromq project with the following dependency - compile group: 'org.zeromq', name: 'jeromq', version: '0.3.2' I have an inproc thread that serves messages from client threads, and I've noticed that if I don't bind() it in the main thread, or put a sleep() before attempting to connect to it, my code hangs. That looks like this commit (https://github.com/zeromq/jeromq/pull/35). Has that made it into 0.3.2? I don't know github well enough to be able to track that - it was fixed over a year ago, so it seems like it should, but I have no idea how to ask git when something got pushed to the maven repository. -- Mark Wright markscottwri...@gmail.com ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] PyZMQ recv() acts crazy on one machine
what is zmq.zmq_version() on each? -MinRK On Dec 29, 2013, at 0:01, Thomas Johnson thomas.j.john...@gmail.com wrote: Consider the following simple program: #CUT HERE #!/usr/bin/python import zmq context_push = zmq.Context(1) socket_push = context_push.socket(zmq.PUSH) context_pull = zmq.Context(1) socket_pull = context_pull.socket(zmq.PULL) socket_pull.bind(tcp://127.0.0.1:2000) socket_push.connect(tcp://127.0.0.1:2000) socket_push.send(Hello) message = socket_pull.recv() print Got message: '%s' % message #CUT HERE On machine A, as expected, it prints: Got message: 'Hello' On machine B, it prints: Got message: '' Separately from this program, Machine B also immediately raises a Resource temporarily unavailable if I create a PULL socket and do a (blocking) socket.recv() call. Strangely, it seems to hang if I do a socket.recv(flags=zmq.NOBLOCK), almost as if the flag was being flipped. Both A and B are Ubuntu precise, using pyzmq 14.0.1. B happens to have its native language set as Russian, if that matters. Any idea what is going on or what I'm doing wrong? ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] A security authentication module for pyzmq
By all means! A PR adding that as zmq.auth would be great. -MinRK On Dec 8, 2013, at 5:07, Chris Laws clawsi...@gmail.com wrote: I was wanting to use ZMQ's security features in my pyzmq applications. As far as I can tell the security API in pyzmq is pretty low level (sockopts and write your own ZAP handler). Hopefully I have not overlooked an existing module that already does what I want. Anyway, I am familiar with czmq and what I think is an elegantly designed zauth module. So I created a security authentication module for pyzmq that is modelled on zauth from czmq. I have posted the implementation in the following gist: https://gist.github.com/claws/7856874 It pretty much does what zauth does - easily configurable ZAP management. The passing tests indicate that it appears to function properly. I think this authenticator module would be a useful addition to pyzmq. Are there any pyzmq users who think this might be of use to them? If there is interest I'll turn it into a pyzmq pull request. Regards, Chris ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] JeroMQ use
Hi Albert, We have a plan to release 0.3 soon. For the time being, please use 0.3.0-SNAPSHOT from the maven repository. Or you can build the jar by yourself with the github repository. Thanks Min 2013. 9. 6. 오전 3:54 St. Aubin, Albert (US SSA) albert.stau...@baesystems.com 작성: I have been looking at JeroMq to use with a Java application. Where would be the best place to start for stability, the release or the current Git Repo Snapshot? I noticed the last release was 9 months ago will there be another one anytime soon? I am using TCP connections and the Pub/Sub examples. Thank you, Albert ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Is ZMQ_PLAIN authentication supposed to do anything?
On Sep 2, 2013, at 9:25, Pieter Hintjens p...@imatix.com wrote: MinRK, I've just pushed a patch that fixes authentication for PLAIN and CURVE, and updated the test cases to match. It all works as expected... :-) One thing about CURVE authentication; client keys are passed to the ZAP handler as 32 binary bytes. I'm wondering whether it would be nicer to pass Z85 text strings instead, as everything else in ZAP is text. I expect keysstored in databases and files to be in Z85, not binary... any thoughts? I would leave it as bytes, personally. To me, z85 is convenience format for humans / text-only storage. If the keystore stores keys that are not raw, I would expect the zap_handler to be responsible for the conversions. -Pieter On Sun, Sep 1, 2013 at 7:14 PM, Pieter Hintjens p...@imatix.com wrote: That seems the simplest and cleanest result. We'll get authentication failed events via the ZAP handler, and we might see connection failed event at the client side too (via context monitor) but these should be invisible to message processing. -Pieter On Fri, Aug 30, 2013 at 11:50 PM, MinRK benjami...@gmail.com wrote: On Fri, Aug 30, 2013 at 1:37 PM, Pieter Hintjens p...@imatix.com wrote: On Thu, Aug 29, 2013 at 1:32 AM, MinRK benjami...@gmail.com wrote: Thanks. By closed, you mean the connecting peer (client) should be closed, or the inner pipe on the server side? What should be the user-visible symptoms of failed authentication, both on the client side and the server side, if any? I'm looking to add a failed-auth test to test_security, but it is unclear to me what the expected behavior is. Is the symptom only that messages sent do not arrive, or should sending a message not succeed in the first place? I think the net result of a failed authentication should be the same as if there was no network connection; no pipes created on either side of the connection, and no route to or from the unauthenticated client. Thanks - so as far as the connecter is concerned, it is as if the peer is unavailable, and for the binder, it is as if nobody connected. -Pieter ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] pyzmq poll pull request results
I wouldn't actually recommend using the green poller, it seems to miss the point. I would use a plain greenlet for each socket if possible. -MinRK On Jun 25, 2013, at 11:04, Brian Knox brian.k...@neomailbox.net wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 (sorry about the new thread - was working on moving my email to a different provider in the middle of this) Min - I merged the poll request on my little laptop and gave it a spin. Before your pull request: mps: 28066 mps: 28116 mps: 29745 mps: 28912 mps: 28679 after your pull request: mps: 78138 mps: 78364 mps: 78411 mps: 78065 mps: 78175 Excellent. By the way, just for fun I imported zmq.green instead of zmq without changing anything else mps: 24738 mps: 24779 mps: 24759 mps: 24630 mps: 24702 And it's performance is well below straight zmq Poller. Thanks for the quick patch! Brian -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) Comment: Using GnuPG with undefined - http://www.enigmail.net/ iQEcBAEBAgAGBQJRybGWAAoJEAIP2j2QzUp7TqYH/3RFEE697uGGgWNSQhC1z+ik oOC3FQnnt1c/y5QUYZOIRyFl4LBZF0PL4u0Awy8Yxaqk7jJeM5wRkpa9cXsMC2xq yr9JhA3RGOhKsyExd9RGuaZzNKB4G5N+NTGYpDTHNSXTkFysBUWoxNsJYOs7HeC+ GMblX+6FDzGbg8jYIULTIg8BPbLRen2n3wtROclpzVKrjoN0k91Duvon4VrJb1fp nZPGYqrEFlm/a3aTks/Mhh3wSq8yqt7AHD1qC7JqC4Yb262GjI0XCl1BAvzBcKCx Dj/9VpKYnesWbmrx/RPnxkkSdOgqJ7zIbhAGOFAEWyRI55WZGMkKUVQqWRfebRQ= =IOv1 -END PGP SIGNATURE- ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] pyzmq poller performance
Thanks for the report, I will look into whether I have introduced a performance degradation in the last few iterations. -MinRK On Jun 24, 2013, at 11:27, Brian Knox bri...@talksum.com wrote: It's been awhile since I've used pyzmq, and I'm running into a performance issue using Poller.poll(). With a simple blocking recv() in a while True loop, I get ~ 300k messages a second. With the simplest case of the same code using a poll on the socket, I get ~ 30k messages a second. (pyzeromq)taotetek@Moya:~/src/performance_example$ python ./receiver.py mps: 226049 mps: 348582 mps: 271728 mps: 300389 mps: 411059 mps: 276749 (pyzeromq)taotetek@Moya:~/src/performance_example$ python ./poll_receiver.py mps: 28066 mps: 28116 mps: 29745 mps: 28912 mps: 28679 I don't remember a 10x performance degradation last time I used a poller - but perhaps I'm remembering something wrong, or making a painfully obvious error in my code .. I've been using czmq from C mostly for the last while. I'm using pyzmq from git master, and the latest Cython (0.19.1) with libzmq 3.2.3. I've attached the test cases I'm using. performance_example.tar ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Two apps on the same JVM
Hi, In a benchmark, jeromq didn't have severe performance degration. http://comments.gmane.org/gmane.network.zeromq.devel/18960 I would be happy if you test on both systems. You don't need to change your application whichever you use. Thanks Min 2013년 5월 25일 토요일에 Erwin Karbasi님이 작성: Siam Rafiee siamraf at gmail.com writes: What container are you using to bring up the applications? The JVM won't let you load the same native library from different classloaders. You'll need to ensure the library is loaded in a parent classloader, available to both application classloaders. For example, in Tomcat you would achieve this by placing the jzmq jar in 'tomcat/lib' and the native libraries in 'tomcat/bin'. You might find it easier to swap out JZMQ for JeroMQ (https://github.com/zeromq/jeromq), which is a pure Java implementation with no need for native libraries. Siam On 24 May 2013 12:51, Erwin Karbasi erwin at optinity.com wrote: Hello Guys, Following our use cases: 1. We have two applications that packaged the SO file (C lib files) and JZMQ.jar as part of the applications jar. We have tried to deploy these apps on the same JVM and encountered following error: java.lang.UnsatisfiedLinkError: jzmq (Not found in java.library.path) 2. We have also tried to set the lib files in the LD_LIBERARY_PATH (instead of above approach of packaging the lib as part of the application jar) and use separate Context object per application on the same JVM but we came across following error: java.lang.UnsatisfiedLinkError: jzmq (Library is already loaded in another ClassLoader) at java.lang.ClassLoader.loadLibraryWithPath(ClassLoader.java:1053) at java.lang.ClassLoader.loadLibraryWithClassLoader(ClassLoader.java:1017) at java.lang.System.loadLibrary(System.java:474) at org.zeromq.ZMQ.clinit(ZMQ.java:36) Your insight regarding above two issues would highly appreciated. Thanks, Erwin Karbasi ATT, Senior Software Architect ___ zeromq-dev mailing listzeromq-dev at lists.zeromq.orghttp://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev at lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev Hello Siam, Thanks a lot for fast turnaround response. I'm using WebSphere application server. 1. I know that if I'll add the library as a shared lib to the JVM I can use two Context object in the same JVM, but is it possible to add the LD_LIBERARY_PATH to the machine properties or using -Djava.lib.path as property of the JVM and to create two separate Context on separate app deployed on the same JVM? 2. I'd happy to use the JeroMQ, but as I know its performance is not a same as JZMQ and it could degrade the messaging performance significantly. Thanks, Erwin ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org javascript:; http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Pros and cons of using Cassandra for storage?
I would agree that 100% guranteed messaging is really expensive.
Cassandra might be good if data fit for it. For example a message can be
accesed by a unique key. Otherwise you should design schema carefully.
Anyway overall performance will be bound to Cassandra's performance.
If you have to store messages for a batch processing or archiving, I would
suggest Kafka or ZPER. They have better performance and are simple.
Thanks
Min
2013년 4월 18일 목요일에 Bennie Kloosteman님이 작성:
The other question is do you want persistant messaging ...To which i would
say no ( in 95% of cases) ...good in theory , crap in the field and
builds the expectation that things just work and when things go pear shape
and it doesnt just work you dont have the systems that deal with
failure.
But yes this has nothing to do with a transport.
Ben
On Wed, Apr 17, 2013 at 11:18 PM, Pieter Hintjens
p...@imatix.comjavascript:_e({}, 'cvml', 'p...@imatix.com');
wrote:
On Wed, Apr 17, 2013 at 4:58 PM, Jason Mulligan
ja...@attack.iojavascript:_e({}, 'cvml', 'ja...@attack.io');
wrote:
I'm preparing to have a MQ discussion with coworkers, and I was asked
if 0mq can use Cassandra for storage, for durability and to lessen the
requirement for another database in play.
So, my question is, has anyone looked into this? What did you find for
pros/cons? Is it too slow?
If anyone knows where I could read up about someone's findings, that'd
be very helpful.
Just to be clear here, 0MQ doesn't use anything for storage, it's a
transport layer. You'd write clients and workers and queues that used
0MQ to connect to each other, and then use Cassandra (or whatever
storage) in the appropriate places where you wanted to hold state.
So the question is really, does Cassandra work well?
-Pieter
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org javascript:_e({}, 'cvml',
'zeromq-dev@lists.zeromq.org');
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] JFS - shared memory Java based sockets
Wow, It looks very interesting. I'll be looking into them right now. Thanks Min On Friday, March 22, 2013, Steven McCoy wrote: Encountered this recently. A potential candidate for JeroMQ IPC? - http://javafastsockets.com/ - http://fastmpj.com/ -- Steve-o ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Why socket.send will block without consumer
Hi, The HWM is a maximum buffer length (not actually identical). For PUSH, send will block if it reach the HWM. Thanks Min On Wednesday, March 20, 2013, He Jie Xu wrote: Hi, all I try zeromq with following code: import zmq import random import time context = zmq.Context() socket = context.socket(zmq.PUSH) socket.bind(ipc://test.sock) socket.setsockopt(zmq.HWM, 1000) while True: zipcode = random.randrange(1, 10) temperature = random.randrange(1, 215) message = %d %d % (zipcode, temperature) socket.send(message) print 'send', message time.sleep(1) When I am running the code without any consumer. it will block at 'socket.send' But after I read the document, I think when I set HWM, the message will send to memory buffer, it shouldn't block. Is there any wrong? How can I make the 'socket.send' won't block? my zmq version was 2.1.11 I will appreciate any help from you! Thanks you. ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] zmq on Android via NDK
Hi,
Please try using jeromq. When I tested it with the Nexus and Galaxy Tab, It
worked well.
Thanks
Min
2013년 3월 19일 화요일에 Victor Perron님이 작성:
Well I had designed a collection of scripts to automate that process,
maybe you could find some inspiration there.
https://github.com/vperron/android-jzeromq
As a matter of fact those scripts are mostly useful to remember the exact
compilation flags you need to have all the symbols resolved right, which
seems to be your issue.
I wasn't using those 'Android-specific' NDK makefiles though, was more
relying on the generated standalone ARM toolchain and using standard
autotools for the rest.
Good luck,
On Mon, Mar 18, 2013 at 3:13 AM, Florian Eckerstorfer
florian.eckerstor...@gmx.net javascript:_e({}, 'cvml',
'florian.eckerstor...@gmx.net'); wrote:
I am developing on a MacBook Pro with OSX 10.7.5. I followed the steps
from the http://www.zeromq.org/build:android to build a static zmq.a
Library, unfortunately i can't build the whole Java-stuff, because I am
always getting an error at the configure step, but that's not the Problem.
I only need the c-stuff, because I'm developing native without java and
just wrap it.
The thing is, I take the library and the zmq.h and zmq_utils.h and put
them into my uni-folder in my Android project, then I add the the Library
in my Android.mk-file. Because of al the goggling I've done I tried it in a
lot of different ways, but none of them seem to work. Currently I do it
this way:
LOCAL_PATH := $(call my-dir)
include $(CLEAR_VARS)
LOCAL_MODULE:= ImageBabbleJNI
LOCAL_SRC_FILES := imagebabbleJNI.cpp
LOCAL_LDLIBS += -llog -lm -ljnigraphics
LOCAL_CPPFLAGS = -std=gnu++11 -fpermissive -fexceptions
-D__GXX_EXPERIMENTAL_CXX0X__
LOCAL_STATIC_LIBRARIES += libzmq
include $(BUILD_SHARED_LIBRARY)
So the static library should work. In my project I am using the zmq.hpp
also and that's where all the errors are. A lot of undefined references
jni/core.hpp:178: error: undefined reference to 'zmq_strerror'
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org javascript:_e({}, 'cvml',
'zeromq-dev@lists.zeromq.org');
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
--
Victor
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] cython inline checkrc change in PyZMQ 13.0.0 changes handling of EINTR?
On Mar 11, 2013, at 13:13, Jonathan Kamens jkam...@quantopian.com wrote:
Greetings,
With PyZMQ versions prior to 13.0.0, we were running into problems with
certain PyZMQ calls getting interrupted by restartable signals (e.g.,
SIGALRM) used by our application. We fixed this problem like this:
while True:
try:
self.context.term()
except zmq.ZMQError as exc:
if exc.errno == EINTR:
log.info('zmq_term interrupted by signal, restarting')
else:
log.exception('Error terminating ZMQ context')
raise
except BaseException as exc:
log.exception('Error terminating ZMQ context')
raise
else:
break
Note that 0MQ allows term() to be restarted when it gets EINTR.
This worked just fine, but has stopped working in PyZMQ 13.0.0. Now, instead
of a ZMQError with errno set to EINTR, we are getting a KeyboardInterrupt
exception. I think this commit to PyZMQ is the cause.
Questions:
Was this change in behavior intentional? If so, it probably should be
documented in the release notes.
Am I understanding correctly that what needs to be done to fix the problem in
my app is to do except KeyboardInterrupt instead of except zmq.ZMQError as
exc and not bother to check errno?
Thanks in advance for any help you can provide.
This was not intentional, and is fixed in master. There should only be a
difference in behavior if you have non-default signal handler registered.
Regards,
Jonathan Kamens
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Frequent error messages
Hi,
And what version of jzmq and libzmq did you
use?
Thanks
Min
2013년 3월 7일 목요일에 Gonzalo Vasquez님이 작성:
This is the client side code that I use to communicate with the server:
private byte[] getByte(final String table, final String name,
final int doc_off, final int doc_len, final int comp_off,
final int comp_len, final char compressionType) throws Exception {
File file = new File(cacheRoot, table.substring(0, 3) + /DOC/ + name);
//$NON-NLS-1$
Context ctx = ZMQ.context(1);
Socket req = ctx.socket(ZMQ.REQ);
req.connect(ENDPOINT);
// TODO Crear POJO en vez de Map
MapString, String params = new HashMapString, String();
params.put(path, file.getAbsolutePath());
params.put(dOff, String.valueOf(doc_off));
params.put(dLen, String.valueOf(doc_len));
params.put(cOff, String.valueOf(comp_off));
params.put(clen, String.valueOf(comp_len));
params.put(cType, String.valueOf(compressionType));
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ObjectOutputStream oos = new ObjectOutputStream(baos);
oos.writeObject(params);
oos.close();
params.clear();
baos.close();
req.send(baos.toByteArray(), NO_FLAGS);
byte[] data = req.recv();
req.close();
ctx.term();
return data;
}
And on the server side I use the three attached classes
public void run() {
// Prepare our context and sockets
Context context = ZMQ.context(1);
Socket frontend = context.socket(ZMQ.ROUTER);
Socket backend = context.socket(ZMQ.DEALER);
frontend.bind(tcp://*:5559);
backend.bind(inproc://workers);
Thread threads[] = new Thread[workers];
for (int i = 0; i threads.length; i++) {
threads[i] = new WorkerThread(i, context);
threads[i].start();
}
System.out.println(launch and connect broker.);
// Initialize poll set
Poller items = context.poller(2);
items.register(frontend, Poller.POLLIN);
items.register(backend, Poller.POLLIN);
boolean more = false;
byte[] message;
// Switch messages between sockets
try {
while (!Thread.currentThread().isInterrupted()) {
// poll and memorize multipart detection
items.poll();
if (items.pollin(0)) {
while (true) {
// receive message
message = frontend.recv(0);
more = frontend.hasReceiveMore();
// Broker it
backend.send(message, more ? ZMQ.SNDMORE : 0);
if (!more) {
break;
}
}
}
if (items.pollin(1)) {
while (true) {
// receive message
message = backend.recv(0);
more = backend.hasReceiveMore();
// Broker it
frontend.send(message, more ? ZMQ.SNDMORE : 0);
if (!more) {
break;
}
}
}
}
} finally {
// We never get here but clean up anyhow
frontend.close();
backend.close();
context.term();
}
}
Does that help solving the problem?
Gonzalo Vásquez Sáez
Gerente Investigación y Desarrollo (RD)
Altiuz Soluciones Tecnológicas de Negocios Ltda.
Av. Nueva Tajamar 555 Of. 802, Las Condes - CP 7550099
+56 2 335 2461
gvasq...@altiuz.cl javascript:_e({}, 'cvml', 'gvasq...@altiuz.cl');
http://www.altiuz.cl
http://www.altiuzreports.com
https://www.facebook.com/altiuz http://twitter.com/altiuz
http://www.linkedin.com/company/altiuz
El 05-03-2013, a las 18:19, Pieter Hintjens
p...@imatix.comjavascript:_e({}, 'cvml', 'p...@imatix.com');
escribió:
On Tue, Mar 5, 2013 at 9:57 PM, Gonzalo Vasquez
gvasq...@altiuz.cljavascript:_e({}, 'cvml', 'gvasq...@altiuz.cl');
wrote:
Hi there!
Upon stressing a Java / ZMQ based piece of software, I'm getting TWO
repeated errors every now and then, they are:
Hi Gonzalo,
As with any error you're trying to report (to any project, I guess), the
best tool is a minimal (I can't emphasize that enough) test case that
provokes the crash. If you can make this in Java we can recreate it in C,
and see where the problem is.
The test case is essential to fixing the problem since after we fix it we
have to run the same test case to be 100% the problem is gone.
-Pieter
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org javascript:_e({}, 'cvml',
'zeromq-dev@lists.zeromq.org');
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] DEALER to ROUTER and reusing identity.
Hi, Please try using ROUTER_MANDATORY at router socket. If you get EHOSTUNREACH at send, you should retry send. Thanks Min 2013년 2월 25일 월요일에 Trevor Bernard님이 작성: Hi, I have a use case where I'd like to re-cycle DEALER identities when a failover situation occurs. I have one service SV1 that creates a DEALER connection to a ROUTER socket on SVn. SV1 has a hot standby and uses the same identity in a failover situation. The first connect goes as you would expect, but in the situation of a failover, the second SV1 never receives any messages from SVn. Can someone point me in the right direction on how to route back the response? -Trev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org javascript:; http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] zmq_setsockopt() returning invalid argument
Hi
You need to set socket options before connect.
Thanks
Min
2013년 2월 17일 일요일에 suraj nellikar님이 작성:
Hi,
I am using zmq 3.2 library on Ubuntu (32 bit machine). I am trying to run
a test PUB-SUB model but the client is failing to set the sockopt to
ZMQ_SUBSCRIBE. Any idea what am I doing wrong here?
Here's the snippet of the code.
main() {
void *context = zmq_ctx_new();
void *req = zmq_socket(context,ZMQ_REQ);
void *sub = zmq_socket(context,ZMQ_SUBSCRIBE);
int rc = zmq_connect(req,tcp://127.0.0.1:);
printf(req connect, rc = %d\n,rc);
rc = zmq_connect(sub,tcp://192.168.72.130:);
printf(sub connect, rc = %d\n,rc);
uint64_t affinity = 1;
rc = zmq_setsockopt (sub, ZMQ_AFFINITY, affinity, sizeof affinity);
//This works only when I use uint64_t for affinity. It fails for int.
printf(setsockopt, rc = %d, errno = %s\n,rc,strerror(errno));
rc = zmq_setsockopt(sub,ZMQ_SUBSCRIBE,,0);
//This always fails! rc is always -1
printf(setsockopt, ZMQ_SUBSCRIBE = %d,rc = %d, errno =
%s\n,ZMQ_SUBSCRIBE,rc,strerror(errno));
Thanks.
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Problem with IPC transport on Java binding
jeromq does not support ipc correctly. But jzmq should work. Can you show us the reproducable code? Thanks Min 2013년 2월 12일 화요일에 Joe Lin님이 작성: Hi, I'm using ZeroMq 3.3.2 between a C and Java process on CentOS 6.2. The transport of using local tcp socket works fine. But it does not work for ipc transport such as ipc://service.topic. Java side cannot receive/send via the ipc transport. Does the Java binding support ipc transport at all? Thanks. ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org javascript:; http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] ZMQ JZMQ Logging
Hi, Basically logging is not supported in jzmq. If you use the latest ZMQ, you can utilize zmq_monitor. https://github.com/zeromq/libzmq/pull/340 At the callbak, you could leave your log4j log. Thanks Min 2013년 2월 12일 화요일에 Sabri Skhiri님이 작성: Dear All, I use ZMQ 2.X with the JZMQ binding. I am trying to debug a typical “java: Socket.cpp:561: void* get_socket(JNIEnv*, jobject, int): Assertion `s' failed.” I am probably sending a message on a closed socket, but it is difficult to correlate my application logs and the assert error. I would like to activate the ZMQ traces or something similar. Is there any way to activate the logs either from JZMQ using a log4j like configuration or directly from ZMQ using the same kind of mechanism? I just found this http://www.zeromq.org/topics:monitoring-support but it seems to be only a proposition not really an implementation. Thank you! Sabri. ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Question for java users
jzmq ZLoop is missing now and I'm working on it.It requires some JNI stuff. Use dase would be exactly same with jeromq. Thanks Min 2013년 2월 8일 금요일에 Ivan Pechorin님이 작성: Hi John, 2013/2/7 John Watson jkwat...@gmail.com javascript:;: Quick question for those of you out there who use 0mq from java. Do you use the jzmq ZLoop, and if so, what is an example use case? I use JeroMQ ZLoop for polling 0mq sockets (PollItems with IZLoopHandlers) and for timers. Single thread. (This is specific to the java users, because I'm trying to figure out when one might use ZLoop, rather than any of the built-in java.util.concurrent facilities, such as a fork-join pool and simple Executors). How are they useful for polling 0mq sockets? And as for the timers, it was very handy for me that ZLoop supports them too. Regards, Ivan ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org javascript:; http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] How to setup timeout in for stages in multi threaded client server app
Meir, Right, we need to elaborate the documentation. Due to the language limitation, JZMQ (jncluding jeromq) could not use the same signature, say setSockOpt. Because some options are string, some are integers and the others are boolean. We could use java.lang.Object but most of java developers might not like it. Anyhow documentation is still important and we need to publish the java doc somewhere under the api.zeomq.org. Most of all we might need volunteers for better documentation. Thank you for pointing a weakness. Thanks Min 2013년 2월 2일 토요일에 Meir Yanovich님이 작성: i do , but im using the java binding , and its luck of documentation. i mean the java part . the commands are not the same as the C api . this is my main problem . from looking on the web i see im not the only one that has this problem with the JAVA api On Fri, Feb 1, 2013 at 6:51 PM, Pieter Hintjens p...@imatix.com wrote: Meir, Before you ask questions about using 0MQ, you should read the Guide fully and digest at least the first chapters. -Pieter On Fri, Feb 1, 2013 at 3:16 PM, Meir Yanovich meiryanov...@gmail.com wrote: can some one please tell me why setting this : m_context = ZMQ.context(1); m_socket = m_context.socket(ZMQ.REQ); m_socket.setReceiveTimeOut(2000); m_socket.setSendTimeOut(2000); just dosnt work ? also does the input is in seconds or milliseconds ? there is no info about java On Fri, Feb 1, 2013 at 3:52 PM, Meir Yanovich meiryanov...@gmail.com wrote: ok reading the ZMQ.java where all the documentation is On Fri, Feb 1, 2013 at 3:31 PM, Meir Yanovich meiryanov...@gmail.com wrote: thanks for the link , but i having trouble to find how to do this in the java wrapper . i dont see any zmq_setsockopt in java . what i use in java to init is simple : m_context = ZMQ.context(1); m_socket = m_context.socket(ZMQ.REQ); On Fri, Feb 1, 2013 at 12:42 PM, Pieter Hintjens p...@imatix.com wrote: Here's a simple example: https://github.com/zeromq/libzmq/blob/master/tests/test_timeo.cpp On Fri, Feb 1, 2013 at 11:10 AM, Meir Yanovich meiryanov...@gmail.com wrote: can you please point me to some example . thanks On Fri, Feb 1, 2013 at 11:10 AM, Pieter Hintjens p...@imatix.com wrote: You can set send/receive timeouts using the ZMQ_SNDTIMEO and ZMQ_RCVTIMEO socket options. -Pieter On Fri, Feb 1, 2013 at 9:18 AM, Meir Yanovich meiryanov...@gmail.com wrote: hello all i want to set few timeouts levels in ZeroMQ Java app . 1. request timeout . 2.response timeout 3. round trip time out . can you please point me on the right way to do this ? Thanks ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dhttp://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Zeromq messages getting dropped
I was able to reproduce the issue on jzmq even on zeromq 3.2.2. What I discovered is about last 30K bytes of 45K message was not sometimes delivered to in-router on raw close. I didn't build equivalent C code, as jzmq is a thin wrapper of native C library it could have the same problem. But I didn't find a clear solution yet. Thanks Min On Thu, Jan 24, 2013 at 6:39 AM, Ritesh Adval riteshad...@gaikai.comwrote: Hello, I have created a bug for this issue with instructions and java test case. Its at https://zeromq.jira.com/browse/LIBZMQ-497 Thanks Ritesh On Tue, Jan 22, 2013 at 6:30 PM, Ritesh Adval riteshad...@gaikai.comwrote: Thanks Min, I will create a bug with instruction and unit test. I was also experimenting with Java only version of zeromq ( https://github.com/zeromq/jeromq). When running same test it does not drop message but has some other issue. -Ritesh On Mon, Jan 21, 2013 at 11:53 PM, Min mini...@gmail.com wrote: Ritesh, If you can reproduce the problem, Java code should be fine. Community could look into it. Thanks Min 2013년 1월 17일 목요일에 Ritesh Adval님이 작성: Hi Charles, I have test program in JAVA, I am not a C programmer so i will probably take me time to reproduce this in C. Can someone first take a look at my JAVA program to see if I am not doing anything stupid. Should I create bug and attach Java maven project? Its very easy to run it, all you need is zeromq 2.2.0 installed and jzmq built and installed by building jzmq ( https://github.com/zeromq/jzmq). I can add instructions to the bug report. Once confirmed that program looks right I can try to create a C version of the test but will take me some time. let me know. Thanks Ritesh On Wed, Jan 16, 2013 at 10:55 PM, Charles Remes li...@chuckremes.comwrote: On Jan 16, 2013, at 4:08 PM, Ritesh Adval riteshad...@gaikai.com wrote: Hi Charles, Yes I close the socket in my thread after sending 100 messages, and I expect that LINGER will make sure messages are sent to the other end, I expected that context termination will block and make sure any pending messages are sent, but thats not happening. context termination returns quickly. Just now tried again in my unit test by setting LINGER to Integer.MAX_VALUE explicitly in all my sockets and ran the test again and it did fail with messages getting dropped. The interesting thing is only the 100th message (The last one) from some of my concurrent threads are getting dropped. Time to show someone the code. That's the easiest way to figure it out. If you can reproduce this in C, that will get a lot more attention. Here's how to open an issue: http://www.zeromq.org/docs:issue-tracking cr ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] How to set zeromq send message block?
Hi, Try using ZMQ_DONTWAIT on your send. Thanks Min 2013년 1월 17일 목요일에 timger™님이 작성: How to configuration parameters that zeromq can Send a block message， When a process sends a message to another process， if another process does not receive the message, the process which send message will block on thesend function does not return timger™ ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Zeromq messages getting dropped
Ritesh,
If you can reproduce the problem, Java code should be fine.
Community could look into it.
Thanks
Min
2013년 1월 17일 목요일에 Ritesh Adval님이 작성:
Hi Charles,
I have test program in JAVA, I am not a C programmer so i will probably
take me time to reproduce this in C. Can someone first take a look at my
JAVA program to see if I am not doing anything stupid. Should I create bug
and attach Java maven project?
Its very easy to run it, all you need is zeromq 2.2.0 installed and jzmq
built and installed by building jzmq (https://github.com/zeromq/jzmq).
I can add instructions to the bug report. Once confirmed that program
looks right I can try to create a C version of the test but will take me
some time.
let me know.
Thanks
Ritesh
On Wed, Jan 16, 2013 at 10:55 PM, Charles Remes
li...@chuckremes.comjavascript:_e({}, 'cvml', 'li...@chuckremes.com');
wrote:
On Jan 16, 2013, at 4:08 PM, Ritesh Adval
riteshad...@gaikai.comjavascript:_e({}, 'cvml',
'riteshad...@gaikai.com');
wrote:
Hi Charles,
Yes I close the socket in my thread after sending 100 messages, and I
expect that LINGER will make sure messages are sent to the other end, I
expected that context termination will block and make sure any pending
messages are sent, but thats not happening. context termination returns
quickly.
Just now tried again in my unit test by setting LINGER to
Integer.MAX_VALUE explicitly in all my sockets and ran the test again and
it did fail with messages getting dropped.
The interesting thing is only the 100th message (The last one) from
some of my concurrent threads are getting dropped.
Time to show someone the code. That's the easiest way to figure it out.
If you can reproduce this in C, that will get a lot more attention.
Here's how to open an issue:
http://www.zeromq.org/docs:issue-tracking
cr
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org javascript:_e({}, 'cvml',
'zeromq-dev@lists.zeromq.org');
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] [JZMQ] JZMQ and IPV6 support
I've submitted a pull request on it. https://github.com/zeromq/jzmq/pull/150 Thanks Min 2013년 1월 12일 토요일에 Min님이 작성: I think I could handle it from the other next week. Thanks Min 2013년 1월 11일 금요일에 Sabri Skhiri님이 작성: Hi Min, Steven Is there any plan for such a patch? Should I propose one? Or asking first for a feature request? Thank you! Sabri. Le 10 janv. 2013 15:22, Min mini...@gmail.com a écrit : Hi, For JZMQ, the option is not supported yet. We need a patch for the option. Thanks Min 2013년 1월 10일 목요일에 Steven McCoy님이 작성: On 10 January 2013 04:25, Sabri Skhiri sabri.skh...@gmail.com wrote: We need to support an IPV6 environment and we currently use JZMQ. By googling IPV6 Support and ZMQ I found the Steven McCoy's patch reference ( http://lists.zeromq.org/pipermail/zeromq-dev/2011-August/012834.html ). Basically it says that we just need to setup a socket option to false ( *ZMQ_IPV4ONLY)*. Correct, the only thing missing which is something I recently discovered is called Happy Eyeballs - RFC6555. http://tools.ietf.org/rfc/rfc6555.txt I found this on a libcurl blog and as they haven't implemented it either: it's a set of requirements for dual-stack environments to manage situations when IPv6 connectivity fails and IPv4 is possible but with minimal delay presented to the user. I'm not aware of an actual algorithm that does implement the requirements though. By that description it may not to be too useful for ZeroMQ either. -- Steve-o ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] [JZMQ] JZMQ and IPV6 support
I think I could handle it from the other next week.
Thanks
Min
2013년 1월 11일 금요일에 Sabri Skhiri님이 작성:
Hi Min, Steven
Is there any plan for such a patch? Should I propose one? Or asking first
for a feature request?
Thank you!
Sabri.
Le 10 janv. 2013 15:22, Min mini...@gmail.com javascript:_e({},
'cvml', 'mini...@gmail.com'); a écrit :
Hi,
For JZMQ, the option is not supported yet.
We need a patch for the option.
Thanks
Min
2013년 1월 10일 목요일에 Steven McCoy님이 작성:
On 10 January 2013 04:25, Sabri Skhiri sabri.skh...@gmail.com wrote:
We need to support an IPV6 environment and we currently use JZMQ. By
googling IPV6 Support and ZMQ I found the Steven McCoy's patch reference (
http://lists.zeromq.org/pipermail/zeromq-dev/2011-August/012834.html).
Basically it says that we just need to setup a socket option to false (
*ZMQ_IPV4ONLY)*.
Correct, the only thing missing which is something I recently discovered
is called Happy Eyeballs - RFC6555.
http://tools.ietf.org/rfc/rfc6555.txt
I found this on a libcurl blog and as they haven't implemented it
either: it's a set of requirements for dual-stack environments to manage
situations when IPv6 connectivity fails and IPv4 is possible but with
minimal delay presented to the user. I'm not aware of an actual algorithm
that does implement the requirements though.
By that description it may not to be too useful for ZeroMQ either.
--
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org javascript:_e({}, 'cvml',
'zeromq-dev@lists.zeromq.org');
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] event monitors and multiple io threads
Hi, A ZMQ socket is not shared by multiple io_threads. Neither the attached monitor_socket is. So it should be thread-safe. Do you see any symptom the thread safety is broken? Thanks Min 2013년 1월 10일 목요일에 Ben Gray님이 작성: I have been adding more event monitoring into a fork of 3.2.2 for auditing at where I work and so far I've been unable to find how the socket::monitor_event calls are thread safe. As far as I can tell they are called via the socket::event_* functions from a number of different io_objects which can happily exist on different io threads. If someone could point me in the correct direction then I can make sure my new events don't break anything important. The particular example I was tracing through is a bound push socket which selects, based on load, one io thread for the tcp_listener and then a potentially different io thread for each stream_engine generated from accepting incoming connections. Both of these objects are appear to have the same socket pointer and call event methods when things happen. Regards, Ben ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org javascript:; http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Basic questions on ZeroMQ functionality
Hi, Comments are under the questions. Thanks Min 2013년 1월 10일 목요일에 shyam valsan님이 작성: I have few basic questions on ZeroMQ functionality - What is the maximum buffer size for zeroMQ receive socket (ZMQ_SUB). Wanted to know how many messages can be queued up on the receive socket. You can control the buffer size by setting send/recv hwm configuration. 1000 is default for 3.x. And unlimited for 2.x - What is the max length of messages that can be send on the socket using zmq_send(). If the length exceeds the limit, will it be handled internally by zeroMQ as two different message or will the message get truncated? If your memory allows, ZMQ can send 2^64 bytes as a single frame. A message would not be truncated. But it is not a good practice. You should split into multiple messages for example sending a large file. FileMQ is a good example. After I have opened a socket for listening [ i.e after successful calls to zmq_connect and zmq_setsockopt() with ZMQ_SUBSCRIBE option],all messages being send on that port (where subscribe socket is listening) will be received by the subscribe socket and queued up in its receive queue. Once zmq_receive() call is made all these messages will be received by application. Is understanding correct. Messages are being sent with the best effort until both sender's and receiver's queue are full. Thanks, Shyam ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] [JZMQ] JZMQ and IPV6 support
Hi,
For JZMQ, the option is not supported yet.
We need a patch for the option.
Thanks
Min
2013년 1월 10일 목요일에 Steven McCoy님이 작성:
On 10 January 2013 04:25, Sabri Skhiri
sabri.skh...@gmail.comjavascript:_e({}, 'cvml', 'sabri.skh...@gmail.com');
wrote:
We need to support an IPV6 environment and we currently use JZMQ. By
googling IPV6 Support and ZMQ I found the Steven McCoy's patch reference (
http://lists.zeromq.org/pipermail/zeromq-dev/2011-August/012834.html).
Basically it says that we just need to setup a socket option to false (*
ZMQ_IPV4ONLY)*.
Correct, the only thing missing which is something I recently discovered
is called Happy Eyeballs - RFC6555.
http://tools.ietf.org/rfc/rfc6555.txt
I found this on a libcurl blog and as they haven't implemented it either:
it's a set of requirements for dual-stack environments to manage situations
when IPv6 connectivity fails and IPv4 is possible but with minimal delay
presented to the user. I'm not aware of an actual algorithm that does
implement the requirements though.
By that description it may not to be too useful for ZeroMQ either.
--
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] error import after installing pyzmq latest version
On Dec 29, 2012, at 1:27, Maxa Jean Aimee maxajeanai...@yahoo.fr wrote: Hi everyone, I have an error when i attempted to update my zeromq to the new version 3.2 This is the ouput that I have: Traceback (most recent call last): File stdin, line 1, in module File /usr/local/lib/python2.7/dist-packages/zmq/__init__.py, line 51, in module from zmq import core, devices File /usr/local/lib/python2.7/dist-packages/zmq/core/__init__.py, line 26, in module from zmq.core import (constants, error, message, context, ImportError: /usr/local/lib/python2.7/dist-packages/zmq/core/socket.so: undefined symbol: zmq_sendmsg I really don't know what's going on ? I'm using ubuntu 12.04 and I develop on python. I've already check on stackoverflow website but i did not see anything. It works well before but I started to have this error when I've finish installing pyzmq 2.2.0.1. Could anyone help me on this ? You need to recompile / reinstall pyzmq when you do a major upgrade of libzmq. Thanks MAXA ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Problem routing packet back to client
Hi, Router drops messege silently when a counter part is busy or doesn't exist. Please try setting ZMQ_ROUTER_MANDATORY in your router sockets. Thanks Min 2012년 12월 27일 목요일에 Hsiao Henry님이 작성: I am using ZeroMQ 3.2.2 with clrzmq 3.0.0 binding on C#. Attempting to use ZeroMQ within Microsoft Azure environment with service bus queue. I have 3 pieces, RECEIVER, WORKER, RESPONDER Below is what is working: RECEIVER have following threads: 1) ZMQ_PROXY, accept outside connection and route it to INPROC (to thread 2 below) 2) receive worker have a ROUTER socket connect to INPROC address and bind to thread-specific INPROC address Like following: receive_worker.connect(inproc://workers); receive_worker.bind(inproc://workers-1); Idea is it will put all request from inproc://workers into Service Bus Queue (WORK_QUEUE), and anything from inproc://workers-1 is sent back to the user. 3) response worker Retrieve data from Queue (RESPONSE_QUEUE), this thread will forward this address to the correct thread via the address contained in the data. WORKER: This part right now just does echo, so it takes everything from WORK_QUEUE and shove it directly into RESPONSE_QUEUE Now what I actually want to accomplish is, because there are multiple instance of RECEIVER, I thought I need a separate RESPONDER instance. Modification is that RECEIVER worker will also attach its own IP to the message, so responder knows which instance of RECEIVER to send the reply to. Problem I am getting is... well... message just gets lost. All this probably sounds very confusing, but if anyone could help by perhaps asking the question I haven't thought of?? *.* Problem with the design that does work is, well once I have multiple instance of RECEIVER, then message don't always get routed back to the instance of receiver the client is connected to. Thanks in advance, Henry ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] timeout value in zeromq poll
In a old version, the unit might be nano second. You should multiply ZMQ_POLL_MSEC to make version compatible poll. Probably, there is comment about this on the guide. Thanks Min 2012년 12월 22일 토요일에 Stefan de Konink님이 작성: As per manpages; If none of the requested events have occurred on any zmq_pollitem_t item, zmq_poll() shall wait timeout milliseconds for an event to occur on any of the requested items. One thing for sure: if I enter 6L the timeout is not 60s. It seems to be that 6000L comes closer to that. That sounds more like microseconds to me (which might be considered overkill). Is this a documentation bug, or an implementation bug? Stefan ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org javascript:; http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Actor Model
Akka on zeromq would be an another fun for you. http://doc.akka.io/docs/akka/snapshot/scala/zeromq.html Thanks Min Dec 15, 2012 9:01 PM Gleb Peregud glebe...@gmail.com 작성: If you want the most natural way to code your program according to actor model, just use Erlang. It has a zeromq library, hence you can communicate with outside world using zmq. On Sat, Dec 15, 2012 at 12:27 PM, wilfr...@hafner.ws wrote: hi peeps, i would like to ask you for some ideas/hints on an actor-model design on top of zeromq. zeromq ships with everything i need, a mailbox, message routing, i don't have to worry about threads etc. i think the straightforward way would be to give every actor one or more sockets for sending and receiving messages, depending on the used patterns (pub-sub, req-rep). but it seems to be a big overhead, since the max. number of sockets per context is limited. on the other hand, i would have to share some sockets and write a lot of messaging features by myself. so zeromq would be useful just for the communication between some processes or nodes. it's similar to an component based entity system, communication between entities-entities, entities-system and system-system should be handled through an event/message bus. Although it's not for game development, 10k+ entities are possible, which would require at least more than 10k sockets. thank you for your suggestions! best regards, wil ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Discussion on ROUTER action
OK, I'm going to send a pull request as the ZMQ_ROUTER_MNDATORY covers the counterpart full case also. Thanks Min 2012. 12. 7. 오후 9:24 Pieter Hintjens p...@imatix.com 작성: Hi Min, I think you could extend ZMQ_ROUTER_MANDATORY to this. Mandatory means the message must be deliverable, or you get an error. -Pieter On Thu, Dec 6, 2012 at 4:36 AM, Yu Dongmin mini...@gmail.com wrote: Hello, As we all know, ROUTER drop messages when 1) a counter part doesn't exist and 2 ) a counter part is full. In case of 1) we can catch the case by setting ZMQ_ROUTER_MANDATORY. But in case of 2) ROUTER drop message silently and the send returns 0. Server side ROUTER-DEALER pattern including the proxy (a.k.a device) is hard to use as I need another treatment at client side. To make a client simple and avoid the second case of silent dropping, I have to increase the HWM or unlimited but it could blow memory. We might need an another option to detect second case. By detecting the case, ROUTER user can decide whether he drop it silently (default action), resend it or wait the counter part is ready. My idea is (please ignore option names, they are just examples) in the case of 2) if ZMQ_ROUTER_WAIT set router.xsend returns -1 with EAGAIN so system can wait the counter part is ready Otherwise returns -1 as user can do his treatment. Thanks Min ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] dynamic discovery problem
I think the zyre would help. https://github.com/zeromq/zyre Thanks Min 2012. 12. 8. 오후 3:16 Maxa Jean Aimee maxajeanai...@yahoo.fr 작성: Hi everyone, I develop an application which simulates upnp protocol, it's involves: Auto-configuration, Auto discovery, publishes subscribe, etc... To give a general overview of my project: UPnP is a set of networking protocols that permits networked devices, such as personal computers, printers, Internet gateways, Wi-Fi access points and mobile devices to seamlessly discover each other's presence on the network and establish functional network services for data sharing, communications, and entertainment To simulate it, i create four process which act like real devices. They can discover dynamically other process and can subscribe on some evenements of other process. This is just a general overview of what i have to do. My problem is that i cannot figure out how to resolve the dynamic discovery problem. I follow some advices on this page http://zguide.zeromq.org/page:all but i didn't found a solution on my problem. If there is a possibility that you show me some pieces of code. it will help me very much. I really need help on this Maxa jean aime Élève ingénieur à l'ESIROI-Spécialité informatique et Télécommunication (École Supérieure d'Ingénieurs Réunion océan indien) Tel:+(262)693915938 ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Problem with fileserver / fileclient
And what if we add an option, blocking ROUTER ? For some application, it can be very annoying that some messages are dropped silently. If the option is set, the ROUTER will block instead of dropping. Sure this can make whole system slow, but its there's is a choose at least. Thanks Min On Mon, Nov 19, 2012 at 10:03 PM, Pieter Hintjens p...@imatix.com wrote: It's not deterministic. In fact the proper value is infinite, because once credit-based flow control works, the HWM is never reached (the fact we reach it at 10 is a kind of fake). The reason for setting it to 10 was to prove that the CBFC worked. We do know that there are never more than 10 messages in flight. 0MQ just counts them asynchronously in a way that sometimes 10 looks like 29... In other words, if you remove the HWM setting so that it is at the default of 1,000, it will work properly and accurately. Hope that helps. -Pieter On Mon, Nov 19, 2012 at 8:09 PM, Stathis Gkotsis stathis.gkot...@commsquare.com wrote: Hi Pieter, Thanks a lot for your help. I have done some further tests and it seems that setting the HWM on the server to some higher value generally helps, but setting it to 20 is not always enough. Setting it to 30 seems to work, but I am not sure if that is deterministic. What would you say a value that always guarantees proper functioning would be, until the issue is fixed? Would this value only depend on the size of the PIPELINE, rather than on other random factors as well? Thanks, Stathis -Original Message- From: zeromq-dev-boun...@lists.zeromq.org [mailto: zeromq-dev-boun...@lists.zeromq.org] On Behalf Of Pieter Hintjens Sent: Monday, November 19, 2012 9:26 AM To: ZeroMQ development list Subject: Re: [zeromq-dev] Problem with fileserver / fileclient It's definitely timing dependent but the limit isn't two times, it still failed with a HWM 3x higher than the calculated amount. Sending more or fewer frames had no impact, as you say. So let's treat this as a documentation issue for now, and we can think about how to process LWM commands faster in the future. -Pieter On Mon, Nov 19, 2012 at 2:36 PM, Min mini...@gmail.com wrote: At the pipe.cpp the variables, msgs_written and peers_msgs_read, which are used at the HWM full calculation are updated only at the last frame. My guess is that the peers_msg_reads can be late updated because it is updated asynchronously. For example, the dealer got chunks and sent subsequent requests, but the router get the new requests before it get 'activate_write'. The outgoing pipe can grow to twice of PIPELINE at maximum in an extreme case. Thanks Min On Mon, Nov 19, 2012 at 10:23 AM, Pieter Hintjens p...@imatix.com wrote: I've updated the doc to make this clear, and will change the examples in the Guide accordingly. Thanks for spotting this problem! https://github.com/hintjens/libzmq/commit/7533ebb33d1cc6f23061e0228bf 7eb22713aaa17 -Pieter On Mon, Nov 19, 2012 at 10:01 AM, Pieter Hintjens p...@imatix.com wrote: Stathis, I've tested this in detail. Tthere seems to be a problem with the HWM in the ROUTER, which is dropping outgoing messages unexpectedly. That is, the HWM setting of 10 isn't actually enough for 10 outgoing messages. What it looks like is that the HWM is in fact per message frame, not per message. So in this case you'd have to use: zsocket_set_hwm (router, PIPELINE * 2); In the router since the identity frame is counted in the HWM calculation. This is a bug IMO, or at least it's rather surprising. -Pieter On Mon, Nov 19, 2012 at 3:13 AM, Stathis Gkotsis stathis.gkot...@commsquare.com wrote: Indeed, sorry about this, here are the two links to the code: http://pastebin.com/1DLa4e2J http://pastebin.com/cbJUxJB4 Thanks, Stathis From: zeromq-dev-boun...@lists.zeromq.org [zeromq-dev-boun...@lists.zeromq.org] On Behalf Of Min [mini...@gmail.com] Sent: Sunday, November 18, 2012 7:36 PM To: ZeroMQ development list Subject: Re: [zeromq-dev] Problem with fileserver / fileclient The two links seem to be identical. Could you upload the client code also ? Thanks Min On Mon, Nov 19, 2012 at 1:22 AM, Stathis Gkotsis stathis.gkot...@commsquare.commailto:stathis.gkotsis@commsquare. com wrote: Hello, I am using zeromq v3.2.1-RC2 on Ubuntu 10.04 / Ubuntu 12.04 64bit. I created two simple apps: one fileserver and one fileclient, based on the File Transfer example code in the documentation (http://zguide.zeromq.org/page:all#header-151 , model 3). The code for these two example apps can be found here: http://pastebin.com/AuGU9EDC and here: http://pastebin.com/LusbjGCa . As you can see, I use a file of 100KB. The problem
Re: [zeromq-dev] Problem with fileserver / fileclient
The two links seem to be identical. Could you upload the client code also ? Thanks Min On Mon, Nov 19, 2012 at 1:22 AM, Stathis Gkotsis stathis.gkot...@commsquare.com wrote: Hello, I am using zeromq v3.2.1-RC2 on Ubuntu 10.04 / Ubuntu 12.04 64bit. I created two simple apps: one fileserver and one fileclient, based on the File Transfer example code in the documentation ( http://zguide.zeromq.org/page:all#header-151 , model 3). The code for these two example apps can be found here: http://pastebin.com/AuGU9EDCand here: http://pastebin.com/LusbjGCa . As you can see, I use a file of 100KB. The problem is that when I start the server and then run the client multiple times, sometimes it receives less than 102400 bytes, like: 98304 or 94208 or 90112 bytes. For example, when I run the client 1000 times with an interval of 100ms between the runs, for 970 runs it receives 12400 bytes, while for the rest (30 runs) it receives less bytes. I am bit puzzled by this, could you help? Thanks, Stathis ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Problem with fileserver / fileclient
At the pipe.cpp the variables, msgs_written and peers_msgs_read, which are used at the HWM full calculation are updated only at the last frame. My guess is that the peers_msg_reads can be late updated because it is updated asynchronously. For example, the dealer got chunks and sent subsequent requests, but the router get the new requests before it get 'activate_write'. The outgoing pipe can grow to twice of PIPELINE at maximum in an extreme case. Thanks Min On Mon, Nov 19, 2012 at 10:23 AM, Pieter Hintjens p...@imatix.com wrote: I've updated the doc to make this clear, and will change the examples in the Guide accordingly. Thanks for spotting this problem! https://github.com/hintjens/libzmq/commit/7533ebb33d1cc6f23061e0228bf7eb22713aaa17 -Pieter On Mon, Nov 19, 2012 at 10:01 AM, Pieter Hintjens p...@imatix.com wrote: Stathis, I've tested this in detail. Tthere seems to be a problem with the HWM in the ROUTER, which is dropping outgoing messages unexpectedly. That is, the HWM setting of 10 isn't actually enough for 10 outgoing messages. What it looks like is that the HWM is in fact per message frame, not per message. So in this case you'd have to use: zsocket_set_hwm (router, PIPELINE * 2); In the router since the identity frame is counted in the HWM calculation. This is a bug IMO, or at least it's rather surprising. -Pieter On Mon, Nov 19, 2012 at 3:13 AM, Stathis Gkotsis stathis.gkot...@commsquare.com wrote: Indeed, sorry about this, here are the two links to the code: http://pastebin.com/1DLa4e2J http://pastebin.com/cbJUxJB4 Thanks, Stathis From: zeromq-dev-boun...@lists.zeromq.org [ zeromq-dev-boun...@lists.zeromq.org] On Behalf Of Min [mini...@gmail.com] Sent: Sunday, November 18, 2012 7:36 PM To: ZeroMQ development list Subject: Re: [zeromq-dev] Problem with fileserver / fileclient The two links seem to be identical. Could you upload the client code also ? Thanks Min On Mon, Nov 19, 2012 at 1:22 AM, Stathis Gkotsis stathis.gkot...@commsquare.commailto:stathis.gkot...@commsquare.com wrote: Hello, I am using zeromq v3.2.1-RC2 on Ubuntu 10.04 / Ubuntu 12.04 64bit. I created two simple apps: one fileserver and one fileclient, based on the File Transfer example code in the documentation ( http://zguide.zeromq.org/page:all#header-151 , model 3). The code for these two example apps can be found here: http://pastebin.com/AuGU9EDCand here: http://pastebin.com/LusbjGCa . As you can see, I use a file of 100KB. The problem is that when I start the server and then run the client multiple times, sometimes it receives less than 102400 bytes, like: 98304 or 94208 or 90112 bytes. For example, when I run the client 1000 times with an interval of 100ms between the runs, for 970 runs it receives 12400 bytes, while for the rest (30 runs) it receives less bytes. I am bit puzzled by this, could you help? Thanks, Stathis ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.orgmailto:zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Serialisation of message with SND_MORE
PULL works on a fair queue basis between active pipes. But which pipe is active depends on when a messege is deliverd to its pipe. If you have enough delay between each message, answer will be 1). Otherwise the order might not be consistence including 4) a Thanks Min 2012. 11. 8. 오전 1:17 Stefan de Konink ste...@konink.de 작성: Hi, How are messages serialised in the following SND_MORE scenario? push1 \ pull0 / push2 push1 sends: A1 A2 push2 sends: B1 B2 What is the expected outcome at pull0? 1) A1 B1 B2 A2 2) B1 B2 A1 A2 3) A1 A2 B1 B2 Stefan ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Serialisation of message with SND_MORE
Sorry it's partially sent and I didn't notice SND_MORE flags. Answer might be 3) or 2) depends on when each pipe is activated. 1) will never happen. Thanks Min 2012. 11. 8. 오전 2:43 Min mini...@gmail.com 작성: PULL works on a fair queue basis between active pipes. But which pipe is active depends on when a messege is deliverd to its pipe. If you have enough delay between each message, answer will be 1). Otherwise the order might not be consistence including 4) a Thanks Min 2012. 11. 8. 오전 1:17 Stefan de Konink ste...@konink.de 작성: Hi, How are messages serialised in the following SND_MORE scenario? push1 \ pull0 / push2 push1 sends: A1 A2 push2 sends: B1 B2 What is the expected outcome at pull0? 1) A1 B1 B2 A2 2) B1 B2 A1 A2 3) A1 A2 B1 B2 Stefan ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Client request and no answer from the Server
Anna,
Could you paste the full source ?
Also, kill -3 server_pid might show what blocks the server.
Thanks
Min
2012/10/10 Anna a...@aphroditel.eu:
Hi! I'm working on ZMQ, I have a class Client that should send 100 request
to the server, you can see here both classes:
public class Client {
public static void main(String[] args){
// Prepare our context and socket
ZMQ.Context context = ZMQ.context(1);
ZMQ.Socket socket = context.socket(ZMQ.REQ);
socket.connect (tcp://localhost:);
// Do 10 requests, waiting each time for a response
for(int request_nbr = 0; request_nbr 100; request_nbr++) {
String requestString = Hello World;
byte[] request = requestString.getBytes();
//request[request.length-1]=0; //Sets the last byte to 0
// Send the message
//System.out.println(Sending request + request_nbr + ...);
socket.send(request, 0)
System.out.println(request sended);
// Get the reply.
byte[] reply = socket.recv(0);
final String value = new String(reply);
System.out.println(answer +value);
// When displaying reply as a String, omit the last byte
because
// our Hello World server has sent us a 0-terminated string:
//System.out.println(Received reply + request_nbr + : [ +
new String(reply,0,reply.length-1) + ]);
}
}
public class Server {
static ZMQ.Context context = ZMQ.context(1);
static ZMQ.Socket socket = context.socket(ZMQ.REP);
public static void main(String[] args) {
socket.bind(tcp://*:);
byte[] request;
request = socket.recv (0);
final String value = new String(request);
final Thread thread1 = new Thread( new Runnable() {
public void run(){
Producer producer = new Producer();
producer.requestToRingBuffer(value);
System.out.println(producer);
}
});
thread1.start();
//replyIp[replyIp.length-1]=0; Sets the last byte of the reply to 0
}
public static void arreplegaDelServer(String loArreplegat){
byte[] reply = loArreplegat.getBytes();
socket.send(reply,0);
}
}
As you can see above, the class Server has a thread that does all the job
and on the method arreplegaDelServer I get a reverse of the request done by
the Client. My problem is when I run it the program, the Client makes the
first request it gets the answer from the Server, but when it does the
second request I just get on my terminal the comment request sended but I
don't get the comment producer which means that the request is on the
thread1 or neither answer which show me the answer from the Server, so has
anyone a clue about what is happening? Thanks!
Anna.
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] jeromq - java pojo zeromq
Thank you for the information. I changed the termination logic a little bit. Please pull the latest code and test. Thanks Min 2012/8/22 Will Heger will.he...@gmail.com: Hi Min, Very thankful for this project. To celebrate, here is an error report ;) ~/d $ git clone https://github.com/miniway/jeromq.git ~/d $ cd jeromq ~/d/jeromq $ mvn -X package Lots of Output ~~~ Stop dealer B write header 6 writer body 30 30 recv 6 OK end Stop client thread Exception in thread Thread-1001 java.lang.IllegalStateException at zmq.SocketBase.recv(SocketBase.java:769) at zmq.Device.device(Device.java:52) at zmq.ZMQ.zmq_device(ZMQ.java:335) at zmq.TestProxyTcp$Main.run(TestProxyTcp.java:274) Full debug output: http://pastebin.com/gYeZc9un Build env: Debian 64bit with Sun/Oracle Java 1.6 Let me know if I can provide any more information or if you want me to post this to Github. Best regards and thanks for the effort, -Will On Tue, Aug 21, 2012 at 10:30 PM, ramesh chandra adroi...@gmail.com wrote: Thanks Min. regards, Ramesh On 08/21/2012 10:44 AM, Min Yu wrote: I'll state what is supporting and what is not in more detail. Actually the pgm is not supported also. I copy and paste c++ code then changed it into java by hand. The Eclipse help me a alot. Thanks Min Aug 21, 2012 9:31 PM Pieter Hintjens p...@imatix.com 작성: On Tue, Aug 21, 2012 at 1:23 PM, Min mini...@gmail.com wrote: 99% means that we can send and recv between jeromq and the original zeromq through tcp. But Java doesn't support unix domain socket, so we cannot mix them for ipc. So I distracted 1%. Actually 90% compatible might have been the right value :) It's worth stating this clearly on the README, 100% compatible for tcp:// transport, does not support ipc://. Did you write all this by hand or did you have help from a C++-to-Java translator of some kind? It's impressive work either way. -Pieter ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] jeromq - java pojo zeromq
Hello,
If you don't set SUBSCRIBE, sub socket will block.
please add the following line before connect.
subscriber.subscribe() // please pull the latest source first
Thanks
Min
2012/8/22 jonas.ad...@epiq.se:
Hello!
Let me first say that this is awesome, and I really love the amount of
work you have put into this!
However! I cannot get it to work on windows :(
I try using this bit of code:
public class TestZMQ {
public static void main(String[] args) {
// Prepare our context and subscriber
ZMQ.Context context = ZMQ.context(1);
ZMQ.Socket subscriber = context.socket(ZMQ.SUB);
// Connect to server
subscriber.connect(tcp://localhost:5563);
while (true) {
// Read envelope with address
String address = new String(subscriber.recv(0));
// Read message contents
String contents = new String(subscriber.recv(0));
System.out.println(address + : + contents);
}
}
}
but the program hangs on subscriber.recv(0)!
Following the call stack i see that the program gets stuck on:
Signaler.wait_event(long) line:108
which calls:
WindowsSelectorImpl(SelectorImpl).selectNow()
Anyone else has this issue? Or any idea on how to solve it? I have tried
some debugging, and i know that on a TCP level, i am connected to the
server...
Hello
I'm happy to announce that I've built a jeromq with POJO based on
zeromq-3x c++ sources.
https://github.com/miniway/jeromq
The primary purpose of the work was to understand deeply how zeromq works.
I've got lots of insight and learned a lot from zeromq internals.
It's in a very early stage and it must have bugs and issues. But I
hope this project help someone who only know java to understand
zeromq.
Thanks
Min
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] jeromq - java pojo zeromq
Jeff, I'm trying to make jzmq and jeromq as compatible as possible but package namespace. On that strategy, missing methods will be added soon and should work exactly same with jzmq, then I'll just add some sugar methods and classes. I don't want to surprise anyone who is already familiar with jzmq. Will, The test failures are known problem. I get same failure at home but not at office. Some ISP, including my home's ISP, responds a default ip address for not existing domain. It might seems that ISP let user redirect to their commercial site. Anyway, I'm going to ignore the test as it is a very trivial case. Thanks Min 2012/8/23 Will Heger will.he...@gmail.com: Hi Min, The fixed project is building now and testing. I get two test failures right now: * zmq.TestConnectResolve * zmq.TestAddress http://pastebin.com/LUcyuGFX Otherwise looks good. Best regards, -Will On Wed, Aug 22, 2012 at 1:17 PM, Jeff Rodenburg jeff.rodenb...@gmail.com wrote: Thanks Min. Can you contrast the differences between this lib wrapper and jzmq? I can perceive some of those, but would prefer to hear your thoughts. thanks, Jeff On Tue, Aug 21, 2012 at 12:22 AM, Min mini...@gmail.com wrote: Hello I'm happy to announce that I've built a jeromq with POJO based on zeromq-3x c++ sources. https://github.com/miniway/jeromq The primary purpose of the work was to understand deeply how zeromq works. I've got lots of insight and learned a lot from zeromq internals. It's in a very early stage and it must have bugs and issues. But I hope this project help someone who only know java to understand zeromq. Thanks Min ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] jeromq - java pojo zeromq
Hello I'm happy to announce that I've built a jeromq with POJO based on zeromq-3x c++ sources. https://github.com/miniway/jeromq The primary purpose of the work was to understand deeply how zeromq works. I've got lots of insight and learned a lot from zeromq internals. It's in a very early stage and it must have bugs and issues. But I hope this project help someone who only know java to understand zeromq. Thanks Min ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] jeromq - java pojo zeromq
Hello, 99% means that we can send and recv between jeromq and the original zeromq through tcp. But Java doesn't support unix domain socket, so we cannot mix them for ipc. So I distracted 1%. Actually 90% compatible might have been the right value :) Thanks Min 2012/8/21 Pieter Hintjens p...@imatix.com: Hi Min, I'm happy to announce that I've built a jeromq with POJO based on zeromq-3x c++ sources. https://github.com/miniway/jeromq Very interesting... one question, when you say 99% compatible with zeromq, what does that mean? -Pieter ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] (py)zmq, fork and cloexec
On Jun 19, 2012, at 7:06 AM, Christian Heimes li...@cheimes.de wrote: Hello all, I've done some tests with pyzmq to check if ZQM sets the CLOEXEC flag on its file descriptors. During the tests I ran into several issues that might be bugs in ZMQ or pyzmq. My setup OS: Ubuntu 12.04 X86_64 zmq: 2.2.1 Python: 2.7.3 64bit pyzmq: 2.1.11 Python, zmq and pyzqm are self compiled with the usual options = #1 pyzmq + fork can crash = At first I tested how ZMQ handles fork(). Even the simple case with just a context object raises an assertion in ZMQ. When I don't call context.term() explicitly and pyzmq's Context.__del__() method run zmq_term() for me, the script crashes with a segfault. This is actually due to pyzmq calling close/term on gc, and was fixed in pyzmq master just this week. -MinRK --- Script 1 --- import zmq import os context = zmq.Context() print os.fork(), os.getpid() context.term() --- Output with context.term(): $ python testzmq.py 622 619 0 622 Assertion failed: ok (mailbox.cpp:84) Output without context.term(): python testzmq.py 777 773 15 0 777 15 Assertion failed: ok (mailbox.cpp:84) Abgebrochen (Speicherabzug geschrieben) The last message is German for segfault (core dumped) GDB output: (gdb) run testzmq.py ... (gdb) bt #0 0x7745e445 in __GI_raise (sig=optimized out) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64 #1 0x77461bab in __GI_abort () at abort.c:91 #2 0x7608ce4a in zmq::zmq_abort(char const*) () from /opt/vls/lib/libzmq.so.1 #3 0x7609061d in zmq::mailbox_t::recv(zmq::command_t*, int) () from /opt/vls/lib/libzmq.so.1 #4 0x7609c1fc in zmq::reaper_t::in_event() () from /opt/vls/lib/libzmq.so.1 #5 0x7608c036 in zmq::epoll_t::loop() () from /opt/vls/lib/libzmq.so.1 #6 0x7608c104 in zmq::epoll_t::worker_routine(void*) () from /opt/vls/lib/libzmq.so.1 #7 0x760a702a in thread_routine () from /opt/vls/lib/libzmq.so.1 #8 0x777ece9a in start_thread (arg=0x72a88700) at pthread_create.c:308 #9 0x7751a4bd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112 #10 0x in ?? () == #2 ZMQ doesn't set CLOEXEC == It seems to me that ZMQ doesn't set CLOEXEC on most of its file descriptors. I did a quick grep over the C++ code and could just find SOCK_CLOEXEC in src/ip.cpp. ZMQ already has an extensive usage of file descriptors. Some may even they it creates FDs like crazy. Without CLOEXEC all child processes inherit the FDs from their parent process, therefore increasing the total amount of FDs in the system and decreasing the amount of free FD slots of the child process. This is an issue for processes that fork()+execv() childs and use ZMQ to communicate with their children. It may even be a security issue when the parent process is privileged because a child process may be abke to directly temper with epoll fds and pipes of the parent's ZMQ context. This script shows that the parent's and child's FD count is both 15 (3 for std streams, 1 proc auxv, 2 eventpoll for ZMQ, 8 unix sockets for ZMQ and 1 unknown). --- script 2 --- import zmq import sys import os import subprocess context = zmq.Context() print parent, len(os.listdir(/proc/%i/fd % os.getpid())) print subprocess.check_output( [sys.executable, -c, import os; print child, len(os.listdir(/proc/%i/fd % os.getpid()))]) --- output: parent 15 child 15 Christian ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] ZeroMQ 3.2.0 stable RC1 is now available
On Jun 5, 2012, at 9:56 PM, Pieter Hintjens p...@imatix.com wrote: On Wed, Jun 6, 2012 at 12:01 AM, MinRK benjami...@gmail.com wrote: I know RC/beta terminology is just semantics, but are we really going straight to stable with no betas with a half-dozen relatively untested new features? Possibly. The new features aren't core, and we've fixed many core bugs so in theory, yes. However if it turns out the RC was a mistake, lesson learned. There's a catch-22 where until we push to stable, people don't use and we don't get stability. I know, and it's a tough choice. As a bindings author, I want to support every new idea so they are easy to play with and iron out, but I also don't want to waste too much time adding/removing support as things change in dev branches. It is very difficult to decide when, as a bindings maintainer, I should start looking to add support for new features, as they have been added and removed so frequently from libzmq. Me too. My own guideline is to ignore stuff until it's at least got a stable API. I think we have that for most of what's been pushed now. What commitment are you making with a libzmq-3.2.x stable release with regard to these new features? We have (now, finally) a contract for APIs that bans breakage. This is what I'd rely on in CZMQ. For instance, pyzmq ships its own implementation of zmq_device, because all of the 3.0/3.1/4.0 betas had removed it, but since the current state of 3.2.x means there will have been zero stable lbzmq releases without zmq_device, I should remove this code from pyzmq altogether. I'm sorry about that, it was a mistake to remove zmq_device but you know the story. However if you trust your own device code more than zmq_device (or if it does more), keep it. The bindings don't have to follow the core API slavishly. It's a verbatim copy from 2.1.6, updated with 3.x names, so not significant. I will probably remove it once there is a stable 3.x. I used to pretty aggressively support changes in 3.0-dev and even Martin's 4.0 experiment branch, but that proved a huge waste of effort, and I regret trying to support new features of libzmq-dev. Yes. We spent a year tracking massive and ultimately useless changes in 3-0 and 4-0, one reason I'm so happy this experimental work has forked off to xs. I don't think any of the changes in 3.2 have this style though; they're all incremental, and optional. It was fun, but a bit of a mess. I might still be doing it that way if I had more time to stay on top of things, and pyzmq was used by fewer folks than it is now. So from now on, I don't plan to add pyzmq support for any new features until they are present in a finished stable release. That's also what I planned with respect to the Guide: only document stable releases. -Pieter Ps. any chance we'll meet this weekend in SFO? Yes, I think I can make it (just added myself on the wiki). See you then, -MinRK ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Error While Building pyzmq
Wow, debug Python 3 on 64b Windows is *far* outside my home court. I guess my first question would have to be: can you build other extensions properly? -MinRK On Apr 23, 2012, at 23:04, Gargi Das gargi@hotmail.com wrote: Hello All, I am facing a problem when i am trying to build pyzmq for a debug version of python. I do the following steps: 1. build libzmq using visual studio the configuration selected is Win64 and debug. 2. cd into pyzmq directory and configure libzmq and pyzmq with command : python_d.exe setup.py configure --zmq=my zmq path 3. now using the same python_d.exe i try to call: python_d.exe setup.py install Step three gives me an error : running install running build running build_py copying zmq\libzmq.dll - build\lib.win-amd64-3.1-pydebug\zmq running build_ext building 'zmq.core.constants' extension C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\BIN\amd64\cl.exe /c /nologo /Ox /MD /W3 /GS- /DNDEBUG -Id:\Gargi\ZMQ\win64debug\zeromq -2.1.11\zeromq-2.1.11\include -Izmq\utils -Izmq\core -Izmq\devices -ID:\dev\GDas\install\products\dotrox\main\windows-amd64-vc_9_0_sp1-debug libs\bin\include -ID:\dev\GDas\install\products\dotrox\main\windows-amd64-vc_9_0_sp1-debuglibs\bin\PC /Tczmq\core\constants.c /Fobuild\temp. win-amd64-3.1-pydebug\Release\zmq\core\constants.obj constants.c zmq\utils\zmq_compat.h(130) : warning C4067: unexpected tokens following preprocessor directive - expected a newline C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\BIN\amd64\link.exe /DLL /nologo /INCREMENTAL:NO /LIBPATH:d:\Gargi\ZMQ\win64debug\zerom q-2.1.11\zeromq-2.1.11\lib /LIBPATH:D:\dev\GDas\install\products\dotrox\main\windows-amd64-vc_9_0_sp1-debuglibs\bin\libs /LIBPATH:D:\dev\GDa s\install\products\dotrox\main\windows-amd64-vc_9_0_sp1-debuglibs\bin\PCbuild\amd64 libzmq.lib /EXPORT:PyInit_constants build\temp.win-amd64 -3.1-pydebug\Release\zmq\core\constants.obj /OUT:build\lib.win-amd64-3.1-pydebug\zmq\core\constants.pyd /IMPLIB:build\temp.win-amd64-3.1-pyd ebug\Release\zmq\core\constants.lib /MANIFESTFILE:build\temp.win-amd64-3.1-pydebug\Release\zmq\core\constants.pyd.manifest LINK : fatal error LNK1104: cannot open file 'python31.lib' error: command 'C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\BIN\amd64\link.exe' failed with exit status 1104 [74795 refs] Important line in above log is LINK : fatal error LNK1104: cannot open file 'python31.lib' Now my question is why setup.py is trying to open python31.lib it should try to open python31_d.lib. Do i need to do some changes in any of the config files i tried to find that but was not able to find it. Just for saying i was able to build , libzmq and pyzmq for release version of python. I will be grateful if anyone can help me on this ?? Regards, Gargi ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] protocol design question
On Nov 15, 2011, at 12:45, Chuck Remes cremes.devl...@mac.com wrote:
On Nov 15, 2011, at 2:36 PM, MinRK wrote:
e.g.
struct sequence_id {
uint64 number;
uchar uuid[16];
};
Is this all right? Or is there a better way to accomplish this framing?
cr
I have a general question related to this - what is the relative cost in
zeromq of these being separate very-small messages versus one packed
message? Has someone done perf tests for this kind of thing?
I haven't tested it, but I assume that reading a multipart message versus a
single-part message is more time consuming due to 2 factors.
1. The I/O thread needs to make sure all parts are accounted for before
atomic delivery.
2. The #recv has to examine the message frame for the RCVMORE bit.
It's minor work, but as we all know, the fastest code is no code.
On a related topic, I packed these two details together because they are
logically related. The unique sequence_id is a combination of sequence_number
and uuid.
Right - I don't doubt that it's faster in this case, I was just wondering what
the general cost might be, noting that this is presumably always part of a
multipart message, so what is the cost of one extra msg part vs two, on top of
the real content of the message.
For instance, let's say I had two small strings to send that are *not* already
packed together in a struct, under what conditions is it preferable to pack
them into one message, and when is it better to leave them as separate?
MinRK
cr
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Problem Compiling Pyzmq
To build python extensions, you need the Python headers with 'apt-get install python-dev'. -MinRK On Jul 25, 2011, at 13:36, Antonio Teixeira eagle.anto...@gmail.com wrote: Hello Guys/Gals. I have tried to install pyzmq in ubuntu 11.04 Linux ubuntu 2.6.38-10-generic #46-Ubuntu SMP Tue Jun 28 15:05:41 UTC 2011 i686 i686 i386 GNU/Linux It fails With running build_ext building 'zmq.core.constants' extension gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -Izmq/utils -Izmq/core -Izmq/devices -I/usr/include/python2.7 -c zmq/core/constants.c -o build/temp.linux-i686-2.7/zmq/core/constants.o -Wno-unused-function -Wno-strict-aliasing zmq/core/constants.c:4:20: fatal error: Python.h: Unknow File / Dir compilation terminated. error: command 'gcc' failed with exit status 1 Installing Cython gave the same error that i was able to solve by using apt-get instead of easy_install. Best Regards A/T ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] Maybe a ZMQ_LOST socket
Hello, I'm enjoying with 0MQ at a job distribution pipeline. I've been using ROUTER-REQ based LRU workers and ROUTER-DEALER based workers with heartbeat which is used PPP for better reliability. In my project, client doesn't get response from workers, so to speak, fire and forget. So I can fully utilize PPP at client side. Anyway heartbeat worker was only answer to me, because LRU could not detect dead or disconnected REQ worker after it send READY. Even though heartbeat can detect the dead workers with higher probability, works can gone between the heartbeat interval. So a message can be lost, as I can utilize client side resend. Sure the Majordomo or shared state DBMS/Legacy MQ might solve the issue. For me it is a little painful and I know great reliability comes from more lines. I just expect the ROUTER socket has a way to keep history which there is an error to send a message because of lost peer or invalid address. As sending messages are asynchronous, returning FALSE at send doesn't make sense. Instead, what if another socket buffer, ZMQ_LOST, could gather the lost messages only when programmer declare the ZMQ_LOST socket and attach to the ROUTER. By polling the ZMQ_LOST socket, broker can retry or log itself. It might break design philosophy of 0MQ.But am I the only person who wants more built-in(??) reliability? :) Thanks Min ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Maybe a ZMQ_LOST socket
Thanks for the response. I just hoped the broker could look into or poll the timeout-ed dead letters at a further milestone. Thanks Min 2011/6/30 Pieter Hintjens p...@imatix.com On Wed, Jun 29, 2011 at 4:21 PM, Martin Sustrik sust...@250bpm.com wrote: This is called dead letter queue in MQ-speak. However, it probably doesn't have the semantics you expect it to have. It's not a queue of lost messages, rather it's a queue of messages with unclear status. They may have been lost, but they may have been delivered (only the delivery acknowledgement was lost). In any case, ROUTER will queue and hold messages to peers that have gone offline, and the only way to dead-letter these would be with a timeout on connection (at which stage any undelivered messages could be dead-lettered). As for invalid addresses, since this is an application error by definition, I'd argue it's worth making this assert. (Especially if we kill explicit identities and thus any scenario where ROUTER discarding a message makes sense). -Pieter ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] pyzmq Windows binaries
It was entirely an accident, due to my lack of experience on Windows. I will push replacements, later today. Thanks! -MinRK On Apr 13, 2011, at 7:06, Ben James bmja...@gmail.com wrote: Hi, I recently found that there are Windows installers for pyzmq at https://github.com/zeromq/pyzmq/downloads - this is really useful! I used pyzmq-2.1.4.win32-py2.7.msi and found that the libzmq DLL which is packaged with this seems to be a debug build, which means that I still need to have Visual Studio installed to use it, or else: Activation context generation failed for C:\Python27\lib\site-packages\zmq\libzmq.dll. Dependent Assembly Microsoft.VC90.DebugCRT,processorArchitecture=x86,publicKeyToken=1fc8b3b9a1e18e3b,type=win32,version=9.0.21022.8 could not be found. Please use sxstrace.exe for detailed diagnosis. Was this deliberate or was it an accident to provide debug builds? Ben ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Important: backward incompatible changes for 0MQ/3.0!
On Mar 22, 2011, at 16:31, Pieter Hintjens p...@imatix.com wrote: On Wed, Mar 23, 2011 at 12:08 AM, MinRK benjami...@gmail.com wrote: I use PAIR quite a bit, because many of my small cases really are symmetric a=b connections (not REQ/REP pattern). Frankly, I can easily use XREQ for both sides if PAIR is gone, and it works as long as additional connections don't happen, but if they do things will go wrong. I'd rather have the error raised by PAIR than weird message loss that would result from XREQ. What would be the recommended socket type(s) for a symmetric pair of sockets with flexible send/recv pattern if PAIR is removed? It's unlikely PAIR will be removed if there's proof of active use. However, you raise an interesting point. Perhaps it's possible to get the same results without having a distinct socket type. For example, DEALER to DEALER (xreq/xreq) with a restriction of 1 connection per socket. That would exactly solve my PAIR cases. Martin S. has already discussed adding this ability to limit connections on a socket. Does this suggest that it would also be possible to query the number of connections? That would be very useful for some cases. -Pieter ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] More security thoughts
On Nov 17, 2010, at 0:15, Martin Sustrik sust...@250bpm.com wrote: Hi Brian, Summary: even if we can figure out how to make message level security bulletproof, there are some serious performance issues. Great analysis! At the moment I see 2 solutions to the performance problem: 1. Create an inproc encrypting/decrypting device, send the messages through the device. That would certainly work for c/c++, but unfortunately not in Python, since you don't really get any benefit from multiple Python threads that touch the GIL (by accessing any Python variables). It could work in Python if the intermediate thread were entirely GIL-less C extension code. 2. The one you proposed: Create a specialised message encrypting transport within 0MQ. We would be *super* excited to see a SSL/TLS transport or similar in 0MQ. Summary: tunnels have a serious security hole in the untrusted localhost environment. Tunnels only work for securing remote connections. No idea how can this be solved. What about using virtual machines to achieve secury multitenancy? VM would certainly address the issue, but at a rather steep resource cost, particularly since the multiuser case it is meant to address means other programs can be expected to be running. -MinRK Martin ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] problem building pyzmq
My apologies for the doc being unclear. pyzmq release downloads are located at: https://github.com/zeromq/pyzmq just click the 'Downloads' button on the right. As soon as I get a confirmation from Brian, there will be 2.0.10 up there. -MinRK On Nov 13, 2010, at 11:09, eric e...@ericjbell.com wrote: Thank you. That explains a lot. FYI, the text and links on http://www.zeromq.org/area:download imply that the current release _is_ 2.0.10, that it was release on October 15th, and that it is stable: ØMQ comes as source code licensed under LGPLv3+. The latest version is 2.0.10 (stable), released on 2010/10/15: * zeromq-2.0.10.tar.gz for POSIX systems * zeromq-2.0.10.zip for Windows If there is a way to download previous versions, stable or not, it is not obvious where those can be found, so from a non-insider's perspective there is no way to get a non-development stable release. --eric On 11/12/2010 09:46 PM, Min RK wrote: You are using the current development branch, the website points to our github development branch, and we haven't cut a release since 2.0.8, due to refactoring and time. We have one feature pending review, then we will cut a 2.0.10 release, hopefully tomorrow. That will not require you to have cython. Yes, I believe we do depend on current cython for development, which is 0.13, and pip/easy_install-able. ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] problem building pyzmq
You did not download a stable release, you downloaded our current development branch. If you get the code via git, it is not a release. It is an imprecise message: you need cython to develop the code *or* to build from our development repository. You checked out our development branch, which the doc currently points to. The reason is that we don't track the autogenerated .c files in git, so you have to use cython to generate them unless you download a zip/tar release where we have done the cython step for you. -MinRK On Nov 13, 2010, at 11:38, eric e...@ericjbell.com wrote: I've actually understood the difference between 0MQ and the PyZMQ binding project. I just figured out that 0MQ is on 2.0.10, whereas PyZMQ is 2.0.8 ... so when MinRK referred to PyZMQ's upcoming 2.0.10 release, I got the version numbering mixed up which version number was referring to the core and which was referring to the Python binding. What I am still confused by is something I read somewhere about requiring Cython *IF* you are doing PyZMQ development, implying that I don't need Cython if I am not doing PyZMQ development. Since I downloaded a stable release of 0MQ, it seemed to me that I didn't need to have Cython, but it appears that I do. --eric On 11/13/2010 11:25 AM, Martin Sustrik wrote: Hi, My understanding is that Eric have been confused by 0MQ core and PyZMQ being two separate projects, having different version numbers and releases. The text Eric mentions discusses versions of 0MQ core while Min have referred to PyZMQ versions. I would say we should explain that 0MQ+language bindings is not a single project on a well-visible spot in the wiki. Ideas anyone? Martin ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] problem building pyzmq
Your cython is broken if you are seeing that error. How did you install cython? -MinRK On Nov 12, 2010, at 20:17, eric e...@ericjbell.com wrote: I have a related problem ... I am unable to build pyzmq following the instructions for non-development release. Specifically: I am trying to install a non-development (stable) release as described on http://www.zeromq.org/intro:get-the-software I downloaded the 2.0.10 release for POSIX using the link on this page, which describes the release as stable. I then followed the instructions to build pyzmq, which are on the same page. When I follow the instructions, I get the following output and error. e...@confidence:~/Desktop/pyzmq$ sudo python setup.py install running install running build running build_py running build_ext error: Cython-generated file 'zmq/core/context.c' not found. Cython is required to compile pyzmq from a development branch. Please install Cython or download a release package of pyzmq. When I install Cython I get the error reported by Martin. Help? --eric On 11/11/2010 03:36 AM, Martin Sustrik wrote: Hi, I've tried to compile pyzmq myself and I am getting this error: cythoning zmq/core/constants.pyx to zmq/core/constants.c Error converting Pyrex file to C: ... # Imports #- # from libc.stdlib cimport free,malloc from cpython cimport bool ^ /home/sustrik/pyzmq/zmq/core/czmq.pxd:28:0: 'cpython.pxd' not found I have cython 0.13 installed. cpython.pxd doesn't appear to be present on my system. Any ideas? Martin ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] problem building pyzmq
You are using the current development branch, the website points to our github development branch, and we haven't cut a release since 2.0.8, due to refactoring and time. We have one feature pending review, then we will cut a 2.0.10 release, hopefully tomorrow. That will not require you to have cython. Yes, I believe we do depend on current cython for development, which is 0.13, and pip/easy_install-able. -MinRK On Nov 12, 2010, at 20:28, eric e...@ericjbell.com wrote: But why am I having to download cython at all if I am trying to install a non-development release? --eric On 11/12/2010 08:20 PM, Min RK wrote: Your cython is broken if you are seeing that error. How did you install cython? -MinRK On Nov 12, 2010, at 20:17, erice...@ericjbell.com wrote: I have a related problem ... I am unable to build pyzmq following the instructions for non-development release. Specifically: I am trying to install a non-development (stable) release as described on http://www.zeromq.org/intro:get-the-software I downloaded the 2.0.10 release for POSIX using the link on this page, which describes the release as stable. I then followed the instructions to build pyzmq, which are on the same page. When I follow the instructions, I get the following output and error. e...@confidence:~/Desktop/pyzmq$ sudo python setup.py install running install running build running build_py running build_ext error: Cython-generated file 'zmq/core/context.c' not found. Cython is required to compile pyzmq from a development branch. Please install Cython or download a release package of pyzmq. When I install Cython I get the error reported by Martin. Help? --eric On 11/11/2010 03:36 AM, Martin Sustrik wrote: Hi, I've tried to compile pyzmq myself and I am getting this error: cythoning zmq/core/constants.pyx to zmq/core/constants.c Error converting Pyrex file to C: ... # Imports #- # from libc.stdlib cimport free,malloc from cpython cimport bool ^ /home/sustrik/pyzmq/zmq/core/czmq.pxd:28:0: 'cpython.pxd' not found I have cython 0.13 installed. cpython.pxd doesn't appear to be present on my system. Any ideas? Martin ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] EncryptedSocket added to pyzmq in branch
There's no encrypted bit or anything. This isn't 'Encrypted ZMQ', it's just an object that encrypts messages prior to sending them. Think of it more as a wrapper than anything. The messaging library has no sense of whether a message is encrypted or not, you have to specify that in user code: a.send(msg, encrypt=True) must be matched with: b.recv(decrypt=True) Otherwise b would just receive the encrypted buffer, which is perfectly valid, if for instance b is just going to relay it to c without looking at it, and c is the real destination. Another example, in the sample code, I used blowfish to encrypt the messages. Any program in any language that has access to a blowfish implementation (and the original password) can understand the messages, and any zmq implementation can receive the encrypted messages. This follows in general: The EncryptedSocket is passed an opaque cypher object, and it is up to the user to ensure that the destination has a complementary cypher. This is encryption *on top of* zmq, not inside it. By the way, we would really love it if an SSL/TLS transport were part of the underlying ZMQ. As the earlier security discussion pointed out, message encryption and transport encryption are different things, and both are valuable. Transport encryption would be part of zmq itself, but message encryption can be done entirely without the knowledge of the library, as is done here, since an encrypted message is itself just a message. -MinRK On Nov 2, 2010, at 0:58, Mikael Helbo Kjær m...@designtech.dk wrote: Hi Brian, We are not security experts and would love feedback on this design. It is quite simple, but for many of us, security is the show stopper with 0MQ. This at least gets us moving in the right direction. I am not a security expert either, but anyway: nice! One problem I can see is that you can enable/disable encryption on per-message basis, which presumably means you have encryption bit stored somewhere in the message body. That in turn means pyzmq has its own wire protocol and is not able to transparently interoperate with other language bindings. Am I wrong? If they've done it like that you're right. Either each binding implements an equivalent or they can't understand the payload. That makes me wonder if the message encryption stuff belongs underneath the 0mq API or maybe in zfl so that all the bindings can at least share the model for communication and collectively improve it over time. I don't mean the transport but rather some form of function that initiates a shared key negotiation (via a few messages akin to SSL/TLS really) and maybe then negotiates the secure session (which is needed to be really secure I fear from my understanding of network security). Regards, Mikael ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] EncryptedSocket added to pyzmq in branch
On Nov 2, 2010, at 8:51, Burak Arslan burak.ars...@arskom.com.tr wrote: On 11/02/10 15:34, Brian Granger wrote: On Tue, Nov 2, 2010 at 3:53 AM, Burak Arslan burak.ars...@arskom.com.tr wrote: On 11/02/10 10:31, Min RK wrote: There's no encrypted bit or anything. This isn't 'Encrypted ZMQ', it's just an object that encrypts messages prior to sending them. Think of it more as a wrapper than anything. The messaging library has no sense of whether a message is encrypted or not, you have to specify that in user code: a.send(msg, encrypt=True) must be matched with: b.recv(decrypt=True) Otherwise b would just receive the encrypted buffer, which is perfectly valid, if for instance b is just going to relay it to c without looking at it, and c is the real destination. objections: 1) this has nothing to do with zeromq. so it shouldn't be part of the python zmq package. i suggest creating a package like zmq.patterns and moving it there. Our approach with pyzmq is divide the cosebase into a core subpackage that only has the python bindings to the zmq library, and then other subpackages that add capabilities on top of that. For example, we have an eventloop and loggin subpackage that provide a simple event loop or pyzmq and pyzmq enabled tools for python's logging module. The same is true of the encryption stuff, it is not in the core. We simply don't have the manpower to start separate projects for each things like this, nor do we want to have projects that depend pyzmq ha this is my mistake, i thought the patch went into core/socket.pyx. it really went into its own module. but, i insist that this should be removed from the official pyzmq package. This isn't in the official pyzmq, it's in my branch, as mentioned in the original post. Hence the post here, to receive input. 1) it's a functionality that core zeromq, nor no other zeromq bindings have. you can always maintain your own fork in your own personal github account, but the one in github.com/zeromq should only be a pure wrapper for the zeromq, nothing more. pyzmq is really two things: zmq.core: python bindings for zmq and anything not in zmq.core is additional functionality built on top of zmq, such as special devices, logging handlers, etc. 2) the proposed encryption scheme is not only useless, but it's dangerous (see below) this is bad advertisement for a library that has two hours spent for every single line of code. We don't propose an encryption scheme at all. We just added an object that performs an arbitrary user-specified transform to messages before/after passing them to zmq. it's up to the user to pick a good one. 2) it can't tell ciphertext from plaintext. as previously mentioned, this ability requires changes to the wire-protocol. if security-related changes are to be introduced, i'd prefer to start at socket identity signing. (a la dnssec) you can imagine that this is a HUGE can of worms, and i think zeromq has much more security-related problems until we come to that. I don't quit see how this follows. In all the cases that we plan on using it, we simple write our endpoint logic to know when it should expect encrypted data and when it should not. Most of the time, everything will be encrypted, so no decisions will have to be made. Also, data that doesn't match the expectation will fail our deserialization attempts and we will just reject it. detecting formats are useful because intra-node networks tend to be trusted. thus you'll save cpu-cycles by avoiding unnecessary encryption. and if the numbers you reported are correct, i think you'd waste too much for unnecessary encryption. on a related note, i also strongly advise against simply using encryption without signing because you can't detect data corruption. in cbc-like encryption modes, this means one bit flip can corrupt the entire message, and you won't be able to detect it. Yes, we definiitely cannot detect data corruption. But, this is no worse than a regular 0MQ connection, so minimally, we are not making the situation worse. i'm sorry brian but this is nonsense. normally, one bit flip is just that, one flipped bit. it's local, it stays where it is. if you're transporting e.g. voip frames, it's won't even be audible. but a bit flip in a chain-encrypted stream messes up *everything* that comes after. yes, your deserializer can notice this if you're transporting tightly structured data, and if you're lucky. but if you're transporting a stream of numbers, or that bit flip happens where you have arbitrary data in your structure, you won't detect anything. not convinced? read how a single bit flip in a handful of messages took the entire amazon service infrastructure down: http://status.aws.amazon.com/s3-20080720.html (...) More specifically, we found that there were a handful of messages on Sunday morning that had a single bit corrupted
