Re: [zeromq-dev] zeromq, rewritten in Rust

2014-06-12 Thread Fantix King
Hi Pieter,

I've updated the README file according to the requests and recommendation
in docs:organization, and also draw some diagrams for a simple design
document here:

https://github.com/decentfox/zmq.rs/wiki/Design

Hopefully this could make the code clearer and easier to be co-worked on.

Please advice if something can be done better. :) Thank you!



BR,
Fantix
--
http://about.me/fantix


On Wed, Jun 11, 2014 at 4:16 PM, Fantix King fantix.k...@gmail.com wrote:

 Oh yes, I'd love to move into the community organization! Let me update
 the README file (MPLv2 and C4.1 look good) and do some more document work.

 Thank you!


 BR,
 Fantix
 --
 http://about.me/fantix


 On Wed, Jun 11, 2014 at 3:59 PM, Pieter Hintjens p...@imatix.com wrote:

 Hi Fantix,

 Nice stuff. If you want to move this into the ZeroMQ community
 organization, see http://zeromq.org/docs:organization

 -Pieter

 On Wed, Jun 11, 2014 at 6:40 AM, Fantix King fantix.k...@gmail.com
 wrote:
  I think it might be easier to have discussions if there is actual code -
  I've started a project zmq.rs with some very basic scratches:
 
  https://github.com/decentfox/zmq.rs
 
  The code now is meant to be discussed and heavily changed incrementally,
  hopefully with tests carefully covered. Please feel free to drop by and
  comment if you are interested, it is truly appreciated.
 
 
  BR,
  Fantix
  --
  http://about.me/fantix
 
 
  On Tue, Mar 4, 2014 at 10:55 AM, Fantix King fantix.k...@gmail.com
 wrote:
 
  Sounds really exciting! Please share the project link when there is
 one if
  possible.
 
  BR,
  Fantix
  --
  http://about.me/fantix
 
 
  On Tue, Mar 4, 2014 at 2:26 AM, Charles Remes li...@chuckremes.com
  wrote:
 
  There was a little twitter chat over the weekend regarding an attempt
 at
  writing a ground-up zeromq library using the new systems language
 Rust. If
  you haven't heard of Rust, it is a new language under development by
 the
  good folks at Mozilla. It's original designer has said that he has
 learned
  quite a few things from implementing dozens of languages over the
 years that
  he felt he could solve some new problems and create a cleaner
 language. Rust
  is his attempt at such a feat.
 
  It supposedly solves the problem by borrowing the best from many
 popular
  languages.
 
  * OOP of C++ without the large, unwieldy syntax
 
  * performance of C while providing good namespacing, OOP, safe
  memory (i.e. no dangling pointers)
 
  * the functional expressiveness of Haskell but not at the
 expense
  of imperative forms
 
  * the massive concurrency of Erlang but with a better syntax
 and
  a more flexible memory model (borrowed pointers, immutable defaults,
 etc)
 
  I recently did a small test project to learn the syntax. The language
 is
  still evolving, so it's a bit of a moving target. It's at release 0.9
 with a
  1.0 slated for later this year, but they've already slipped on
 delivering a
  1.0 for at least a year so I assume it will slip again.
 
  Anyway, I'd like to volunteer to try and spike a simple example to get
  things started. However, I'd like to start a thread here to discuss
 lessons
  learned from the existing codebase. We already have a great write-up
 from
  Martin Sustrik (primary author of earlier versions of zeromq) here:
 
  http://250bpm.com/blog:4
 
  http://250bpm.com/blog:8
 
  I'm hoping that others who have read through the source have
 additional
  insights that they'd like to share. For instance, I have seen
 comments that
  zeromq might have more consistent performance it it was wrapped
 around a
  Disruptor (google for that pattern if it's new to you). People also
 seem to
  really dislike the concept of the context (nanomsg has already
 eliminated
  this... it still exists but is hidden by the library).
 
  Any other insights?
 
  cr
 
  ___
  zeromq-dev mailing list
  zeromq-dev@lists.zeromq.org
  http://lists.zeromq.org/mailman/listinfo/zeromq-dev
 
 
 
 
  ___
  zeromq-dev mailing list
  zeromq-dev@lists.zeromq.org
  http://lists.zeromq.org/mailman/listinfo/zeromq-dev
 
 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev



___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev


Re: [zeromq-dev] on scalability of PUB/SUB and PUSH/PULL

2014-06-12 Thread Goswin von Brederlow
On Wed, Jun 11, 2014 at 02:05:13AM -0700, Jun Li wrote:
 Hi,
 
 I am using PUB/SUB socket pattern to distribute commands from the
 coordinator to the many worker processes, and I also have the PUSH/PULL to
 have each worker process to push the processing results to the coordinator.
 The coordinator is bound to the PUB socket and also the PULL socket, with
 the current context to set to 1 thread.   In my test environment, there
 would be one single coordinator process and up to 200 worker processes.
 
 I have just started the scalability testing. But it seems that with 15
 worker processes, the end-to-end communication latency is about 15 ms, for
 the coordinator to distribute (via PUB) the commands and finally aggregate
 the results back (via PULL) from the worker processes. But when I increased
 the number of worker processes to 50, I then observed the end-to-end
 communication latency of about 80 ms. This implies that as the number of
 the worker processes grow, the latency also grows and thus brings up the
 scalability issue.

You can hardly say anything with just to points. Is that a linear
increase? exponential? logarithmic? Does is jump between 49 and 50? 
Does it stay at 80ms up to 10 workers?

 The message size communicated between the coordinator and the worker
 processes are not that big, less than 100 Bytes.
 
 While I am planning to measure the latency spent on each hop, I would like
 to seek suggestions:
 
 *for a large number of the worker processes to be handled by a single
 coordinator with low latency, should the context at the coordinator be set
 to   1 thread?
 
 *Should I use the other socket pattern such as Router/Dealer, instead of
 pub/sub and push/pull, in order to address the scalability issue?
 
 Regards,
 
 Jun

Personally I think that if you depend on latency then you always have
a problem. That will be your bottleneck and seriously harm
scalability. You need to pipeline your work, send out more jobs ahead
of time while the workers are still busy with the last job. That way
the latency gets combletly absorbed and becomes irelevant.

MfG
Goswin
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev


Re: [zeromq-dev] How to view current SUBs and their filters from PUB side?

2014-06-12 Thread Goswin von Brederlow
You can get infos through a monitoring socket and through a ZAP handler.

MfG
Goswin

On Thu, Jun 12, 2014 at 10:44:49AM +0800, ? wrote:
 I'm looking for debug and/or log information. Thanks for your suggestion!
 - Zhichang
 
 
 2014-06-12 10:25 GMT+08:00 Michel Pelletier pelletier.mic...@gmail.com:
 
  That would break the transport abstraction in 0mq, not all transports are
  IP based.  If you're thinking of using IP addresses as a
  whitelist/blacklist security mechanism, I would suggest looking at the
  curve security stuff instead.  IP based security is easily defeated.
 
  http://curvezmq.org/
 
  If you're looking to just log connections, you can get that from the OS,
  for example on Linux:
 
  http://gr8idea.info/os/tutorials/security/iptables5.html
 
  -Michel
  On Wed, Jun 11, 2014 at 7:16 PM, ? yuzhich...@gmail.com wrote:
 
  Hi Michel,
  XPUB looks good. However I want more details info: the SUBs' address
  info(IP + TCP port). Is that possible?
  Regards,
  Zhichang
 
 
  2014-06-12 9:40 GMT+08:00 Michel Pelletier pelletier.mic...@gmail.com:
 
  Subscriptions are received in the form of incoming messages you get by
  calling recv on the xpub socket.
 
  http://api.zeromq.org/3-2:zmq-socket
 
  Same as ZMQ_PUB except that you can receive subscriptions from the
  peers in form of incoming messages. Subscription message is a byte 1 (for
  subscriptions) or byte 0 (for unsubscriptions) followed by the 
  subscription
  body.
 
  -Michel
 
 
  On Wed, Jun 11, 2014 at 6:26 PM, ? yuzhich...@gmail.com wrote:
 
  I don't see routines to dump xpub_t::subscriptions.
 
  Regards,
  Zhichang
 
  ___
  zeromq-dev mailing list
  zeromq-dev@lists.zeromq.org
  http://lists.zeromq.org/mailman/listinfo/zeromq-dev
 
 
 
  ___
  zeromq-dev mailing list
  zeromq-dev@lists.zeromq.org
  http://lists.zeromq.org/mailman/listinfo/zeromq-dev
 
 
 
  ___
  zeromq-dev mailing list
  zeromq-dev@lists.zeromq.org
  http://lists.zeromq.org/mailman/listinfo/zeromq-dev
 
 
 
  ___
  zeromq-dev mailing list
  zeromq-dev@lists.zeromq.org
  http://lists.zeromq.org/mailman/listinfo/zeromq-dev
 
 

 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev

___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev


Re: [zeromq-dev] How to view current SUBs and their filters from PUB side?

2014-06-12 Thread 余志昌
Hi MfG,
Do you mean use zmq_proxy() to monitor inbound messages of a XPUB
socket?
- Zhichang


2014-06-12 16:06 GMT+08:00 Goswin von Brederlow goswin-...@web.de:

 You can get infos through a monitoring socket and through a ZAP handler.

 MfG
 Goswin

 On Thu, Jun 12, 2014 at 10:44:49AM +0800, ? wrote:
  I'm looking for debug and/or log information. Thanks for your suggestion!
  - Zhichang
 
 
  2014-06-12 10:25 GMT+08:00 Michel Pelletier pelletier.mic...@gmail.com
 :
 
   That would break the transport abstraction in 0mq, not all transports
 are
   IP based.  If you're thinking of using IP addresses as a
   whitelist/blacklist security mechanism, I would suggest looking at the
   curve security stuff instead.  IP based security is easily defeated.
  
   http://curvezmq.org/
  
   If you're looking to just log connections, you can get that from the
 OS,
   for example on Linux:
  
   http://gr8idea.info/os/tutorials/security/iptables5.html
  
   -Michel
   On Wed, Jun 11, 2014 at 7:16 PM, ? yuzhich...@gmail.com
 wrote:
  
   Hi Michel,
   XPUB looks good. However I want more details info: the SUBs'
 address
   info(IP + TCP port). Is that possible?
   Regards,
   Zhichang
  
  
   2014-06-12 9:40 GMT+08:00 Michel Pelletier 
 pelletier.mic...@gmail.com:
  
   Subscriptions are received in the form of incoming messages you get by
   calling recv on the xpub socket.
  
   http://api.zeromq.org/3-2:zmq-socket
  
   Same as ZMQ_PUB except that you can receive subscriptions from the
   peers in form of incoming messages. Subscription message is a byte 1
 (for
   subscriptions) or byte 0 (for unsubscriptions) followed by the
 subscription
   body.
  
   -Michel
  
  
   On Wed, Jun 11, 2014 at 6:26 PM, ? yuzhich...@gmail.com
 wrote:
  
   I don't see routines to dump xpub_t::subscriptions.
  
   Regards,
   Zhichang
  
   ___
   zeromq-dev mailing list
   zeromq-dev@lists.zeromq.org
   http://lists.zeromq.org/mailman/listinfo/zeromq-dev
  
  
  
   ___
   zeromq-dev mailing list
   zeromq-dev@lists.zeromq.org
   http://lists.zeromq.org/mailman/listinfo/zeromq-dev
  
  
  
   ___
   zeromq-dev mailing list
   zeromq-dev@lists.zeromq.org
   http://lists.zeromq.org/mailman/listinfo/zeromq-dev
  
  
  
   ___
   zeromq-dev mailing list
   zeromq-dev@lists.zeromq.org
   http://lists.zeromq.org/mailman/listinfo/zeromq-dev
  
  

  ___
  zeromq-dev mailing list
  zeromq-dev@lists.zeromq.org
  http://lists.zeromq.org/mailman/listinfo/zeromq-dev

 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev

___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev


[zeromq-dev] zmsg.hpp

2014-06-12 Thread ??????
Hi,everyone!
Where could i find zmsg.hpp this head file???


Thanks!___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev


Re: [zeromq-dev] PUB ZMQ_SNDHWM doesn't work

2014-06-12 Thread Brian Knox
Setting the HWM on a PUB socket to 0 would have the opposite effect of what
you want.  PUB sockets, when exceeding the HWM, drop messages.

For a description of what different socket types do in a mute state, please
see the documentation:

http://api.zeromq.org/4-0:zmq-socket

You should probably spend some time reading the zeromq guide as well.

Cheers,
Brian


On Wed, Jun 11, 2014 at 9:24 PM, 余志昌 yuzhich...@gmail.com wrote:

 Hi,
 I tried to set hwm to zero at both sides of pubsub in order to avoid
 dropping messages. (I'm  aware with possible memory overflow at this time.)
 However the PUB side ZMQ_SNDHWM customization doesn't work. Here's my
 PUB side code piece:

 // Socket to send messages to
 void *publisher = zmq_socket(context, ZMQ_PUB);

  // Raise the high water mark for outbound messages to unlimited.
 int sndhwm = 0;
 rc = zmq_setsockopt(publisher, ZMQ_SNDHWM, sndhwm, sizeof(int));
  if(rc!=0)
 printf(%s:%d zmq_setsockopt() errno %d!\n, __FILE__, __LINE__, errno);
 assert(rc==0);
  sndhwm = -9;
 size_t s = sizeof(int);
 rc = zmq_getsockopt(publisher, ZMQ_SNDHWM, sndhwm, s);
  if(rc!=0)
 printf(%s:%d zmq_getsockopt() errno %d!\n, __FILE__, __LINE__, errno);
 printf(sndhwm: %d\n, sndhwm);
  assert(rc==0);
  rc = zmq_bind(publisher, SWITCH_OUT);
  if(rc!=0)
 cerr  __FILE__  :  __LINE__   zmq_bind() errno   errno 
 endl;
  assert(rc == 0);

 Here's gdb output(hwm is 1000. This doesn't match my expectation):

 (gdb) break pipe.cpp:157
 Breakpoint 2 at 0x7fdbad9a7f30: file ../../src/pipe.cpp, line 157.
 (gdb) c
 Continuing.

 Breakpoint 2, zmq::pipe_t::check_write (this=0x134c8c0) at
 ../../src/pipe.cpp:157
 157 in ../../src/pipe.cpp
 (gdb) bt
 #0  zmq::pipe_t::check_write (this=0x134c8c0) at ../../src/pipe.cpp:157
 #1  0x7fdbad9a95a3 in zmq::pipe_t::write (this=0x134c8c0,
 msg_=0x7fff1280abf0) at ../../src/pipe.cpp:166
 #2  0x7fdbad99be01 in zmq::dist_t::write (this=0x1302cb8,
 pipe_=0x134c8c0, msg_=0x3e8) at ../../src/dist.cpp:181
 #3  0x7fdbad99bf93 in zmq::dist_t::distribute (this=0x1302cb8,
 msg_=0x7fff1280abf0, flags_=value optimized out) at ../../src/dist.cpp:145
 #4  0x7fdbad99c17e in zmq::dist_t::send_to_matching (this=0x1302cb8,
 msg_=0x7fff1280abf0, flags_=2) at ../../src/dist.cpp:118
 #5  0x7fdbad9bc2be in zmq::xpub_t::xsend (this=0x1302900,
 msg_=0x7fff1280abf0, flags_=2) at ../../src/xpub.cpp:129
 #6  0x7fdbad9b02d0 in zmq::socket_base_t::send (this=0x1302900,
 msg_=0x7fff1280abf0, flags_=2) at ../../src/socket_base.cpp:656
 #7  0x7fdbad9c365a in s_sendmsg (s_=0x1302900, msg_=0x7fff1280abf0,
 flags_=2) at ../../src/zmq.cpp:337
 #8  0x7fdbad9c3c7e in zmq_send (s_=0x1302900, buf_=0x12f4110, len_=2,
 flags_=2) at ../../src/zmq.cpp:362
 #9  0x00404810 in main (argc=value optimized out, argv=value
 optimized out) at ../src/switch.cpp:395
 (gdb) p hwm
 $1 = 1000
 (gdb) p lwm
 $2 = 500


 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev


___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev


Re: [zeromq-dev] ROUTER tcp socket stuck in CLOSE_WAIT with large receive queue

2014-06-12 Thread Pieter Hintjens
I've seen something similar (I think) with Zyre, where dealer sockets
connecting with the same identity do weird things. Try setting
ZMQ_ROUTER_HANDOVER on the router socket, see if that helps (you'll
need libzmq master).

On Thu, Jun 12, 2014 at 4:15 AM, Sash Nagarkar s...@dronedeploy.com wrote:
 Hello ZMQ devs,

 We're using PyZMQ 14.3.0 and libzmq 4.0.4 with a ROUTER-DEALER pattern
 for a service we're providing.  Sorry if this is too verbose, and I
 hope this is the right place to ask the question.

 TL;DR: ROUTER socket doesn't receive messages from a DEALER even
 though netstat shows several megabytes in the TCP receive queue
 (nothing in the send queue).  Other connected DEALERs work fine.

 The ROUTER socket is running on a server with ample CPU  memory
 headroom, with several DEALER clients that connect, exchange messages,
 and can abruptly disconnect repeatedly.  We're exclusively using
 multipart messages with the first part always being the ZMQ socket
 identity, which persists across DEALER connect/disconnects.  In other
 words, each DEALER client uses the same socket identity across many
 connects and disconnects.

 Most of the time, things hum along smoothly (several thousand messages
 exchanged, several dozen connect/disconnects).  However, every once in
 a rare while, we see that one of the DEALER clients connects and sends
 messages to the ROUTER that end up never making it to the ROUTER
 process.  The ROUTER process continues to receive messages from other
 DEALER clients.

 Further debugging on the ROUTER server shows one (or more) TCP
 connections from the client DEALER that are in the CLOSE_WAIT state
 with several megabytes of data sitting in the receive queue to the
 ROUTER.  We also see one connection from the client DEALER in the
 ESTABLISHED state with a receive queue that is growing.

 It's clear that the DEALER client died abruptly once, but then
 returned with the same identity and resumed sending messages to the
 ROUTER.  However, none of the subsequent messages are delivered to the
 ROUTER process.  Any ideas on why this would be the case?

 I would have provided a test case, but we aren't able to consistently
 reproduce the issue.  I've copied the output from netstat (with
 obfuscated IPs) below, in case it helps.


 Questions:
 - What would cause the receive queue to fill up like this on a ROUTER
 while it continues to receive messages from other clients?  It's clear
 that the messages are all making it to the ROUTER machine.
 - Is it safe for DEALER sockets to abruptly disconnect and then reuse
 their socket identity?
 - How can we mitigate this situation?  The closest thing I see is
 ZMQ_LINGER, but that applies only to the outgoing queue and not the
 incoming one.
 - Is there anything I could investigate myself to figure out whether
 this is an issue in PyZMQ vs. libzmq?  Where should I start?


 Other potentially relevant info:
 - The ROUTER uses PyZMQ's zmq.Poller() to receive messages from the
 problem socket and some others.  All other nodes in the system
 continue to send and receive messages just fine.
 - The ROUTER's send queues are pretty much empty.
 - We see the same behavior with libzmq 4.0.4 and libzmq 2.2.x, on Ubuntu 
 14.04.


 $ netstat -a
 Active Internet connections (servers and established)
 Proto Recv-Q Send-Q Local Address   Foreign Address State
 tcp0  0 *:12501  *:* LISTEN
 tcp   1816956  0 server-ip.:12501 clientA-ip:42571 CLOSE_WAIT
 tcp   1551036  0 server-ip.:12501 clientA-ip:42858 CLOSE_WAIT
 tcp0  0 server-ip.:12501 clientB-ip:34000 ESTABLISHED
 tcp   5265541  0 server-ip.:12501 clientA-ip:43469 ESTABLISHED


 Please let me if further information would help.  Thank you for
 helping build ZMQ, it's been a huge pleasure to work with so far.

 Cheers,
 Sash
 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev


Re: [zeromq-dev] zeromq, rewritten in Rust

2014-06-12 Thread Pieter Hintjens
Hi Fantix, you are now a member of the ZeroMQ owners team, with all
the power and responsibility. :-) Welcome.

The next step is to rename/move the repository to zeromq and then fork
it back to your personal account.

It's really nice to have (I think) our first Chinese-originated project.

-Pieter

On Thu, Jun 12, 2014 at 9:25 AM, Fantix King fantix.k...@gmail.com wrote:
 Hi Pieter,

 I've updated the README file according to the requests and recommendation in
 docs:organization, and also draw some diagrams for a simple design document
 here:

 https://github.com/decentfox/zmq.rs/wiki/Design

 Hopefully this could make the code clearer and easier to be co-worked on.

 Please advice if something can be done better. :) Thank you!



 BR,
 Fantix
 --
 http://about.me/fantix


 On Wed, Jun 11, 2014 at 4:16 PM, Fantix King fantix.k...@gmail.com wrote:

 Oh yes, I'd love to move into the community organization! Let me update
 the README file (MPLv2 and C4.1 look good) and do some more document work.

 Thank you!


 BR,
 Fantix
 --
 http://about.me/fantix


 On Wed, Jun 11, 2014 at 3:59 PM, Pieter Hintjens p...@imatix.com wrote:

 Hi Fantix,

 Nice stuff. If you want to move this into the ZeroMQ community
 organization, see http://zeromq.org/docs:organization

 -Pieter

 On Wed, Jun 11, 2014 at 6:40 AM, Fantix King fantix.k...@gmail.com
 wrote:
  I think it might be easier to have discussions if there is actual code
  -
  I've started a project zmq.rs with some very basic scratches:
 
  https://github.com/decentfox/zmq.rs
 
  The code now is meant to be discussed and heavily changed
  incrementally,
  hopefully with tests carefully covered. Please feel free to drop by and
  comment if you are interested, it is truly appreciated.
 
 
  BR,
  Fantix
  --
  http://about.me/fantix
 
 
  On Tue, Mar 4, 2014 at 10:55 AM, Fantix King fantix.k...@gmail.com
  wrote:
 
  Sounds really exciting! Please share the project link when there is
  one if
  possible.
 
  BR,
  Fantix
  --
  http://about.me/fantix
 
 
  On Tue, Mar 4, 2014 at 2:26 AM, Charles Remes li...@chuckremes.com
  wrote:
 
  There was a little twitter chat over the weekend regarding an attempt
  at
  writing a ground-up zeromq library using the new systems language
  Rust. If
  you haven’t heard of Rust, it is a new language under development by
  the
  good folks at Mozilla. It’s original designer has said that he has
  learned
  quite a few things from implementing dozens of languages over the
  years that
  he felt he could solve some new problems and create a cleaner
  language. Rust
  is his attempt at such a feat.
 
  It supposedly solves the problem by borrowing the best from many
  popular
  languages.
 
  * OOP of C++ without the large, unwieldy syntax
 
  * performance of C while providing good namespacing, OOP,
  safe
  memory (i.e. no dangling pointers)
 
  * the functional expressiveness of Haskell but not at the
  expense
  of imperative forms
 
  * the massive concurrency of Erlang but with a better syntax
  and
  a more flexible memory model (borrowed pointers, immutable defaults,
  etc)
 
  I recently did a small test project to learn the syntax. The language
  is
  still evolving, so it’s a bit of a moving target. It’s at release 0.9
  with a
  1.0 slated for later this year, but they’ve already slipped on
  delivering a
  1.0 for at least a year so I assume it will slip again.
 
  Anyway, I’d like to volunteer to try and spike a simple example to
  get
  things started. However, I’d like to start a thread here to discuss
  “lessons
  learned” from the existing codebase. We already have a great write-up
  from
  Martin Sustrik (primary author of earlier versions of zeromq) here:
 
  http://250bpm.com/blog:4
 
  http://250bpm.com/blog:8
 
  I’m hoping that others who have read through the source have
  additional
  insights that they’d like to share. For instance, I have seen
  comments that
  zeromq might have more consistent performance it it was wrapped
  around a
  Disruptor (google for that pattern if it’s new to you). People also
  seem to
  really dislike the concept of the context (nanomsg has already
  eliminated
  this… it still exists but is hidden by the library).
 
  Any other insights?
 
  cr
 
  ___
  zeromq-dev mailing list
  zeromq-dev@lists.zeromq.org
  http://lists.zeromq.org/mailman/listinfo/zeromq-dev
 
 
 
 
  ___
  zeromq-dev mailing list
  zeromq-dev@lists.zeromq.org
  http://lists.zeromq.org/mailman/listinfo/zeromq-dev
 
 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev




 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev

___
zeromq-dev mailing 

Re: [zeromq-dev] ROUTER tcp socket stuck in CLOSE_WAIT with large receive queue

2014-06-12 Thread Sash Nagarkar
Thanks Pieter!  I'll try that and see if we encounter it again.  Love
the work you guys are doing with ZMQ.

On Thu, Jun 12, 2014 at 7:53 AM, Pieter Hintjens p...@imatix.com wrote:
 I've seen something similar (I think) with Zyre, where dealer sockets
 connecting with the same identity do weird things. Try setting
 ZMQ_ROUTER_HANDOVER on the router socket, see if that helps (you'll
 need libzmq master).

 On Thu, Jun 12, 2014 at 4:15 AM, Sash Nagarkar s...@dronedeploy.com wrote:
 Hello ZMQ devs,

 We're using PyZMQ 14.3.0 and libzmq 4.0.4 with a ROUTER-DEALER pattern
 for a service we're providing.  Sorry if this is too verbose, and I
 hope this is the right place to ask the question.

 TL;DR: ROUTER socket doesn't receive messages from a DEALER even
 though netstat shows several megabytes in the TCP receive queue
 (nothing in the send queue).  Other connected DEALERs work fine.

 The ROUTER socket is running on a server with ample CPU  memory
 headroom, with several DEALER clients that connect, exchange messages,
 and can abruptly disconnect repeatedly.  We're exclusively using
 multipart messages with the first part always being the ZMQ socket
 identity, which persists across DEALER connect/disconnects.  In other
 words, each DEALER client uses the same socket identity across many
 connects and disconnects.

 Most of the time, things hum along smoothly (several thousand messages
 exchanged, several dozen connect/disconnects).  However, every once in
 a rare while, we see that one of the DEALER clients connects and sends
 messages to the ROUTER that end up never making it to the ROUTER
 process.  The ROUTER process continues to receive messages from other
 DEALER clients.

 Further debugging on the ROUTER server shows one (or more) TCP
 connections from the client DEALER that are in the CLOSE_WAIT state
 with several megabytes of data sitting in the receive queue to the
 ROUTER.  We also see one connection from the client DEALER in the
 ESTABLISHED state with a receive queue that is growing.

 It's clear that the DEALER client died abruptly once, but then
 returned with the same identity and resumed sending messages to the
 ROUTER.  However, none of the subsequent messages are delivered to the
 ROUTER process.  Any ideas on why this would be the case?

 I would have provided a test case, but we aren't able to consistently
 reproduce the issue.  I've copied the output from netstat (with
 obfuscated IPs) below, in case it helps.


 Questions:
 - What would cause the receive queue to fill up like this on a ROUTER
 while it continues to receive messages from other clients?  It's clear
 that the messages are all making it to the ROUTER machine.
 - Is it safe for DEALER sockets to abruptly disconnect and then reuse
 their socket identity?
 - How can we mitigate this situation?  The closest thing I see is
 ZMQ_LINGER, but that applies only to the outgoing queue and not the
 incoming one.
 - Is there anything I could investigate myself to figure out whether
 this is an issue in PyZMQ vs. libzmq?  Where should I start?


 Other potentially relevant info:
 - The ROUTER uses PyZMQ's zmq.Poller() to receive messages from the
 problem socket and some others.  All other nodes in the system
 continue to send and receive messages just fine.
 - The ROUTER's send queues are pretty much empty.
 - We see the same behavior with libzmq 4.0.4 and libzmq 2.2.x, on Ubuntu 
 14.04.


 $ netstat -a
 Active Internet connections (servers and established)
 Proto Recv-Q Send-Q Local Address   Foreign Address State
 tcp0  0 *:12501  *:* LISTEN
 tcp   1816956  0 server-ip.:12501 clientA-ip:42571 CLOSE_WAIT
 tcp   1551036  0 server-ip.:12501 clientA-ip:42858 CLOSE_WAIT
 tcp0  0 server-ip.:12501 clientB-ip:34000 ESTABLISHED
 tcp   5265541  0 server-ip.:12501 clientA-ip:43469 ESTABLISHED


 Please let me if further information would help.  Thank you for
 helping build ZMQ, it's been a huge pleasure to work with so far.

 Cheers,
 Sash
 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev
 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev


[zeromq-dev] Distribution of ZeroMQ contributors

2014-06-12 Thread Pieter Hintjens
Hi all,

I was playing with the GitHub API and worked out the distribution of
committers to ZeroMQ projects (that is, in the zeromq organization).
Here is a pretty picture of the distribution, and a list of the top
contributors, since the start of the project (the current code base
has quite a different breakdown, which I'll get at some point):
http://i.imgur.com/q7GnPWr.png

-Pieter
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev


[zeromq-dev] Lazy pirate alternative implementation

2014-06-12 Thread Dmitri Toubelis
Hi,

I needed to implement a pattern similar to lazy pirate from the guide for my 
project. However, the idea of reconnecting the socket upon timeout looks kind 
of lame to me, so I implemented it differently:

http://pastebin.com/KFSBtK8C

It works okay but is there anything obviously wrong with this approach? Are 
there any potential gotchas?

P.S.: I'm using ZMQ 2.1.x
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev


Re: [zeromq-dev] Lazy pirate alternative implementation

2014-06-12 Thread Pieter Hintjens
You're polling the same socket twice, which isn't necessary. Also, if
you use libzmq/4.x you can use ZMQ_REQ_RELAXED: and that makes Lazy
Pirate obsolete.

On Thu, Jun 12, 2014 at 10:54 PM, Dmitri Toubelis
dmitri.toube...@alkeron.com wrote:
 Hi,

 I needed to implement a pattern similar to lazy pirate from the guide for 
 my project. However, the idea of reconnecting the socket upon timeout looks 
 kind of lame to me, so I implemented it differently:

 http://pastebin.com/KFSBtK8C

 It works okay but is there anything obviously wrong with this approach? Are 
 there any potential gotchas?

 P.S.: I'm using ZMQ 2.1.x
 ___
 zeromq-dev mailing list
 zeromq-dev@lists.zeromq.org
 http://lists.zeromq.org/mailman/listinfo/zeromq-dev
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev


Re: [zeromq-dev] Distribution of ZeroMQ contributors

2014-06-12 Thread Pieter Hintjens
And here's the breakdown of the current codebase (that is, all
projects in the organization, except stabilization forks). The total
LOC is 494,019 and there are 404 contributors, and the distribution is
rather different: http://imgur.com/xG4ucGz

On Thu, Jun 12, 2014 at 10:49 PM, Pieter Hintjens p...@imatix.com wrote:
 Hi all,

 I was playing with the GitHub API and worked out the distribution of
 committers to ZeroMQ projects (that is, in the zeromq organization).
 Here is a pretty picture of the distribution, and a list of the top
 contributors, since the start of the project (the current code base
 has quite a different breakdown, which I'll get at some point):
 http://i.imgur.com/q7GnPWr.png

 -Pieter
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev


Re: [zeromq-dev] Distribution of ZeroMQ contributors

2014-06-12 Thread Pieter Hintjens
Here's the full list of contributors with LOC:

Pieter Hintjens 114235
Dongmin Yu) 71055
Paulmichael Blasucci 40118
John Murphy 19584
Min Ragan-Kelley 19288
Sergei Almazov 18371
Martin Sustrik 17462
Trevor Bernard 16838
evoskuil 15540
John Gozde 13154
miniway 10439
Martin Hurton 9222
tobsen 8301
Ben Gray 7832
somdoron 6784
Martin Lucina 5810
Brian E. Granger 5156
Amir Taaki 4610
Steven McCoy 4535
unknown 4496
Chris Laws 3525
Andreas Schultz 3238
Michel Pelletier 3041
John Watson 2637
Gonzalo Diethelm 2569
tobi-tobsen 2537
Arnaud Loonstra 1961
BDeus 1947
Erik Hugne 1902
Michael Truog 1813
Brian Buchanan 1789
Christian Kamm 1626
Serge Aleynikov 1591
Maarten Ditzel 1589
Mikko Koppanen 1522
Lourens Naudé 1413
Richard Newton 1338
Frank 1277
Erik Algell 1266
Kapp Arnaud 1265
Thomalla, Marti 1234
Yurii Rashkovskii 1210
Pablo Fernandez 1156
Wei Li 1097
Stefan Majer 1066
Johnny Gozde 1023
bebopagogo 993
serge 972
Alois Bělaška 957
Matt Arsenault 949
malosek 901
Mike Gatny 858
Chris Busbey 838
giuseppe 800
Felipe Cruz 750
Sergey KHripchenko 722
Stephen Wolf 712
Ian Barber 689
Chuck Remes 678
Evax Software 562
AJ Lewis 561
Bruno D. Rodrigues 506
danielkr 499
Albert 467
Chris Baxter 467
Eric Merritt 448
Brandon Carpenter 443
sappo 441
Lucas Johnson 409
Tobias Hintze 397
Robert G. Jakabosky 388
Staffan Gimåker 383
Laurent Alebarde 376
Lourens Naudé 371
XMbIPEK 358
sammedic 351
Gran Ville Lintao 322
Will Strang 313
Uli Köhler 309
Olaf Lenz 299
Josh Blum 294
= 293
KIU Shueng Chuan 293
Stefan Radomski 285
Cyril Holweck 281
sjohnr 281
benjamg 276
Michal Laskowski 273
Matt Connolly 256
Gianni Bortolo Bossini 255
Stuart Webster 254
Joshua Foster 253
Hardeep 249
Ivan Pechorin 241
Olaf Mandel 239
Tim M 227
Travis Watkins 224
Robert S. Edmonds 219
MinRK 213
Robert Horvick 213
Richard Smith 205
Guido Goldstein 197
Doron Somech 194
Fabien Ninoles 188
Mark Barbisan 182
Andre Caron 179
Goswin von Brederlow 179
rimmington 178
Mikko 172
Sappo 172
Justin Karneges 169
Stoian Ivanov 167
Brian Knox 161
skaller 158
Zachary Kessin 157
Rei Roldan 152
dhammika 144
nikita kozlov 144
Sakari 144
Mika Fischer 142
Fred Eisele 139
John Muehlhausen 136
Giuseppe Santoro 128
m 125
Justin Riley 120
Jon Dyte 119
dinka 118
Dan Aloni 117
somdo...@gmail.com 112
Shannen Saez 111
Ilya Kulakov 109
Krzysztof Rapacki 108
FELD Boris 104
Fernando Perez 102
Arthur O'Dwyer 98
Erick Tryzelaar 88
Adrian Muraru 86
Dave Meehan 86
Stephen Riesenberg 86
David Budworth 84
Jonathan Wood 82
bo...@boressoft.ru 80
Daniel Lundin 80
William Roberts 78
David Mitchell 77
Scott Gibbons 77
spez 77
Cliff Evans 76
Stefan Kaes 76
Joe Eli McIlvain 71
Victor Perron 71
Heinrich Hartmann 70
Gyula Laszlo 69
Patrick Trantham 67
kwo 64
Martin Vala 63
claws 62
shripchenko 59
Kobolog 57
psl-felipefarinon 57
Jeremy Rossi 55
HughPowell 54
Chris de Vries 51
Grant Birchmeier 50
John Gallagher 49
Chris Perks 47
François Beausoleil 47
Shane Hubred 46
Dmitry Odzerikho 45
imleon 45
Arkadiusz Orzechowski 44
Timothee TTimo Besset 44
root 43
Botond Ballo 40
Sebastien Pierre 39
Apostolis Xekoukoulotakis 38
Garrett Smith 38
Godefroid Chapelle 38
Jonathan Booth 38
Shawn J. Goff 38
Felipe cruz 36
TJ Holowaychuk 36
baboune 34
Dhammika Pathirana 34
Kevin Smith 34
Momo the Monster 34
hugo shi 33
Lucas Russo 33
rpedde 33
Andrew Gwozdziewycz 28
Eren Güven 28
Not Committed Yet 28
Trevor R.H. Clarke 28
Mark Riley 27
Michael Bridgen 27
Rob Hubbard 27
Alois Belaska 26
cossin 26
Julian Taylor 26
Lindley French 26
Paul Betts 26
Stephen Hemminger 26
xantares 26
Sebastien Rombauts 25
Alexey Ermakov 24
crocket 24
Javier Lopez 24
Ransom Richardson 24
h4cc 23
Maximilian Karasz 22
Akira Kitada 21
Torsten Landschoff 21
FT/RD-TOK) 20
Michael Granger 20
Oleksandr Kozachuk 20
Attila Mark 19
Phillip Mienk 19
Samuel Martin 19
Colin Steifel 18
cresnick 18
Ken Steele 18
Patrick Noffke 18
Sébastien Rombauts 18
Stefan Friesel 18
Ryan Kelly 17
Scott Sadler 17
Steve-o 17
Kaustubh Rawoorkar 16
Kevin Locke 16
Markus Rothe 16
Your Name 16
Alexander Pyhalov 15
Chia-liang Kao 15
Ricardo Catalinas Jiménez 15
Bjorn Topel 14
Cosmo Harrigan 14
Jan Kryl 14
Kristoffer Carlson 14
Nathan Marz 14
Philip Kovacs 14
Thijs Terlouw 14
Brett Cameron 13
Mark Marsella 13
Artem Martynovich 12
Bob Beaty 12
Christophe Juniet 12
Curtis 12
Franco Fichtner 12
Jose Pedro Oliveira 12
Matthew Aburn 12
Volodymyr Korniichuk 12
Devin Smith 11
Jonny Dee 11
Marc Abramowitz 11
Mikael Helbo Kjaer 11
Aja Walker 10
ganesh.vr 10
Ivo Danihelka 10
Justin Bronder 10
Martin Pales 10
Matthew Brush 10
Nadav Samet 10
Phus Lu 10
Jason Chown 9
Michael Haberler 9
plemanach 9
Robert Gallas 9
TobiasSimon 9
Brandon Ehle 8
Bruno Bigras 8
Neale Ferguson 8
Pavel Pimenov 8
Sam Smith 8
Scott Dial 8
Silent Johnny 8
xinchuan 8
Charles McGarvey 7
Emmanuel Taurel 7
Gregor Fabritius 7
Jens Rantil 7
Kevin Sapper 7
Loïc Hoguin 7
Paul Colomiets 7
Alexey Melnichuk 6
Drew Crawford 6
Evan Burkitt 6
Greg Banks 6
Greg Ward 6
John Swinbank 6
Joshua Leahy 6
Matthew Metnetsky 6
Ondrej Certik 6
Zbigniew 

Re: [zeromq-dev] Distribution of ZeroMQ contributors

2014-06-12 Thread Trevor Bernard
That's super cool -- nice graphics
On 12 Jun 2014 19:28, Pieter Hintjens p...@imatix.com wrote:

 Here's the full list of contributors with LOC:

 Pieter Hintjens 114235
 Dongmin Yu) 71055
 Paulmichael Blasucci 40118
 John Murphy 19584
 Min Ragan-Kelley 19288
 Sergei Almazov 18371
 Martin Sustrik 17462
 Trevor Bernard 16838
 evoskuil 15540
 John Gozde 13154
 miniway 10439
 Martin Hurton 9222
 tobsen 8301
 Ben Gray 7832
 somdoron 6784
 Martin Lucina 5810
 Brian E. Granger 5156
 Amir Taaki 4610
 Steven McCoy 4535
 unknown 4496
 Chris Laws 3525
 Andreas Schultz 3238
 Michel Pelletier 3041
 John Watson 2637
 Gonzalo Diethelm 2569
 tobi-tobsen 2537
 Arnaud Loonstra 1961
 BDeus 1947
 Erik Hugne 1902
 Michael Truog 1813
 Brian Buchanan 1789
 Christian Kamm 1626
 Serge Aleynikov 1591
 Maarten Ditzel 1589
 Mikko Koppanen 1522
 Lourens Naudé 1413
 Richard Newton 1338
 Frank 1277
 Erik Algell 1266
 Kapp Arnaud 1265
 Thomalla, Marti 1234
 Yurii Rashkovskii 1210
 Pablo Fernandez 1156
 Wei Li 1097
 Stefan Majer 1066
 Johnny Gozde 1023
 bebopagogo 993
 serge 972
 Alois Bělaška 957
 Matt Arsenault 949
 malosek 901
 Mike Gatny 858
 Chris Busbey 838
 giuseppe 800
 Felipe Cruz 750
 Sergey KHripchenko 722
 Stephen Wolf 712
 Ian Barber 689
 Chuck Remes 678
 Evax Software 562
 AJ Lewis 561
 Bruno D. Rodrigues 506
 danielkr 499
 Albert 467
 Chris Baxter 467
 Eric Merritt 448
 Brandon Carpenter 443
 sappo 441
 Lucas Johnson 409
 Tobias Hintze 397
 Robert G. Jakabosky 388
 Staffan Gimåker 383
 Laurent Alebarde 376
 Lourens Naudé 371
 XMbIPEK 358
 sammedic 351
 Gran Ville Lintao 322
 Will Strang 313
 Uli Köhler 309
 Olaf Lenz 299
 Josh Blum 294
 = 293
 KIU Shueng Chuan 293
 Stefan Radomski 285
 Cyril Holweck 281
 sjohnr 281
 benjamg 276
 Michal Laskowski 273
 Matt Connolly 256
 Gianni Bortolo Bossini 255
 Stuart Webster 254
 Joshua Foster 253
 Hardeep 249
 Ivan Pechorin 241
 Olaf Mandel 239
 Tim M 227
 Travis Watkins 224
 Robert S. Edmonds 219
 MinRK 213
 Robert Horvick 213
 Richard Smith 205
 Guido Goldstein 197
 Doron Somech 194
 Fabien Ninoles 188
 Mark Barbisan 182
 Andre Caron 179
 Goswin von Brederlow 179
 rimmington 178
 Mikko 172
 Sappo 172
 Justin Karneges 169
 Stoian Ivanov 167
 Brian Knox 161
 skaller 158
 Zachary Kessin 157
 Rei Roldan 152
 dhammika 144
 nikita kozlov 144
 Sakari 144
 Mika Fischer 142
 Fred Eisele 139
 John Muehlhausen 136
 Giuseppe Santoro 128
 m 125
 Justin Riley 120
 Jon Dyte 119
 dinka 118
 Dan Aloni 117
 somdo...@gmail.com 112
 Shannen Saez 111
 Ilya Kulakov 109
 Krzysztof Rapacki 108
 FELD Boris 104
 Fernando Perez 102
 Arthur O'Dwyer 98
 Erick Tryzelaar 88
 Adrian Muraru 86
 Dave Meehan 86
 Stephen Riesenberg 86
 David Budworth 84
 Jonathan Wood 82
 bo...@boressoft.ru 80
 Daniel Lundin 80
 William Roberts 78
 David Mitchell 77
 Scott Gibbons 77
 spez 77
 Cliff Evans 76
 Stefan Kaes 76
 Joe Eli McIlvain 71
 Victor Perron 71
 Heinrich Hartmann 70
 Gyula Laszlo 69
 Patrick Trantham 67
 kwo 64
 Martin Vala 63
 claws 62
 shripchenko 59
 Kobolog 57
 psl-felipefarinon 57
 Jeremy Rossi 55
 HughPowell 54
 Chris de Vries 51
 Grant Birchmeier 50
 John Gallagher 49
 Chris Perks 47
 François Beausoleil 47
 Shane Hubred 46
 Dmitry Odzerikho 45
 imleon 45
 Arkadiusz Orzechowski 44
 Timothee TTimo Besset 44
 root 43
 Botond Ballo 40
 Sebastien Pierre 39
 Apostolis Xekoukoulotakis 38
 Garrett Smith 38
 Godefroid Chapelle 38
 Jonathan Booth 38
 Shawn J. Goff 38
 Felipe cruz 36
 TJ Holowaychuk 36
 baboune 34
 Dhammika Pathirana 34
 Kevin Smith 34
 Momo the Monster 34
 hugo shi 33
 Lucas Russo 33
 rpedde 33
 Andrew Gwozdziewycz 28
 Eren Güven 28
 Not Committed Yet 28
 Trevor R.H. Clarke 28
 Mark Riley 27
 Michael Bridgen 27
 Rob Hubbard 27
 Alois Belaska 26
 cossin 26
 Julian Taylor 26
 Lindley French 26
 Paul Betts 26
 Stephen Hemminger 26
 xantares 26
 Sebastien Rombauts 25
 Alexey Ermakov 24
 crocket 24
 Javier Lopez 24
 Ransom Richardson 24
 h4cc 23
 Maximilian Karasz 22
 Akira Kitada 21
 Torsten Landschoff 21
 FT/RD-TOK) 20
 Michael Granger 20
 Oleksandr Kozachuk 20
 Attila Mark 19
 Phillip Mienk 19
 Samuel Martin 19
 Colin Steifel 18
 cresnick 18
 Ken Steele 18
 Patrick Noffke 18
 Sébastien Rombauts 18
 Stefan Friesel 18
 Ryan Kelly 17
 Scott Sadler 17
 Steve-o 17
 Kaustubh Rawoorkar 16
 Kevin Locke 16
 Markus Rothe 16
 Your Name 16
 Alexander Pyhalov 15
 Chia-liang Kao 15
 Ricardo Catalinas Jiménez 15
 Bjorn Topel 14
 Cosmo Harrigan 14
 Jan Kryl 14
 Kristoffer Carlson 14
 Nathan Marz 14
 Philip Kovacs 14
 Thijs Terlouw 14
 Brett Cameron 13
 Mark Marsella 13
 Artem Martynovich 12
 Bob Beaty 12
 Christophe Juniet 12
 Curtis 12
 Franco Fichtner 12
 Jose Pedro Oliveira 12
 Matthew Aburn 12
 Volodymyr Korniichuk 12
 Devin Smith 11
 Jonny Dee 11
 Marc Abramowitz 11
 Mikael Helbo Kjaer 11
 Aja Walker 10
 ganesh.vr 10
 Ivo Danihelka 10
 Justin Bronder 10
 Martin Pales 10
 Matthew Brush 10
 Nadav Samet 10
 Phus Lu 10
 Jason Chown 9
 Michael Haberler 9
 plemanach 9
 Robert Gallas 9
 TobiasSimon 9
 Brandon Ehle 8
 

[zeromq-dev] zsock test hangs in pyczmq

2014-06-12 Thread Michel Pelletier
I'm working on adding zsock and zactor support to pyczmq, and I have most
of the work done, but my test for zsock is hanging.  The test runs and
works, zsock_destroy is called correctly (not that it would matter the
default linger is 0) but the process never exits.  gdb says it hanging on
poll in zsys's s_terminate_process:

#0  0x778e2f7d in poll () at ../sysdeps/unix/syscall-template.S:81
#1  0x75361caa in poll (__timeout=-1, __nfds=1,
__fds=0x7fffdd10) at /usr/include/x86_64-linux-gnu/bits/poll2.h:46
#2  zmq::signaler_t::wait (this=this@entry=0x9e7dc8,
timeout_=timeout_@entry=-1)
at signaler.cpp:158
#3  0x75350177 in zmq::mailbox_t::recv (this=this@entry=0x9e7d68,
cmd_=cmd_@entry=0x7fffdd70, timeout_=timeout_@entry=-1) at
mailbox.cpp:72
#4  0x7533fc0c in zmq::ctx_t::terminate (this=0x9e7cd0) at
ctx.cpp:141
#5  0x75378921 in zmq_ctx_term (ctx_=optimized out) at zmq.cpp:157
#6  0x75816891 in s_terminate_process () at zsys.c:273
#7  0x77831071 in __run_exit_handlers (status=0,
listp=0x77bb66a8 __exit_funcs,
run_list_atexit=run_list_atexit@entry=true)
at exit.c:77
#8  0x778310f5 in __GI_exit (status=optimized out) at exit.c:99
#9  0x77816dec in __libc_start_main (main=0x46a2a1 main, argc=2,
ubp_av=0x7fffdf18, init=optimized out, fini=optimized out,
rtld_fini=optimized out, stack_end=0x7fffdf08) at libc-start.c:294
#10 0x005735fe in _start ()

A little more digging looks like libzmq is waiting for a message from a
'reaper' thread that never arrives, context termination waits for this
message with a timeout of -1 so it never exits.

Anyone have any thoughts on this?  I don't know much about the
reaper/shutdown side of things.

-Michel
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev


Re: [zeromq-dev] zeromq, rewritten in Rust

2014-06-12 Thread Fantix King
On Thu, Jun 12, 2014 at 11:05 PM, Pieter Hintjens p...@imatix.com wrote:

 Hi Fantix, you are now a member of the ZeroMQ owners team, with all
 the power and responsibility. :-) Welcome.


Thank you, sir! ;)



 The next step is to rename/move the repository to zeromq and then fork
 it back to your personal account.


Ack, I'm on it.



 It's really nice to have (I think) our first Chinese-originated project.


What an honor! I'll try my best to work it out. :) Thank you!


BR,
Fantix
--
http://about.me/fantix




 -Pieter

 On Thu, Jun 12, 2014 at 9:25 AM, Fantix King fantix.k...@gmail.com
 wrote:
  Hi Pieter,
 
  I've updated the README file according to the requests and
 recommendation in
  docs:organization, and also draw some diagrams for a simple design
 document
  here:
 
  https://github.com/decentfox/zmq.rs/wiki/Design
 
  Hopefully this could make the code clearer and easier to be co-worked on.
 
  Please advice if something can be done better. :) Thank you!
 
 
 
  BR,
  Fantix
  --
  http://about.me/fantix
 
 
  On Wed, Jun 11, 2014 at 4:16 PM, Fantix King fantix.k...@gmail.com
 wrote:
 
  Oh yes, I'd love to move into the community organization! Let me update
  the README file (MPLv2 and C4.1 look good) and do some more document
 work.
 
  Thank you!
 
 
  BR,
  Fantix
  --
  http://about.me/fantix
 
 
  On Wed, Jun 11, 2014 at 3:59 PM, Pieter Hintjens p...@imatix.com wrote:
 
  Hi Fantix,
 
  Nice stuff. If you want to move this into the ZeroMQ community
  organization, see http://zeromq.org/docs:organization
 
  -Pieter
 
  On Wed, Jun 11, 2014 at 6:40 AM, Fantix King fantix.k...@gmail.com
  wrote:
   I think it might be easier to have discussions if there is actual
 code
   -
   I've started a project zmq.rs with some very basic scratches:
  
   https://github.com/decentfox/zmq.rs
  
   The code now is meant to be discussed and heavily changed
   incrementally,
   hopefully with tests carefully covered. Please feel free to drop by
 and
   comment if you are interested, it is truly appreciated.
  
  
   BR,
   Fantix
   --
   http://about.me/fantix
  
  
   On Tue, Mar 4, 2014 at 10:55 AM, Fantix King fantix.k...@gmail.com
   wrote:
  
   Sounds really exciting! Please share the project link when there is
   one if
   possible.
  
   BR,
   Fantix
   --
   http://about.me/fantix
  
  
   On Tue, Mar 4, 2014 at 2:26 AM, Charles Remes li...@chuckremes.com
 
   wrote:
  
   There was a little twitter chat over the weekend regarding an
 attempt
   at
   writing a ground-up zeromq library using the new systems language
   Rust. If
   you haven't heard of Rust, it is a new language under development
 by
   the
   good folks at Mozilla. It's original designer has said that he has
   learned
   quite a few things from implementing dozens of languages over the
   years that
   he felt he could solve some new problems and create a cleaner
   language. Rust
   is his attempt at such a feat.
  
   It supposedly solves the problem by borrowing the best from many
   popular
   languages.
  
   * OOP of C++ without the large, unwieldy syntax
  
   * performance of C while providing good namespacing, OOP,
   safe
   memory (i.e. no dangling pointers)
  
   * the functional expressiveness of Haskell but not at the
   expense
   of imperative forms
  
   * the massive concurrency of Erlang but with a better
 syntax
   and
   a more flexible memory model (borrowed pointers, immutable
 defaults,
   etc)
  
   I recently did a small test project to learn the syntax. The
 language
   is
   still evolving, so it's a bit of a moving target. It's at release
 0.9
   with a
   1.0 slated for later this year, but they've already slipped on
   delivering a
   1.0 for at least a year so I assume it will slip again.
  
   Anyway, I'd like to volunteer to try and spike a simple example to
   get
   things started. However, I'd like to start a thread here to discuss
   lessons
   learned from the existing codebase. We already have a great
 write-up
   from
   Martin Sustrik (primary author of earlier versions of zeromq) here:
  
   http://250bpm.com/blog:4
  
   http://250bpm.com/blog:8
  
   I'm hoping that others who have read through the source have
   additional
   insights that they'd like to share. For instance, I have seen
   comments that
   zeromq might have more consistent performance it it was wrapped
   around a
   Disruptor (google for that pattern if it's new to you). People also
   seem to
   really dislike the concept of the context (nanomsg has already
   eliminated
   this... it still exists but is hidden by the library).
  
   Any other insights?
  
   cr
  
   ___
   zeromq-dev mailing list
   zeromq-dev@lists.zeromq.org
   http://lists.zeromq.org/mailman/listinfo/zeromq-dev
  
  
  
  
   ___
   zeromq-dev mailing list
   zeromq-dev@lists.zeromq.org