Re: [zeromq-dev] Core componets / files of the library
Hi, sust...@250bpm.com said: However, AFAIK, there's no fully detailed description of the architecture available. You'll have to figure out the minutiae from the code itself and comments in it. I'm fairly sure there's a class diagram spanning several A3 pages floating around somewere in south-east Asia :-) -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Problems cross compiling for ARM
Hi Michael, mwpowell...@gmail.com said: Hello, I want to build ZMQ for ARM but have problems. If I just build with the plain old i386 arch, no problems. I do get a .a library I can statically link against, BTW. Good. Now if I ./configure ... CC=arm-none-linux-gnueabi-gcc --host=arm It builds, but the output files are still targeting 80386. This isn't right? How do we build for ARM? You want: $ ./configure --host=arm-none-linux-gnueabi Cheers, -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Problems cross compiling for ARM
Unfortunately not, see above. E.g. I have arm-linux-gnueabi here since I use the EmDebian toolchain. FYI, I couldn't get zeromq-3.2.2 to compile using my toolchain (Debian 4.4.5-8) due to a bug in decoder.hpp. Fixed the bug myself and was going to submit a patch but this appears to have been fixed since in commit 6d4e2ce to zeromq3-x. -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] RE : AF_BUS for efficient 1:M IPC comms
sust...@250bpm.com said: On 23/01/13 09:44, DEBROUX Lionel wrote: http://lwn.net/Articles/504722/ Sadly, while it has some clear use cases, AF_BUS is not part of the mainline kernel, it remains an out-of-tree patch... It was added to a patched third-party 3.4.x kernel yesterday, as part of the LTSI (used only by a tiny minority): https://lwn.net/Articles/533534/ . That might speed up the inclusion of AF_BUS to mainline, only time will tell. IIRC, this is part of the DBus-in-kernel saga. What's important is that it shows that there a real need for messaging on kernel level. From my perspective, it would be nice to have a ZeroMQ-like one in the Linux kernel. There's an initial implementation here: https://github.com/250bpm/linux/tree/sp-linux However, more work would be needed to make it fully functional and eventually get it into mainline kernel. Also, more recently, this: http://lwn.net/Articles/537017/ You might want to get in touch with greg k-h about his plans. At the very least, maybe he can be inspired by the ZeroMQ/XS/nanomsg APIs. -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] (Fwd) [crossroads-dev] [PATCH] Correct handling of connect() errors in tcp, ipc_connecter_t
Hi,
FYI, the patch below is for an old bug, also affects ZeroMQ 3.1.0 and
2.1.11 (based on the last released versions I have local copies of, I can't
seem to access GH from here).
-mato
- Forwarded message from Martin Lucina mar...@lucina.net -
Date: Thu, 3 May 2012 13:08:49 +0200
From: Martin Lucina mar...@lucina.net
To: crossroads-...@groups.crossroads.io
Cc: Martin Lucina mar...@lucina.net
Subject: [crossroads-dev] [PATCH] Correct handling of connect() errors in tcp,
ipc_connecter_t
EAGAIN was being used as a translation value for EINPROGRESS, thus
shadowing a real EAGAIN return value from the OS. This caused later
assertions of Invalid argument in stream_engine.cpp when it attempted to
use a socket which was not connected.
I also add EINTR to mean EINPROGRESS, as per the POSIX and FreeBSD
documentation which specifies that a connect() call interrupted due to a
signal will complete asynchronously.
Signed-off-by: Martin Lucina mar...@lucina.net
---
src/ipc_connecter.cpp |9 +
src/tcp_connecter.cpp | 11 ++-
2 files changed, 11 insertions(+), 9 deletions(-)
diff --git a/src/ipc_connecter.cpp b/src/ipc_connecter.cpp
index af3665c..0722532 100644
--- a/src/ipc_connecter.cpp
+++ b/src/ipc_connecter.cpp
@@ -133,7 +133,7 @@ void xs::ipc_connecter_t::start_connecting ()
}
// Connection establishment may be delayed. Poll for its completion.
-else if (rc == -1 errno == EAGAIN) {
+else if (rc == -1 errno == EINPROGRESS) {
xs_assert (!handle);
handle = add_fd (s);
set_pollout (handle);
@@ -196,9 +196,10 @@ int xs::ipc_connecter_t::open ()
if (rc == 0)
return 0;
-// Asynchronous connect was launched.
-if (rc == -1 errno == EINPROGRESS) {
-errno = EAGAIN;
+// Translate other error codes indicating asynchronous connect has been
+// launched to a uniform EINPROGRESS.
+if (rc == -1 errno == EINTR) {
+errno = EINPROGRESS;
return -1;
}
diff --git a/src/tcp_connecter.cpp b/src/tcp_connecter.cpp
index de105bb..5ae01e7 100644
--- a/src/tcp_connecter.cpp
+++ b/src/tcp_connecter.cpp
@@ -140,7 +140,7 @@ void xs::tcp_connecter_t::start_connecting ()
}
// Connection establishment may be delayed. Poll for its completion.
-else if (rc == -1 errno == EAGAIN) {
+else if (rc == -1 errno == EINPROGRESS) {
xs_assert (!handle);
handle = add_fd (s);
set_pollout (handle);
@@ -236,17 +236,18 @@ int xs::tcp_connecter_t::open ()
if (rc == 0)
return 0;
-// Asynchronous connect was launched.
+// Translate other error codes indicating asynchronous connect has been
+// launched to a uniform EINPROGRESS.
#ifdef XS_HAVE_WINDOWS
if (rc == SOCKET_ERROR (WSAGetLastError () == WSAEINPROGRESS ||
WSAGetLastError () == WSAEWOULDBLOCK)) {
-errno = EAGAIN;
+errno = EINPROGRESS;
return -1;
}
wsa_error_to_errno ();
#else
-if (rc == -1 errno == EINPROGRESS) {
-errno = EAGAIN;
+if (rc == -1 errno == EINTR) {
+errno = EINPROGRESS;
return -1;
}
#endif
--
1.7.10
-
Full text of this topic in Crossroads Development:
http://groups.crossroads.io/r/topic/2QCfCK2pongMdlQQIzGubl
To leave Crossroads Development, email
mailto:crossroads-...@groups.crossroads.io?Subject=unsubscribe
Start your own free groups and site with
OnlineGroups.Net http://onlinegroups.net
Host your own online groups site with
GroupServer http://groupserver.org
- End forwarded message -
--
Martin Lucina
http://lucina.net/ (interwebs/blogs/rants/consulting)
mar...@lucina.net (smtp/xmpp/jabber/gtalk)
@matolucina(twitter)
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Binding to TCP port 0
Hi Ian!
Thanks for the patch!
You asked me to review it, so here goes...
This is based on https://github.com/zeromq/libzmq/pull/238.diff as of
today; there doesn't seem to be a better way to uniquely identify a Github
pull request's *content* since it can change all the time :-(
diff --git a/src/ipc_listener.cpp b/src/ipc_listener.cpp
index 07a7dff..6797344 100644
--- a/src/ipc_listener.cpp
+++ b/src/ipc_listener.cpp
@@ -95,8 +95,31 @@ void zmq::ipc_listener_t::in_event ()
send_attach (session, engine, false);
}
+int zmq::ipc_listener_t::get_address (unsigned char *addr, size_t *len)
Coding style: Please use addr_ and len_ for function parameter names.
+{
+struct sockaddr_un sun;
+int rc;
+
+// Get the details of the IPC socket
+socklen_t sl = sizeof(sockaddr_un);
+rc = getsockname (s, (sockaddr *)sun, sl);
+if (rc != 0) {
+return rc;
+}
+// Store the address for retrieval by users using wildcards
+*len = sprintf((char *)addr, ipc://%s, sun.sun_path);
+
+return 0;
+}
+
The use of sprintf() is a security hole if the user allocated not enough
space at *addr. Please use snprintf() to ensure a maximum of len bytes
(including the string terminator) are written to *addr.
int zmq::ipc_listener_t::set_address (const char *addr_)
{
+
+// Allow wildcard file
+if(*addr_ == '*') {
+addr_ = tempnam(NULL, NULL);
+}
+
// Get rid of the file associated with the UNIX domain socket that
// may have been left behind by the previous run of the application.
::unlink (addr_);
@@ -124,7 +147,7 @@ int zmq::ipc_listener_t::set_address (const char *addr_)
rc = listen (s, options.backlog);
if (rc != 0)
return -1;
-
+
return 0;
}
The use of tempnam() is a potential security hole, although we don't make
any guarantees about untrusted local users. I'm not sure if there is an
equivalent of mkstemp() for UNIX domain sockets, can't remember off the top
of my head.
diff --git a/src/ipc_listener.hpp b/src/ipc_listener.hpp
index e1f4817..57e04ef 100644
--- a/src/ipc_listener.hpp
+++ b/src/ipc_listener.hpp
@@ -48,6 +48,9 @@
// Set address to listen on.
int set_address (const char *addr_);
+
+// Get the bound address for use with wildcards
+int get_address(unsigned char *addr, size_t *len);
Coding style: Use addr_ and len_.
private:
diff --git a/src/options.cpp b/src/options.cpp
index 4db1a6c..c8790a8 100644
--- a/src/options.cpp
+++ b/src/options.cpp
@@ -30,6 +30,7 @@
rcvhwm (1000),
affinity (0),
identity_size (0),
+last_endpoint_size(0),
rate (100),
recovery_ivl (1),
multicast_hops (1),
@@ -213,7 +214,6 @@ int zmq::options_t::setsockopt (int option_, const void
*optval_,
ipv4only = val;
return 0;
}
-
}
errno = EINVAL;
@@ -385,7 +385,15 @@ int zmq::options_t::getsockopt (int option_, void
*optval_, size_t *optvallen_)
*((int*) optval_) = ipv4only;
*optvallen_ = sizeof (int);
return 0;
-
+
+case ZMQ_LAST_ENDPOINT:
+if (*optvallen_ last_endpoint_size) {
+errno = EINVAL;
+return -1;
+}
+memcpy (optval_, last_endpoint, last_endpoint_size);
+*optvallen_ = last_endpoint_size;
+return 0;
}
Security hole: You want to copy last_endpoint_size bytes, or optvallen_
bytes, whichever is lesser, and return the actual # of bytes copied in
*optvallen_.
errno = EINVAL;
diff --git a/src/options.hpp b/src/options.hpp
index bfc9dc7..7feea95 100644
--- a/src/options.hpp
+++ b/src/options.hpp
@@ -46,6 +46,10 @@
// Socket identity
unsigned char identity_size;
unsigned char identity [256];
+
+// Last socket endpoint URI
+unsigned char last_endpoint [256];
+size_t last_endpoint_size;
I guess you wanted to use the ZMQ_ENDPOINT_MAX define here rather than
hardcoding 256?
// Maximum tranfer rate [kb/s]. Default 100kb/s.
int rate;
diff --git a/src/socket_base.cpp b/src/socket_base.cpp
index 3761b46..74c807f 100644
--- a/src/socket_base.cpp
+++ b/src/socket_base.cpp
@@ -161,6 +161,7 @@ int zmq::socket_base_t::parse_uri (const char *uri_,
}
protocol_ = uri.substr (0, pos);
address_ = uri.substr (pos + 3);
+
if (protocol_.empty () || address_.empty ()) {
errno = EINVAL;
return -1;
@@ -340,6 +341,8 @@ int zmq::socket_base_t::bind (const char *addr_)
delete listener;
return -1;
}
+
+rc = listener-get_address (options.last_endpoint,
(options.last_endpoint_size));
launch_child (listener);
return 0;
}
@@ -354,6 +357,7 @@ int
Re: [zeromq-dev] atomic ops
John, skal...@users.sourceforge.net said: I am looking at the atomic ops stuff, and I find this a bit suspicious, and also suboptimal: #elif (defined __i386__ || defined __x86_64__) defined __GNUC__ #define ZMQ_ATOMIC_COUNTER_X86 Sure, it's suboptimal for a build on non-x86 (x86_64) and not using GCC (or compilers pretending to be GCC, such as at least Intel icc). #elif defined ZMQ_ATOMIC_COUNTER_X86 integer_t oldval = -decrement; volatile integer_t *val = value; __asm__ volatile (lock; xaddl %0,%1 : =r (oldval), =m (*val) : 0 (oldval), m (*val) : cc, memory); return oldval != decrement; First, I surely wouldn't trust inline assembler, and I'm suspicious of lock; xadd. This code also won't be selected except on x86 or x86_64. What exactly do you find suspicious about lock; xadd? Standard way to implement atomics on x86. Wouldn't it be better to use this function: type __sync_fetch_and_add (type *ptr, type value, ...) Yes. which is guaranteed to work on all processors? I'd trust gcc to do the right thing a lot more than inline assembler. That function is always available if __GNUC__ is set, on recent enough implementations of gcc. The spec comes from gcc 4.1.2. [Presumably config can double check..] When I last reviewed that code I considered using the gcc intrinsics, but the are only available since GCC 4.something and there are still many people using older compilers. It takes several *years* for a new compiler feature to trickle down to users. So it'd have to be optional, feel free to implement it but you'll need to get the autoconf-fu working to not break performance for users with older gcc. It's a shame unixy OSes aren't doing the right thing and collaborating to provide a common atomic.h interface to userspace like NetBSD and Solaris do (both of which the zmq atomics code supports). -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Thread Safe sockets
skal...@users.sourceforge.net said: On 07/02/2012, at 7:37 AM, Nadav Samet wrote: I like the concept of having thread-safe sockets, but it seems that this implementation does not really deliver what a user would expect from a thread-safe socket. For instance, if two threads try to receive simultaneously from the same socket, each of them might obtain different frames that belong to one multipart message. Yes indeed, that is a design fault with multi-part messages being delivered over time instead of space. My idea to fix this is to provide a way to fetch or deliver them all in an array. zmq_sendmmsg(), zmq_recvmmsg(), as based on the Linux calls of a similar name, and similar calls I forget implemented in the Windows Vista kernel? http://lwn.net/Articles/441169/ (sendmmsg) http://lwn.net/Articles/334532/ (recvmmsg) The only thing is, even if you add those, you can't take away the old calls since that would break existing code ... so AFAICS we're stuck with it. -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Thread Safe sockets
On Mon, 6 Feb 2012 18:57:35 -0500 Joshua Foster jhaw...@gmail.com wrote: I would like to put forth the motion that the sockets should not be thread safe. The multipart messages do not facilitate the ability for thread safety without some major complexity. This would involve the complexity of changing the API that breaks backwards compatibility or adding locks which would slow the API down. My personal experience is that when people were trying to use a socket over multiple threads, it was wrong anyways. The asserts in libzmq catch the problem very quickly during development so we never see it in production which I think is better than having demons in the system and not understanding why. +1 The second is to use an inproc socket per thread and then create a device that transfers this data to the external socket. This has the least amount of problems with concurrency. If the community really wanted to create a thread safe socket, I would recommend this approach. We would need to create inproc sockets and store them as ThreadLocal. This would then properly abstract out each socket for each thread. +1, also from experience developing applications using ZeroMQ. The each thread has its own inproc socket model is really nice and we should emphasise it more. It leads to developing your applications around messaging and concurrency, and IMO is a much more elegant way of dealing with concurrency than the old school locks and shared state model. Can you elaborate on why such a socket would have to be stored as ThreadLocal? Given that in ZeroMQ a single listening (bound/connected) socket can have many other sockets connected to it, if threads A, B, C want to talk to thread D then A, B, C each have their own socket and all connect to D's inproc:// endpoint. -mato -- Martin Lucina mar...@lucina.net ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Thread Safe sockets
On Mon, 06 Feb 2012 19:37:28 -0500 Joshua Foster jhaw...@gmail.com wrote: I'm not sure if other languages have the same mechanism, but Java has a ThreadLocal object. It allows the developer to create and store objects local for each thread. In the ZMQ case, it would be an inproc socket for each thread so that we can hide the complexity of thread safety under the covers. If we do this, I would rather see completely new sockets like ZMQ_PUB_CONCURRENT instead of modifying the existing ones. I don't follow. Making an object thread local as I understand it is making a single identifier/symbol/name X refer to multiple instances of something. How does that need a new socket type? Maybe Java ThreadLocal means something else? I don't do much Java... -mato -- Martin Lucina mar...@lucina.net ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Thread Safe sockets
On Mon, 6 Feb 2012 16:39:51 -0800 Nadav Samet thesa...@gmail.com wrote: Can you elaborate on why such a socket would have to be stored as ThreadLocal? Given that in ZeroMQ a single listening (bound/connected) socket can have many other sockets connected to it, if threads A, B, C want to talk to thread D then A, B, C each have their own socket and all connect to D's inproc:// endpoint. One use case is when you write a library on top of zeromq that runs user code (maybe in form of callbacks) on threads A, B and C. Those callbacks needs a way to obtain a socket connected to D in order to reply. It could be passed to them as an argument, and they can pass that socket again to each function that they are calling, making the API more explicit than what you'd like. So you want to make it a bit easier on your users by having your library figuring out the right socket to use from the identity of the currently running thread. 1) IMHO callbacks are evil and in 90% of cases should be eradicated :-) Of course if you've got a huge legacy code base then you're stuck with 'em, so it's a valid use case. 2) The way I did this in a Python project was have each thread generate a well-known inproc://enpoint based on id(self). I guess you could make a general solution by having a TLS own_socket, but that only helps for referring to the socket from inside the thread that owns it. Anyone external (or any other thread) trying to connect to it still needs to somehow determine the correct inproc:// endpoint, either by using some well-known generation scheme, or interrogating the target thread. -- Martin Lucina mar...@lucina.net ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Thread Safe sockets
On Tue, 7 Feb 2012 12:11:10 +1100 john skaller skal...@users.sourceforge.net wrote: On 07/02/2012, at 11:23 AM, Martin Lucina wrote: The each thread has its own inproc socket model is really nice and we should emphasise it more. It leads to developing your applications around messaging and concurrency, and IMO is a much more elegant way of dealing with concurrency than the old school locks and shared state model. It's not old school. Messaging/process and shared state have their own uses. There are plenty of applications where messages would be ridiculous. Sure. What I meant by old school is that there's a lot of code out there which has had threading bolted-on after the fact, usually by using one or a few big locks. But what I really want to address here is the misconception that 0MQ provides a complete way to deal with synchronisation, using inproc. I'm not suggesting that messaging is the one true way, or that using inproc sockets in ZeroMQ encompasses all possible use-cases for synchronization. If anything, I try to stay away from statements like that :-) AFICS, it doesn't (correct me if I'm wrong!): ZMQ sockets are buffered, so the synchronisation is sloppy. This is often OK, but it is isn't a universal solution. It's unreliable (messages can be dropped) and it's slow compared to a lock. Buffered in the sense that it's asynchronous and can be delayed, yes. It's reliable in the sense that *if* you use the correct socket type (*NOT* PUB/SUB!) and set infinite HWMs, then no messages will be dropped as long as the socket is around. So no, not unreliable. The thing is: at present, the thread-safe socket interface provided does not in any way interfere with anything. If you don't use it, you have the old 0MQ non-safe socket model. Yup, but IMHO it goes against encouraging people to use ZeroMQ the right way :-/ -mato -- Martin Lucina mar...@lucina.net ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Thread Safe sockets
On Tue, 07 Feb 2012 07:27:32 +0100 Marten Feldtmann itli...@schrievkrom.de wrote: - thread safe socket I'm not on a level as the other people on this list are, but I only would like to be able to use creation/closing-socket-calls also be possible from another thread (not acually using the socket). Is ths possible - on all platforms ? Context creation and destruction [zmq_init(), zmq_term()] - yes. Socket creation [zmq_socket()] - yes. Socket close [zmq_close()], and any other operation on an existing socket - no. In other words, you can safely create a socket in thread A, then do a pthread_create (socket) and start using it in thread B. You can destroy the socket's context from any thread, and any in-progress or subsequent socket calls on that context will get errno=ETERM back. The context is only deallocated once you zmq_close() all sockets in it. After that, any further calls on that context are undefined (i.e. segfault or possibly EFAULT if you're lucky). For more details see the manpages or http://api.zeromq.org/. Actually I like the way zeromq is done. The few API calls and the external thread sending and receiving. Thanks :-) -mato -- Martin Lucina mar...@lucina.net ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Dev Process
skal...@users.sourceforge.net said: The cards have been dealt, Pieter Hintjens has shown his hand and Martin Sustrik has folded. I can't win this round, and attempting to do so will just look silly. Don't try to win, it's not about winning. There is always politics but this is about software :) Politics matters of course. But you can *use* the policy: you're free to patch anything you like! Fair enough, and I will try and do that, at least for those parts I care enough about. I will also keep my own tree around and ignore the patches I don't find worth fixing, that's what Git is for. As for the politics, there are political issues far more important and worth arguing about, to me, than those surrounding this software project. Also, there's usually not much point in arguing with dictators... where I come from, before 1989, you'd probably be forced to leave the country (best option, rare), relegated to work in the boiler-room (standard solution #1), sent to prison (standard solution #2) or shot (worst option, rare). The beer part would be fun if I was coming to the Portland conference, but schedule- and geography-wise I can't fit it in :-( It's a long trip in my yacht from Australia. Just email me the beer :) A, a fellow seadog! Any chance you'll be sailing 1000-odd miles in an easterly direction? I'll be in New Zealand from 14th Feb through 7th March, beer or rum would be fun :-) You're absolutely right. That's unusual .. can I have that in writing, so I can frame it? Sure, which harbour do I mail a signed copy of the email to? :-) For the record: I will continue to submit patches in accordance with the established process, but I will refrain from being drawn into revert-wars of best patch wins; Have there been any revert wars yet? Actually one thing this new policy really needs is the email hook. It's hard to get into a revert war if you can't see the enemies' patches .. :) I'm experimenting with this: https://github.com/Dieterbe/rss2email/tree/xdg Pick the xdg branch specifically. GH exports a private RSS feed of Your Dashboard, which is everything that happens on repos that you Watch. So you can run an onshore cronjob a couple of times a day that will send you emails off that feed. It's still way suboptimal to actual email (no threads, no replies, etc...), and you don't get the content of the patches in there, but at least you know something has happened. There's probably more than can be done via the Github API, but I don't care enough right now. -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] C++ assertion failed with Java client
p...@imatix.com said: On Sat, Feb 4, 2012 at 11:42 AM, Dirkjan Ochtman dirk...@ochtman.nl wrote: I'm kind of wondering about the private email conversations you, PH and MS had. Would it be possible to publish some of those conversations on this list? I'm not OK with that. Private emails are not for public display, and publishing them would be a breach of confidence, and copyright. But open discussion, sure. This is an important matter and affects us all, and my intention is to make sure everyone's voice is heard and consensus opinion is implemented. I have reviewed the relevant email threads in question, and it is my opinion that there is nothing of commercial importance in them. On the contrary, they do concern the ZeroMQ community. I would like to go on the record as stating: The threads in question are the full conversations defined by the following root messages: Date: Sun, 18 Dec 2011 11:35:32 -0600 From: Pieter Hintjens p...@imatix.com To: Martin Lucina mar...@lucina.net, Martin Sustrik sust...@250bpm.com Subject: 3.1 release Message-ID: cadl5_sid35nnfgeikyylqoamsgnc4xnzzr3duyddv7kk237...@mail.gmail.com (sfid-20111218_183555_619641_B879003B) Date: Tue, 20 Dec 2011 09:16:54 +0900 From: Martin Lucina mar...@lucina.net To: Pieter Hintjens p...@imatix.com Cc: sust...@250bpm.com Subject: 3.1 releases, Distributions Wiki page Message-Id: 20111220091654.05261dc03816046900f83...@lucina.net If, and only if, both Pieter Hintjens and Martin Sustrik give their consent to making these conversations public, in their entire and unredacted form, then I also give my consent to making these conversations public, in their entire and unredacted form. -mato signature.asc Description: Digital signature ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] C++ assertion failed with Java client
dirk...@ochtman.nl said: It feels like we got to see part of the results, but not the cause or origin or what happened behind the scenes. IMO, private email conversations on topics like this are very bad for the community, because there's no level playing field allowing everyone to participate equally. You're right, and Martin Sustrik and myself have also been at fault on this during the time we were maintaining ZeroMQ. I don't have any excuse for that, but there never was any *intent* to keep anything concerning the community private. Also, a lot of discussion between Martin Sustrik and myself got done in person, simply because we live and work in the same city (Bratislava, Slovakia). If I was in that position again, I would make an effort to take minutes of those discussions; although sometimes they might suffer from being drenched in beer :-) Cheers, -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] C++ assertion failed with Java client
p...@imatix.com said: So the problem is that we need to run more tests on libzmq, more often, and we have excellent work that can do this but we're not using it fully. I can think of a few solutions: * Run the PyZMQ test scripts as part of the regular Jenkins build process +1, and there is also the rspec stuff in the ruby-ffi binding. * Carve off these test scripts as a more formal 0MQ test package that can be extended. +1 And: * A good C++ hacker could extend and formalise the existing tests which are already part of libzmq. -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] C++ assertion failed with Java client
Pieter, Martin, Ok, please confirm: p...@imatix.com said: On Sat, Feb 4, 2012 at 2:03 PM, Martin Sustrik sust...@250bpm.com wrote: Sure, I have no objections to making the emails public. No objections either. I think we've done a very good job in the last couple of years in turning the private agreements into public contracts. Everyone involved in this project has a right to know what agreements are made, on what basis. Level playing field, indeed. Do you trust me to take the complete text of these emails and, for the avoidance of doubt, for each each email include the following headers: - From: - To: - Cc: - Subject: - In-Reply-To: - Message-Id: and send the full, unredacted, text of these emails to this mailing list? -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] C++ assertion failed with Java client
p...@imatix.com said: On Sat, Feb 4, 2012 at 2:24 PM, Martin Lucina mar...@lucina.net wrote: Pieter, Martin, Do you trust me to take the complete text of these emails and, for the avoidance of doubt, for each each email include the following headers: Well, I'm at FOSDEM and don't have time to review these emails. Tomorrow, travelling the whole day, and this continues until end of next week. Furthermore this entire thread is wearisome. So in retrospect, and thanks for asking twice, no thank you. I did not write those emails (whatever they are) with the intention of publication. Right, so I got a no, then a yes, and then a no. I will take that to mean no. If anyone wishes to pursue this further, feel free. My word on this matter stands. -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] EPGM packets
sust...@250bpm.com said: Hi Matteo, we just switched our PUB/SUB architecture on 0MQ from TCP to EPGM to reduce traffic and get over the need for a central repeater to have multiple publisher/multiple subscribers. I tcpdump'ed the traffic to see what was going on, but I can only see repeated multicast messages from the publisher, but nothing from the subscribers. Is that normal? Is that because of UDP encapsulation? Yes. It's normal. Subscribers complain only if there's packet loss on the network. For more details check RFC 3208 Specifically, subscribers send NAKs back via unicast to the publisher. Therefore, if you run tcpdump on the publisher, you should see that traffic. -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] C++ assertion failed with Java client
p...@imatix.com said: So, in summary, let's cut the philosophy and focus on targeted *minimal* patches to solve identified problems. +1, although minimal is a matter of opinion. Pieter, with all due respect, it's a shame that this community is turning into one driven by personal ideology rather than a meritocracy based on quality *technical* contributions. That was not what I signed up for when I started contributing to this project. Maybe you could also tone down the ideology a couple of points? I repeat the word *minimal* for emphasis. If you are spending fifteen emails arguing a design, it isn't minimal. Such as your design of defaulting to ZMQ_SUBSCRIBE () for SUB sockets? -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] xpub.cpp line 82
ian.bar...@gmail.com said: On Fri, Feb 3, 2012 at 12:30 PM, Emmanuel TAUREL tau...@esrf.fr wrote: Hello all, Just for your information. I have downloaded one hor ago libzmq from github. The compilation fails due to line 82 of file xpub.xpp. Doh! Sent a pull req for this major change. Huh? How did that happen? -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] ZeroMQ contribution policy - updates
Hi, p...@imatix.com said: Hi Folks, Two small additions to the policy [1]: * Please use real names on github, not aliases (so we know who we're getting code from) Could the maintainers please revert pull request #225 on the basis that it does not fulfill the above requirement? Also, I would like to propose that the following points are added to the contribution policy: * A patch must adhere to the ZeroMQ Coding Style Guidelines, as documented here: http://www.zeromq.org/docs:style. * If your patch includes a substantial amount of code from a foreign project, such code must be placed into its own source file(s), preferably in the foreign/ directory in the ZeroMQ source tree unless this is not technically feasible, and any license and copyright information must be copied verbatim into the corresponding source files. Both of the above should be fairly uncontroversial and are IMO essential for the project. Thanks, -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] ZeroMQ contribution policy - updates
p...@imatix.com said: On Fri, Feb 3, 2012 at 2:03 PM, Martin Lucina mar...@lucina.net wrote: Could the maintainers please revert pull request #225 on the basis that it does not fulfill the above requirement? Not done, instead we clarified the requirement. To clarify; I wasn't referring to Github accounts, but to the Git commits themselves: commit 1e5a48f5217edf874d5771b29ceb7680a930d4b3 Refs: v3.1.0-6-g1e5a48f Author: m 415...@gmail.com AuthorDate: Fri Jan 27 15:24:47 2012 -0800 Commit: m 415...@gmail.com CommitDate: Fri Jan 27 15:24:47 2012 -0800 If this project ever has to endure a SCO-like attack, IMO it'll be very hard to defend with pseudonymous *commits*. There was a discussion about this on LKML some time ago; IIRC the resolution was no pseudonymous commits, and it *may* have later been revised as unless you can convince a gatekeeper that you have a very good reason for not using your real name. Also, I would like to propose that the following points are added to the contribution policy: Done, thanks! Thanks! -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] C++ assertion failed with Java client
aj.le...@quantum.com said: What you're not taking into account with your meritocracy idea is all the quiet users of this library that *aren't* contributing patches, but rely on the functionality of the library for their work. To clarify what I meant by meritocracy: An Open Source project, with a benevolent dictator (or gatekeeper), and a community of contributors. The standard model, as employed by many successful Open Source projects out there. When I started contributing to ZeroMQ in late 2009 [1], I did so on the understanding that this project followed that model, and that the benevolent dictator was Martin Sustrik. It turns out those were false premises; and I only realise that now, *two years later*, after a long conversation with Pieter Hintjens yesterday. The crux of many of my arguments with Pieter comes from this single misunderstanding from two years ago; according to Pieter this was never the case, and it was only by his good grace that we (Martin and myself) were allowed to maintain ZeroMQ or parts thereof from late 2009 until mid-2011 in my case, and December 2011 in Martin Sustrik's case. So, to set things straight, as I was told yesterday in no uncertain terms: 1. Pieter Hintjens claims this community as *his* [2] 2. Martin Sustrik has resigned as the benevolent dictator [3] To borrow John Skaller's term from another thread, Pieter was the meta-dictator all along, and I didn't realise that. My bad. If we're going to tip zmq on it's head because a couple very vocal contributors decide they don't like the way things have been working, it makes me very concerned about continuing to use this project. What you are seeing now is Pieter implementing his vision for the community. Please note that this is not an attack against Pieter -- this is being done in consultation with the community, but ultimately rests on Pieter's power as meta-dictator. Is this something that I can rely on, or will it turn into an open source project that bounces all over the place depending on the whims of the contributor of the week? I want people to contribute - I'm very happy that my fixes for various platforms have been accepted without issue - but I am concerned that project could be hijacked by someone that has a very specific use case for it and doesn't understand the history and philosophy of the proejct and what its current users are doing with it. That depends on where the new process leads the community. I have argued consistently against the more controversial points, with little success -- my rhetoric is not up to meta-dictator standards :-) If you care about this, the only way [4] to achieve change today is to be vocal, and to criticise and argue those points of the process which you care about. Having known Pieter personally for more than 10 years now, and spent many many hours arguing with him, I wish you luck! TL;DR: One last point: In my humble but correct opinion: Under Martin Sustrik's lead, we had a mediocre community, and a great product. Under Pieter Hintjens' lead, we have a great community, but are rapidly progressing towards a mediocre product. Welcome to the Brave New World of software-engineering-as-a-wiki! -mato [1] At the time we published the LWN article with Martin Sustrik: http://lwn.net/Articles/370307/ [2] Stated by Pieter many times on this mailing list, if not in so few words. [3] Private e-mail thread between PH, MS and myself, December 2011. [4] The other option is a fork (in the formal sense, not the Github sense). This is an option of *last resort*, and is not something to be taken lightly. However, it does put you on a fair playing ground; build your own community and process, and ultimately users will follow the leader with their feet. ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] C++ assertion failed with Java client
cremes.devl...@mac.com said: Well that's enough pontificating I'll go back to hacking the broken Node binding now (It breaks under Node's move to libuv ...nothing really to do with 0MQ which I'm barred from hacking directly :( ) So sad. The library needs some serious love on the Windows platform (ipc transport, zmq_poll, etc) but most of the contributors are UNIX-only. Please do some lobbying if you can! Indeed! IOCP, please! Lots of the current performance and scalability problems on Windows come from the fact that we're using select(). IOCP is the obvious way to go there, but needs a hardcore hacker to map the IOCP concepts onto libzmq internals. -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] C++ assertion failed with Java client
p...@imatix.com said: The debate here is whether the old meritocracy can deliver that quality or not. The evidence suggests it can, but only with huge effort. However, over the last years I've been able to maintain several 2.x releases with minimal effort, and maintain high quality, by allowing _anyone_ to submit patches and at the same time apply a tough fix test process. Agreed, delivering quality is hard and costly. The new process appears to be an experiment in eliminating that cost. Please note that you maintained the 2.1 stable releases using a completely different process than that which is now being applied to master. Further, under that process, you only had to deal with a small amount of fixes to existing bugs in the code base. You did not have to deal with new code or contributions. IMO in the days of Martin Sustrik's lead, there was no unstable release, or at least not in the sense that we have now, since master was carefully vetted and maintained. Sure, there were other problems with what went on master, but code quality was not one of them. -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] Foreign library dependencies
skal...@users.sourceforge.net said: I also asked a question earlier that didn't seem to get answered: what is the attitude introducing a dependence on a foreign library? (Source can be included in zmq without licence issues). In this case, Judy, since that determines the overhead. I'd been meaning to answer this, but got swamped by all the other emails :-) The policy that we followed with Martin Sustrik while maintaining ZeroMQ was to avoid external dependencies where possible. Any external dependency creates a burden on users, distributors (Debian, etc.) and maintainers of the depending project. The major exception was OpenPGM, which got packaged with libzmq both because originally we used a specific version, and because a significant part of our target user base are people who want to use PGM multicast. A major part of software development (IMHO) is knowing what to keep out just as much as what to put in. Keeping the libzmq core lean mean follows this principle. Regarding Judy, I had a brief look at the website, and grabbed the code. The download is a ~1MB tarball, and a quick count of the src directory gives: $ find . -type f | xargs wc -l [...] 29314 total Doing the same for libzmq master gives: 20352 total So, if it were my decision, I would need to consider the benefit of Judy arrays vs. the cost of including 29000 lines of code (more than libzmq core!) as a dependency. Of course, there is no such policy now, so feel free to submit a patch! -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Dev Process
skal...@users.sourceforge.net said: Under Martin Sustrik's lead, we had a mediocre community, and a great product. Under Pieter Hintjens' lead, we have a great community, but are rapidly progressing towards a mediocre product. Can you point to issues in the actual source and documentation that explain that view in more detail? Yes, but doing so would continue this war of words, and I have no wish to do that, now that I understand the positions of the players involved. The cards have been dealt, Pieter Hintjens has shown his hand and Martin Sustrik has folded. I can't win this round, and attempting to do so will just look silly. You're welcome to re-read my past posts to the list on this topic, and you and anyone else interested are also invited to discuss this further, off-list, in the lets have a fun argument about this over beer sense. The beer part would be fun if I was coming to the Portland conference, but schedule- and geography-wise I can't fit it in :-( I'm too new here to have a view, but I think that as we're in Pieter's contribution phase for 3.1, and are yet to get up to the rigorous testing phase intended to stabilise the product .. well you don't seem to be giving Pieter's model a chance. You're absolutely right. In fact, Pieter has asked me privately to please give the model a chance, so out of respect for Pieter as a person, not Pieter as Meta-Dictator, I'll stop actively criticising the process now. For the record: I will continue to submit patches in accordance with the established process, but I will refrain from being drawn into revert-wars of best patch wins; instead I will continue to review and comment on interesting patches as and when I have time. In other words I am opting out of this point: Anyone who is discontent with a patch will make their own patch to fix or reverse the previous patch. Again, maintainers have no opinion on this. ^^^ -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] conventions in bindings
skal...@users.sourceforge.net said: [Good definitions of message/fragment/frame/message-buffer/message-container snipped] Really, the biggest problem is that zmq_msg_t object is called a message. The use of message comes from the term message-oriented-middleware and message queue, both of which can be used to describe ZeroMQ to some extent. [semantic meaning of message snipped] I have one further input here: if there were a small team which would rewrite the API specification to improve precision and consistency .. and historically, Standards bodies that do this sometimes **deliberately** change away from existing practice to avoid confusion, I suggest such term should research existing POSIX terminology. It would also be good to copy the description model used by the Open Group: http://pubs.opengroup.org/onlinepubs/007904975/mindex.html http://pubs.opengroup.org/onlinepubs/007904975/functions/send.html and I would note that this already seems to be the goal. Note in this selected example: Speaking as the original author of the ZeroMQ manual pages / API reference, the POSIX/opengroup texts were a direct inspiration for me when I was writing the ZeroMQ API manual. Thanks for noticing :-) I agree that the terms message and message part are used interchangeably in the current documentation, and that this is confusing. Multi-part messages were introduced while I was already writing the API documentation, and thus their documentation is a patch onto the original terminology, rather than a proper rethink of terminology. The API documentation is in need of some work as it stands today and I would certainly like to improve this; an off-the-top-of-my-hat estimate is that it needs a couple of man-weeks of concentrated time to address these issues. John, I would be very interested in working with you on this, however right now I'm on the tail end of a fairly large project for a client, so the earliest I can make time for this is mid-March. Another thing I would like to improve is the code examples in the API reference, both by improving the snippets of code in the individual reference pages for API calls, and by writing a new zmq_tutorial(7) page consisting of a series of worked examples; one per messaging pattern, using the libzmq C API, and being enough to get something working. -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] conventions in bindings
p...@imatix.com said: On Thu, Feb 2, 2012 at 2:19 AM, Gary Wright at2002+...@me.com wrote: message: fragment: frame: Makes sense. fragment is long, for an API word. But in fact the only real issue to solve here is consistency: any semantics will work, if repeated consistently. So there will be a lot of changes to make in bindings, code, and docs. Anyone proposing changes has to be willing to take this work on. There is no other way it will happen. (I think someone should do X does not work and never has). IMHO most of the changes discussed in this thread (but I haven't followed the whole discussion, so I may have missed some) are documentation changes. If I end up working on this, I will treat renaming of actual stable API calls or datatypes with extreme prejudice. -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] C++ assertion failed with Java client
skal...@users.sourceforge.net said: On 02/02/2012, at 10:18 PM, Pieter Hintjens wrote: On Thu, Feb 2, 2012 at 6:42 AM, Kishore Kopalle kkopa...@gmail.com wrote: I am receiving the following errors when I try to work in a multi-threaded environment (10 threads) where ZeroMQ is used to send/receive messages You are using 0MQ sockets from multiple threads? That will crash. Q: is it technically feasible to make a thread-safe wrapper around 0MQ sockets? Sure, chuck a bunch of locks around socket-related calls? A mutex implies a full memory barrier, so things should work OK at least on cache-coherent architectures. Performance will take a nose-dive of course, but if that is not a concern, go for it ... -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] [PULL] Fix for LIBZMQ-268 (first message sent on PUB socket with PGM is always lost)
Hi, please pull branch 'libzmq-268' of git://git.lucina.net/libzmq.git. From the commit message: Fix data loss for PUB/SUB and unidirectional transports (LIBZMQ-268) With the introduction of subscription forwarding, the first message sent on a PUB socket using a unidirectional transport (e.g. PGM) is always lost due to the subscribe to all being done asynchronously. This patch fixes the problem and also refactors the code to have a single point where the subscribe to all is performed. Signed-off-by: Martin Lucina mar...@lucina.net Available for review at: http://git.lucina.net/libzmq.git/commit/?h=libzmq-268id=0319cb2cd16aa40911855a1765312886bf081db2 Thanks, -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] [PULL] Fix for LIBZMQ-268 (first message sent on PUB socket with PGM is always lost)
On Thu, 02 Feb 2012 14:24:01 -0600 Chuck Remes cremes.devl...@mac.com wrote: Merged in commit 4f4d72a. Thanks! For the record, you also pulled 82d9353 as a bonus, my fault for not sticking that on a separate topic branch. I had completely forgotten about it! Anyway, no harm done, it's the fix for LIBZMQ-294; I'll update that JIRA and close the other JIRAs as well. -mato -- Martin Lucina mar...@lucina.net ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] [PULL] Fix for LIBZMQ-268 (first message sent on PUB socket with PGM is always lost)
On Fri, 3 Feb 2012 09:17:12 +0900 Martin Lucina mar...@lucina.net wrote: On Thu, 02 Feb 2012 14:24:01 -0600 Chuck Remes cremes.devl...@mac.com wrote: Merged in commit 4f4d72a. Thanks! For the record, you also pulled 82d9353 as a bonus, my fault for not sticking that on a separate topic branch. I had completely forgotten about it! Whoops, sorry, I misread the git log... sustrik committed this on Dec 15th. -mato -- Martin Lucina mar...@lucina.net ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] C++ assertion failed with Java client
On Fri, 3 Feb 2012 00:39:39 +1100 john skaller skal...@users.sourceforge.net wrote: BTW: are you so sure performance will suffer? Yes. The reason I ask is related to existing overheads: (a) malloc/free is used extensively in real applications, it is already thread-safe so it is using at least some combination of locks and TLS libzmq goes to great lengths to optimise memory allocation where possible. See e.g. the VSM functionality. There is several *years* tuning work that has been done on this. (b) the use of errno by 0MQ is a performance overhead already: errno is in TLS which has a nasty overhead. If you want to improve performance give up on, or at least provide a sane alternative to, errno. One extra indirection per load, that has nowhere near the overhead of a lock. Or does TLS do something more than that? If so, why? Fast barriers are provided on multi-cores, implemented directly in the hardware. There are also lock free synchronisation primitives on all modern CPUs. ^^ Primitives yes. You still need to implement the full lock free algorithms, e.g. ypipe_t. Speaking from experience, getting lock free algorithms right is really hard. -mato -- Martin Lucina mar...@lucina.net ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] conventions in bindings
On Tue, 31 Jan 2012 18:06:06 -0800 MinRK benjami...@gmail.com wrote: On Tue, Jan 31, 2012 at 17:23, Martin Sustrik sust...@250bpm.com wrote: Hi, * SUB sockets default to SUBSCRIBE() instead of None This becomes a problem with 3.0 and subscription forwarding. The problem occurs when you don't want to subscribe to all messages. You open a socket and unsubscribe from . However, between the two operations you can be overloaded by huge amount of irrelevant messages from the publisher. In fact you could get the same problem with 2.x, but with subscription forward it gets worse. To put this into perspective, imagine you connect to a firehose feed of twitter using ZeroMQ. If your default SUB socket behaviour is SUBSCRIBE () then in all likelyhood you've probably just DoSed yourself. ZMQ_SUB defaulting to SUBSCRIBE() is broken behaviour, period. * LINGER is a property of the Context, and sets a default value for its sockets, which is 0 by default, instead of -1 The problem with default LINGER equal to 0 is that following program doesn't send any message (the socket is destroyed before connection is established): s = zmq_socket (ctx, ZMQ_PUSH); zmq_connect (tcp://server0001:); zmq_send (s, ABC, 3, 0); zmq_close (s); That's an extremely unintuitive behaviour. If default linger time of infinity is causing troubles I would rather suggest going for something like 1 second. It sounds like the recommendation here is that czmq should stop doing what it does, rather than other bindings following suit. +1 Both the ZMQ_LINGER defaults to inifinity and SUB socket defaults to no subscriptions were discussed to death on this list a year or so ago. I can dig up pointers to the threads if you want. I would recommend that language bindings follow the libzmq core behaviour. -mato -- Martin Lucina mar...@lucina.net ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Feedback on pubsub pull requests (trie/mtrie fixed, etc.)
skal...@users.sourceforge.net said: On 30/01/2012, at 8:49 PM, Staffan Gimåker wrote: Hi, Pieter asked me to ask for feedback on some pull requests which got merged to master, so here we go: I'm confused.. ~/zeromq/libzmqgit pull Already up-to-date. You're not confused, these changes have already been merged into master, so you probably already have them in your clone of the libzmq repo if you did a git pull[1] recently. As Pieter says, check the commit log on Github, git log on your local repository, or if you like console-based tools I find that tig[2] is really useful. AFAIK they have been discussed either in JIRA and/or in the long Exact matching on subscription topics thread on this list. -mato [1] You only need the full git pull origin master if you have a pretty old version of Git, and/or an old clone. Otherwise just git pull will pull from wherever you cloned your repo from. [2] http://jonas.nitro.dk/tig/ ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Feedback on pubsub pull requests (trie/mtrie fixed, etc.)
Pieter, p...@imatix.com said: Thus, I'd advise people to send patches early, rather than late, since it takes about 6 months for new code to make its way into a stable release. If you time it right, you can roll into production on a stable release that contains the fixes you depend on. The current process does a great job of documenting how one can get code into the official libzmq master, and also how bugfixes are applied to and reviewed for the current stable release (i.e. 2.1). What is missing is documentation on how / when a release / branch is declared stable, or as you write above, how code makes its way into a stable release. Cheers -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Contribution policy and quality of commits
Hi Chuck, cremes.devl...@mac.com said: Mato, I understand that you have little desire to join github. I'll volunteer to process any patches that you create. Send them to the list for comment (like before) and I'll handle pulling from your non-github repository and pushing to libzmq. Thanks! Would you prefer actual patches sent directly to the list as before, or an email asking to pull from branches x, y and z from my repository? (e.g. see the email I sent yesterday) All my repositories are browseable at http://git.lucina.net/, and I can add links to the branches in question to the pull request email so that anyone can review the patch easily without needing to pull it if they want to do that. I have no prefrence either way, what is easier for you? -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Contribution policy and quality of commits
p...@imatix.com said: Mikko is working on this. We are seriously overloaded at the moment. If anyone wants to help set this up, I believe https://github.com/samwho/GitHub-Pull-Request-Email-Bot is a good place to start. In the meantime I'll edit the contribution page to ask people to email zeromq-dev when they make new PRs. Great, thanks Pieter. -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] Pull request for patches to LIBZMQ-303, LIBZMQ-205
Hi, these have been sitting in my staging repo for some time now. It would be good to get these into libzmq master, but it seems it's not possible to create a pull request on Github without hosting the actual repo to be pulled from on Github. For various reasons which it would be OT to discuss here, I do not want to host my staging repo or a mirror thereof, on Github. Anyhow, the patches are on branches 'libzmq-205' and 'libzmq-303' of git://git.lucina.net/libzmq.git, so if the maintainers would like these someone needs to do a: git pull git://git.lucina.net/libzmq.git libzmq-303 libzmq-205 in the libzmq repository, or I can just send the patches to this list directly. Thanks, -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Patch build issues on old-linux
Hi Michael, 415...@gmail.com said: I submitted a pull request on github, to fix build issues on old Linux (Redhat circa 2002, 2.4 kernel, 3.2 gcc). Thanks for the patches. But it looks like it may also be necessary to post the patch here. I'm not sure how you want it exactly, so what I'm doing is: git format-patch 6f32361fea61619fec94348de693a9e3ff8981e0 It'd be nicer to have the patches inline rather than as attachments, since then I could quote from the patch directly for review instead of cut and pasting. Anyway, here goes. It says: 0001-Fix-bad-combination-of-gcc3-Werror-and-private-destr.patch +1 0002-Fix-basic_string-unsigned-char-not-implemented-in-st.patch 1) The borrowed code belongs in its own header file, say src/gcc3_strings.hpp (for want of a better name), *not* in blob.hpp. 2) This file MUST have the id3lib license copied to it verbatim. 3) Your commit message should state the exact origin of this code with a URL or similar. 4) The patch does not conform to http://www.zeromq.org/docs:style. If the code went verbatim into its own file this is fine, but again, not in blob.hpp. 0003-Fix-runtime-patch-for-when-system-has-clock_gettime-.patch 1) The patch does not conform to http://www.zeromq.org/docs:style. 2) You yourself state: + // This should be a configuration check, but I looked into it and writing an + // AC_FUNC_CLOCK_MONOTONIC seems beyond my powers. I would report an issue and/or ask for help with writing AC_FUNC_CLOCK_MONOTONIC, but YMMV. 0004-Fix-Case-where-system-library-has-epoll-but-kernel-d.patch Seems fine. Cheers, -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] Contribution policy and quality of commits
Hi all, as you may have noticed, a new contribution policy for libzmq has been put in place: http://www.zeromq.org/docs:contributing I would like to draw your attention to the following points: The team that manages patches back to libzmq are the maintainers. Maintainers are not developers and they have no opinion on patches; their job is to enforce process. and The maintainers will aggressively and optimistically merge pull requests back to libzmq master, with minimal opinion on quality and relevance. The only hard requirement is that the patch can be automatically merged. and Anyone who is discontent with a patch will make their own patch to fix or reverse the previous patch. Again, maintainers have no opinion on this. and finally, If you want an analogy for the Maintainer role, think of Wikipedia's Edit this page function. Reading the points above it seems to me that the maintainers have absconded themselves of any QA role, based on a view of patches to code being equivalent to literary prose. Based on the process above, the role of a maintainer equates to a robot clicking on apply this pull request. If you look at the quality of commits to libzmq since this policy has been put in place, from a software engineering point of view the quality of commits is suboptimal. While it is a laudable goal, if you are Wikipedia, to enable anyone to contribute and rewrite anyone elses prose, libzmq is primarily a *software engineering* project. The cost of rewriting prose is low compared to the cost of rewriting code. Thoughts? Most of us here are software engineers by trade, surely you can see where this is leading. -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Contribution policy and quality of commits
cremes.devl...@mac.com said: What specific suggestions do you have for improving the process? (Demanding that all patches be rigorously inspected and tested by the maintainers is not a reasonable, though specific, suggestion.) Code review is in fact specifically what I am asking for. Every major FOSS project has a code review policy: Linux kernel: (see section 14), and elsewhere in this document: http://lxr.linux.no/#linux+v3.2.2/Documentation/SubmittingPatches Mozilla: http://www-archive.mozilla.org/hacking/code-review-faq.html Android: http://source.android.com/source/submit-patches.html FreeBSD: (the best I could find, maybe the process is better documented elsewhere) http://www.freebsd.org/doc/en_US.ISO8859-1/books/developers-handbook/policies-maintainer.html The libzmq process as defined leaves no provision for code review whatsoever. I realise that code review is time consuming, costly, and often boring. If my suggestion is unreasonable due to the fact that the current maintainers are unpaid volunteers, then the real problem is elsewhere; the community should get its act together and: a) recruit a competent maintainer with domain expertise from within its ranks b) put in place a mechanism to fund such a maintainer -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Contribution policy and quality of commits
mar...@lucina.net said: The libzmq process as defined leaves no provision for code review whatsoever. I realise that code review is time consuming, costly, and often boring. If my suggestion is unreasonable due to the fact that the current maintainers are unpaid volunteers, then the real problem is elsewhere; the community should get its act together and: a) recruit a competent maintainer with domain expertise from within its ranks b) put in place a mechanism to fund such a maintainer I missed a very important point which has been brought up before - When patches were sent to the mailing list *before* being committed to the libzmq repo, this provided a window of time for voluntary code review and discussion. This works because there is almost no effort required on the part of potential lurker reviewers. Email is a push system with PUB/SUB topic filtering using the human mind; we're really good at this. One can lurk on the mailing list, skim threads that one has no opinion on or are outside of ones domain expertise. When a relevant thread or *PATCH* comes up, you can trivially comment on it. This is no longer possible with the current process. I would have to actively spend time on a) going to the libzmq github page with pull requests b) somehow figuring out which I have read, have not read, and which are relevant. lots of clicks involved. c) commenting on the pull request itself. Even then, the PUB/SUB model is broken. Only those people who *actively* monitor Github lest they miss some important commit get a chance to review it. So the current process actually *decreases* the amount of voluntary code review that could be provided by the community. -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Binding to TCP port 0
aj.le...@quantum.com said: On Thu, Jan 26, 2012 at 07:53:06AM -0600, Pieter Hintjens wrote: On Thu, Jan 26, 2012 at 2:17 AM, Ian Barber ian.bar...@gmail.com wrote: Is it possible that this API should simply be on a different calls, a zmq_bind_assign or similar, to avoid this clash? This could then work with all transports: tcp/pgm: bind to random port ipc: bind to random free name in system tmp inproc: bind to random free name That would work, but so would a less surprising zmq_bind plus a getsockopt with return value depending on transport type. Forgive me if I'm being dense, but couldn't the N case work if you made sure to do the getsockopt() call after each zmq_bind()? zmq_bind(foo, tcp://:0) zmq_getsockopt(foo, GET_LAST_BOUND_PORT, port1, sizeof(port1)) zmq_bind(foo, tcp://:0) zmq_getsockopt(foo, GET_LAST_BOUND_PORT, port2, sizeof(port2)) Yup, that would work. Sorry for missing the point about zmq_bind() being synchronous. Ian also made a great point that this can be extended to bind to unnamed endpoint (for want of a better name) for all transports. So, what I think we want is that the getsockopt API should return a string (so as to be usable for multiple transports, plus returning a different-data-type-per-transport is a PITA). It'd also be nice to define a consistent way to specify this unnamed endpoint for all transports that might want to provide such functionality. :0 happens to work for the tcp:// case, but does not really make sense for inproc:// or ipc://. This leaves us with something like this proposal: zmq_bind(foo, tcp://:*);// tcp://*:* if you want INADDR_ANY char endpoint [ZMQ_ENDPOINT_MAX]; zmq_getsockopt(foo, ZMQ_GET_ENDPOINT, endpoint, sizeof endpoint); = endpoint is filled as tcp://:12345. zmq_bind(foo, ipc://*); char endpoint [ZMQ_ENDPOINT_MAX]; zmq_getsockopt(foo, ZMQ_GET_ENDPOINT, endpoint, sizeof endpoint); = endpoint is filled as ipc:///tmp/Xyz358hfA7. inproc:// semantics would be identical to ipc:// (w/o the /tmp/ prefix obviously). The use of * seems fairly uncontroversial -- note that this means an ipc:// endpoint cannot therefore contain * which is an (albeit niche) backward-incompatible change. Thoughts? I'll ping Martin Sustrik tomorrow to see if he thinks there's any reason why this wouldn't work; I believe he's ignoring this thread as TL;DR :-) -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] Contribution process (was Re: Fix for alignment issue of vsm_data inside zmq_msg_t message type.)
Pieter, On Thu, 26 Jan 2012 11:44:03 -0600 Pieter Hintjens p...@imatix.com wrote: Can I assume this affects both 2.1 and 3.1? Could you make a pull request on the libzmq repository then? We'll backport to 2-1. See here for contributing: http://www.zeromq.org/docs:contributing Is the new process formally in place now? I have not seen any announcement. -mato -- Martin Lucina mar...@lucina.net ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] EPGM on Linux 4.1.2
Hi William, On Wed, 25 Jan 2012 18:45:37 -0500 William Brown william.br...@ericsson.com wrote: On my Mac, this works great, except that I do have to substitute my IP address for the interface name. On my Linux system the bind and connect work and the publisher appears to be publishing the messages. The problem is that the receiver is getting nothing. The eth0 interface does contain the following UP BROADCAST RUNNING MULTICAST MTU:1500 etc, etc. I've got plenty of sleep in the example program between subscriber start and publisher start, so I shouldn't be missing any messages. Any thoughts on why this program doesn't work on Linux? Sanity check - does the same program work on your Linux box if you use tcp:// endpoints instead of epgm://? You might want to check on the publisher side with Wireshark or tcpdump to see if the multicast packets are actually hitting the network. Cheers, -mato -- Martin Lucina mar...@lucina.net ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Binding to TCP port 0
Hi Pierre! On Wed, 25 Jan 2012 18:22:55 +0100 Pierre Ynard linkfa...@yahoo.fr wrote: If you're using the high level C binding this works. It's not a core libzmq functionality. Some other bindings may also provide it. You can check the CZMQ code that does this: https://github.com/zeromq/czmq/blob/master/src/zsocket.c Thanks, but I don't really want to write an ugly for loop to (badly) reinvent a feature of the socket API. Also this code ignores the OS settings for local port assignation. +1, I have proposed this functionality before, but we could not figure out a way to make the API work. Its obvious that many people need this, and have implemented it with the same hack as czmq. Why is it disabled in libzmq? It is disabled because in ZeroMQ, by design, there is no way to work with individual underlying connections of a socket. Hence, there is no API equivalent to e.g. getsockname() which you could use to get the port back from the OS. If you can think of a way to make this work, without exposing the details of the individual connections to the application, and without a backward-incompatible API change, please propose it (or even better, make a patch). -mato -- Martin Lucina mar...@lucina.net ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] EPGM on Linux 4.1.2
Hi William, On Wed, 25 Jan 2012 19:17:20 -0500 William Brown william.br...@ericsson.com wrote: Hi Martin, Thanks for the quick reply. Yes, I have several test cases developed for most of the ZeroMQ example patterns, including standard pub/sub, fan-in, fan-out, peer-to-peer and router/dealer (broker pattern). All work perfectly well when using TCP and/or IPC endpoints. Ok, so it looks like a problem with PGM. Moving back to ZeroMQ 2.1.11 seems to have resolved my problem with EPGM. I'll stick with ZeroMQ 2.1.11 for the time being and will wait for 3.1 to come out of beta before trying again. I have been using PGM with ZeroMQ 3.1 at a client quite extensively and it mostly works. It'd be nice to figure out why its not working for you; it may just be some small detail, OpenPGM can be somewhat fiddly to get to work. If you do get a chance do try and investigate with tcpdump to see what is happening on the network. As an aside, what Linux kernel and distribution version are you using exactly? You wrote Linux 4.1.2 and mention Redhat, but I presume you're not actually using Redhat 4 which is ancient. Cheers, Martin -- Martin Lucina mar...@lucina.net ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Binding to TCP port 0
On Wed, 25 Jan 2012 18:39:02 -0600 Pieter Hintjens p...@imatix.com wrote: On Wed, Jan 25, 2012 at 6:19 PM, Martin Lucina mar...@lucina.net wrote: It is disabled because in ZeroMQ, by design, there is no way to work with individual underlying connections of a socket. Hence, there is no API equivalent to e.g. getsockname() which you could use to get the port back from the OS. The functionality regards binding not connections, this isn't the concern. s/connections/endpoints/. If you can think of a way to make this work, without exposing the details of the individual connections to the application, and without a backward-incompatible API change, please propose it (or even better, make a patch). Current working code would not break by returning the port number via the zmq_bind API. How? zmq_bind() does not have an out parameter. You mean abusing the (int) return value of zmq_bind() to return a port number? That's horrible and not all transports have such a thing as a port number, so it's not generic either. The other solution, if this is too ugly (which is debatable) is via a zmq_getsockopt call. That was your suggestion previously, iirc. Both are worth exploring. I don't recall any valid technical concerns with this change proposal, it came down to opinion. The major technical problem which everyone (including me) forgets every time this comes up is that you can bind a single ZeroMQ socket to multiple endpoints. Cf. what I said above, some of the bound endpoints may not even have such a thing as a port number. To create an API for this you basically end up with some way to enumerate the bound endpoints and get them to the application. And at that point we're coming dangerously close to letting people enumerate connected endpoints, etc. -mato -- Martin Lucina mar...@lucina.net ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Disconnecting from permanently dead endpoints
Hi Pierre, On Wed, 25 Jan 2012 16:25:59 +0100 Pierre Ynard linkfa...@yahoo.fr wrote: I want to set up a messaging system where a zeromq socket connects to several endpoints (with TCP). Some of these endpoints may go permanently down, and the corresponding transport addresses could even be reused by a totally unrelated service. So I need to a way to disconnect the zeromq socket from this endpoint, as keeping around zombie endpoints to periodically try to reconnect to them is unnecessary, undesirable and does not scale. Understood. The only way I see to do this is to close the socket, create a new one, and reconnect it to all the remaining endpoints, but that's quite suboptimal. Did I overlook something? Correct, recreating the socket is the only way to accomplish what you need right now. An API for zmq_disconnect() and zmq_unbind() would probably not be controversial, but the implementation would be ... hard. This dives right in to the termination mechanisms in libzmq which are really complex. -mato -- Martin Lucina mar...@lucina.net ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Disconnecting from permanently dead endpoints
On Thu, 26 Jan 2012 14:00:55 +1100 john skaller skal...@users.sourceforge.net wrote: On 26/01/2012, at 12:32 PM, john skaller wrote: Isn't it also a design fault? I mean, if you can connect/bind to multiple endpoints dynamically you obviously should be able to un-connect and un-bind. API: int disconnect(void *socket, void *connection); int unbind(void *socket, void *connection); Yup, this is almost the same as what I was thinking. Semantics: When you connect or bind a socket, 0MQ associates the char* address of the endpoint name with the internal representation of the endpoint. void *connection won't work; you would then need to get a handle for the internal representation of the endpoint out of libzmq when you call connect() or bind(), in order to be able to call unbind() or disconnect () on it. I would go with just calling unbind() or disconnect() with the char* you originally passed to bind() or connect(). Semantics are pretty much the same; it is obviously an error to unbind () or disconnect() an endpoint which was not bound or connected in the first place. -mato -- Martin Lucina mar...@lucina.net ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Binding to TCP port 0
On Wed, 25 Jan 2012 19:28:00 -0600 Pieter Hintjens p...@imatix.com wrote: No-one is asking for an API that lets you bind 10 times to port zero and get all ten results. 100% of use cases here are for binding once to port zero and getting exactly one result back. Indeed. In fact, the OP was specifically asking for a use case where he could bind an unspecified number of times to port zero, and get zero results back. :-) It is bad design process to exaggerate the user's requirements into absurd extremity and then state there is no simple solution. It's far wiser to understand the actual problem people are trying to solve, and solve that simply and minimally. Given that an API will stick around for a long time, it is good API design process to extrapolate current and possible future use cases and arrive at the simplest possible API covering all these use cases. -- Martin Lucina mar...@lucina.net ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Disconnecting from permanently dead endpoints
On Thu, 26 Jan 2012 17:50:19 +1100 john skaller skal...@users.sourceforge.net wrote: On 26/01/2012, at 2:54 PM, Martin Lucina wrote: I would go with just calling unbind() or disconnect() with the char* you originally passed to bind() or connect(). That's exactly what I said :) Semantics are pretty much the same; it is obviously an error to unbind () or disconnect() an endpoint which was not bound or connected in the first place. You may be right, however that's not what my spec says: when you disconnect you are disconnecting a SET of endpoints. This is because connect/bind can be called with a char* to a buffer which can be memcpy() into with the end point name. So all such endpoints have the same char* address. *click* Oh, I get it - saw the void * and didn't realise you wanted to use the char * address as a handle to the endpoint. Neat trick, but rather too far in terms of overloading for my taste. Also, it won't work terribly well for language bindings since it relies on a string also being a pointer, which just happens to be the case for C. What I have in mind is more conventional: ZeroMQ takes a copy of the string passed to bind() or connect(), stashes it in the socket. unbind () or disconnect() are again passed a string, trawl thru all the bound/connected endpoints for the socket, and succeed if a match is found. So since you're disconnecting a SET .. well there's nothing wrong with an empty set! Instead, the number of matched endpoints is returned by the function. The client can then decide if 0 is an error or not. Sure, this makes sense if the char * is also considered as a handle. If it's just a string then there is no set involved since it is an error to bind or connect[1] to the same endpoint more than once. -mato [1] Actually, Martin Sustrik tells me you can probably connect (but not bind) more than once to the same endpoint in the current implementation, but that makes no real sense anyway so should return an error if it does not already. -- Martin Lucina mar...@lucina.net ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] multi-part messages
Pieter, On Tue, 24 Jan 2012 11:59:18 -0600 Pieter Hintjens p...@imatix.com wrote: On Mon, Jan 23, 2012 at 6:45 PM, Martin Sustrik sust...@250bpm.com wrote: It's a backward incompatible change. The problem with backward incompatible changes is that you can do them only so often, if at all. The reason being that you are forcing 1000s of users through the pain of upgrading, rewriting their application code, re-testing etc. Well... this is just a matter of will power. There is no reason new features have to break old ones. ... if and only if you have unlimited developer resources and prefer maintaining backward compatibility above all else! [1] In the case of libzmq, the core developer resources are very much limited, so the choices going forward are also limited. This may actually be a good thing since even though it annoys users expecting a stable product, the end result is a smaller, simpler and more elegant iteration of ZeroMQ. Say you want to introduce labels. What you do first is allow the protocol to support these without breaking. There have been proposals for that, we can revisit them if needed. Second, you expose labels via NEW socket types so you don't break old code. Third, as the new features become stable you deprecate the old features. Finally, some years later, you remove them. Having done some substantial work on the libzmq core code recently [2] I can tell you that the multipart message support affects all parts of the codebase. Same goes for the old identities/durable sockets which got ripped out of 3.1; there are still various leftovers from that in the codebase which unnecessarily complicate things and should be excised ASAP. Martin Sustrik has said several times that the identity support is like a cancer on the codebase and I completely agree. This is tried and tested practice. The fault with 3.0 and 4.x was ignoring cost/benefit and simply breaking old code with no attempt to provide an upgrade path. That's what people disliked, and that's why those branches were not used. I don't think Martin is ignoring anyone intentionally, just trying to get the best result with the least possible resources. -mato [1] your name is Bill and what you end up with is Windows. [2] hopefully to be released publicly some time in the next month or so, watch this list... -- Martin Lucina mar...@lucina.net ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Testing framework
Hi Stefan, staf...@spotify.com said: Are there any plans to migrate libzmq to use a more fully-featured testing framework (e.g. googletest)? Not at the moment, but it's mostly a case of finding someone interested and persistent enough to do the work. I believe that Steven McCoy was working on something with googletest at some stage, not sure how far he got. The current automake test system is a bit of a pain to work with since it's missing stuff like a nice way to group related tests, a standardized way to do fixtures, sensible messages from failed assertions, etc. While googletest does seem to have more features, at least some of the above (sensible messages, for example) could be achieved by improving the existing test programs. It'd be nice if someone volunteered to improve the existing testsuite (or create a new one, assuming it's at least as good as the existing one). I know that some of the language bindings (Python, Ruby) have extensive test suites which would be a good start. Aside from an added (compile-time) dependency, are there any reasons why the automake way is preferable? Portability? We currently have autobuilds on all the following platforms (more welcome, btw): http://www.zeromq.org/docs:builds From the googletest site it's unclear how good the support for at least Solaris and FreeBSD is, and dropping those platforms would be a regression. -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] does 0mq keep-alive for connections?
se...@yandex-team.ru said:
21.01.2012 01:56, Sergey Matveychuk wrote:
clients. netstat -a on server shows no connection with a client, but on
the client netstat -a shows an established connection:
I think the patch will resolve my problem.
As Martin Sustrik says, SO_KEEPALIVE is probably not what you want. You can
do heartbeating/keep-alive at the application level instead.
--- src/ip.cpp.orig 2012-01-22 15:17:01.0 +0400
+++ src/ip.cpp2012-01-22 15:21:21.0 +0400
@@ -49,6 +49,13 @@
if (s == retired_fd)
return retired_fd;
+// Turn keep-alive on for TCP
+if (protocol_ == IPPROTO_TCP) {
+ int optval = 1, optlen = sizeof(int);
+ int rc = setsockopt (s, SOL_SOCKET, SO_KEEPALIVE, optval, optlen);
+ errno_assert (rc != -1);
+}
+
// If there's no SOCK_CLOEXEC, let's try the second best option. Note
that
// race condition can cause socket not to be closed (if fork happens
// between socket creation and this point).
open_socket() is not the right place to put this. I'd probably add a
keepalive_socket() and call that in the same places from where
stream_engine_t calls unblock_socket().
-mato
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] List Guidelines and Project/Source modifications - Windows/MSVC
skal...@users.sourceforge.net said: On 22/01/2012, at 12:13 PM, Martin Sustrik wrote: On 19/01/12 17:05, john skaller wrote: This is seriously bugged. You may not use the name DLL_EXPORT. You must use a command line macro name specific to the library! For example ZMQ_DLL_EXPORT. You mean that if zmq.h is included in a header file of another library that happens to define DLL_EXPORT, zmq functions would be exported from that library as well, which is not desirable for everyone. Right? Yes, precisely. You need to use a name unique to your own library. The convention in MS land is something like BUILD_ZMQ but it doesn't matter as long as you have ZMQ in it :) DLL_EXPORT is probably my fault, I vaguely remember implementing the ZMQ_EXPORT stuff a long time ago. I *think* there was some reason why DLL_EXPORT got used instead of a ZMQ_ namespaced identifier. Anyhow, I don't really use Windows so I suggest that the users that care about this make it work and submit a patch, just be aware that any changes need to be tested to ensure all the other build platforms aren't broken. Oh, and while you're at it, the topic of being able to build a static lib on Windows comes up fairly regularly, so you might want to make ZMQ_STATIC work as well :-) -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] multi-part messages
Martin, sust...@250bpm.com said: I believe this discussion misses what I wanted to say. There are many issues mixed here: 1. Should 0MQ provide internal message parts? 2. Should 0MQ provide application-level message parts? 3. Expose the functionality as gather-scatter arrays? 4. Should the 0MQ-related parts of the protocol separate from application-level parts? Good summary! Answer to 1 is yes (because of way REQ/REP works). Answer to 2 is yes due to the popular demand. Answer to 3 is no, to preserve backward compatibility. 3) could be implemented as a new API, in the style of the Linux recvmmsg() and sendmmsg() calls. My point was that the answer for 4 should be yes and the fact that it is not so is unfortunate. Too late to fix that, anyway. I've tried with LABELs and failed. 0MQ is a stable product and bad design decisions are already baked in. Not much we can do about that. I did not follow the LABEL work, but IIUC the problem was you tried to satisfy point 4) above by breaking point 2), right? -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] multi-part messages
skal...@users.sourceforge.net said: I see no technical reason that you cannot *add* an array based interface: two extra functions. Sure, there is no technical reason, but it's a question of demand and (more importantly) API maintenance. Regarding demand, if enough people see the benefit in an array-based sendmmsg() / recvmmsg() interface, then such an interface can be added. Regarding API maintenance, we have the same problem as the Linux kernel; once an API is added, it will be used by applications, hence that API will have to stick around for ever (but see below). Having done that you could deprecate the old interface. In 4.x you can remove it from the core API. It can still be supported by a legacy API. After all the legacy API would simply populate the array and the call send function (and the reverse for recv). That's how it works in theory. In practice once you have a sizeable amount of users, you have to keep such legacy APIs around for a long time - deprecation becomes a slow process. Anyhow, in this particular case, IMHO, the sendmmsg () / recvmmsg () APIs can coexist nicely with the existing APIs and we *don't* want to deprecate ZMQ_MORE since it's the only way to write devices which can handle multi-part messages in a generic fashionb with the current model. Just as a matter of curiosity .. how much 0MQ use is from C? C++? Other languages? [The question means: is the C API primarily for application programming, or it there primarily to support bindings to better languages?] That's kind of like asking how many people still program in C. A lot :-) The libzmq C API serves 4 purposes: a) a lowest common denominator baseline b) the API that language binding developers use c) for those application programmers who are familiar with the POSIX APIs and do not need/want anything more abstract d) for those application programmers that need the performance and control of using it directly from C IMHO one of the reasons ZeroMQ is so successful is precisely because we decided early on to stick with a POSIX-like C API. There are a lot of developers out there who have some degree of familiarity with these APIs, so it makes the barriers to adoption (how quickly can I get something working) very low. -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Proposal for libzmq maintenance
On Tue, 10 Jan 2012 09:16:44 -0600 Pieter Hintjens p...@imatix.com wrote: On Tue, Jan 10, 2012 at 7:12 AM, Martin Sustrik sust...@250bpm.com wrote: +1 from myself I also second Mato's concern about fragmenting the discussion about individual patches. Care should be taken keep it in one place (ML, presumably) where anyone interested can follow it easily. Excellent, thanks. If we cross-post pull requests to the ML, discussion can happen either there on the ML or on the PR thread. People will converge on whatever works best. Nice idea; i'm curious to see how you implement it :-) The important (and probably hard) part is cross-posting the discussion threads from the PRs to the ML in a way that makes sense. -mato -- Martin Lucina mar...@lucina.net ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Proposal for libzmq maintenance
Hi Pieter, all, On Mon, 9 Jan 2012 16:58:54 -0600 Pieter Hintjens p...@imatix.com wrote: Hi all, This is a proposal and request for comments on onward maintenance of libzmq. Authors of this proposal are Pieter Hintjens and Mikko Koppanen. Our goals are: * To make it easier for anyone to become a core contributor to libzmq. * To remove the need for separate repositories for stable releases. * To make the libzmq repository structure simple and unsurprising. * To reduce the risk of human error or negligence. * To ensure there are no single points of failure, i.e. individuals whose absence would be catastrophic. This is great. +1 for coming back to the single-repository model. Some basic principles: * Anyone can become a contributor to libzmq by making a fork of the repository and making changes to their private copy. * We do not mix development and packaging in the same repository: there will no longer be direct changes to libzmq. * Experimental and raw work exists as unofficial repositories that we do not document nor attempt to manage. * Changes are merged from forked repos into libzmq as pull requests, after discussion and consensus. * A separate team is responsible for merging changes, enforcing process (e.g. issues and test cases) and making official releases. * One may be on both teams but not at the same time (i.e. one may not ship one's own work). * Authors of such repositories can apply any change process they like, privately. The organization of the libzmq repository will be simple and predictable. Master will hold the latest version and is assumed to always build though will usually be considered unstable. Releases each have their own branch (major/minor, e.g. 3-1, 3-2) which goes from unstable to stable, with tags for each formal release. So the official zeromq/libzmq repository master branch essentially becomes a staging tree for proposed updates to the next release, a.k.a. linux-next, pu or similar approaches used by other projects, right? The above, along with the no direct changes to libzmq means that libzmq/master should only see merge commits of pull requests. Right? As for how the release branches will work, I presume that is yet to be fleshed out in detail, but the commits there would also consist mostly of merge commits or cherry-pick, with the occasional direct commit to update e.g. NEWS, or zmq_version.h. To contribute a new feature, one forks libzmq, writes and tests the feature and makes a pull request. All pull requests would be automatically published to zeromq-dev for review and comment. When a feature gets consensus approval the maintainers merge the pull request onto master. From then on, it will be included in automatic builds and tested by early adopters. It would be good to see pull request explicity defined as an e-mail sent to the zeromq-dev mailing list by the repository owner, asking the maintainers to pull from git://.../some-branch. Further, pull requests created on Github but *not* also sent to the mailing list must be ignored by the maintainers, and this would be prominently documented. Rationale: This is needed to keep the process of accepting new changes into master transparent to all interested parties (maintainers, developers, users, lurkers, ...) and to avoid fragmenting the discussion or code review. This doesn't stop anyone from using Github or other Web 3.0 coolness to create pull requests, but it does ensure that everyone else is kept in the loop on any discussion while not being forced to use the same Web 3.0 coolness. To contribute a bug fix, one forks libzmq, writes and tests the bug fix, and makes a pull request to master. The maintainers check that there is a proper test case (for fixes to stable branches) and retest the fix after merging the pull request. The maintainers then merge that fix to the various branches it affects, and the bug fix is then included in automatic builds of that branch and will go into new packages. To make a new official package, the maintainers create a branch, where the branch evolves through to maturity and eventual shuttering. Comments and discussion welcome. Good stuff, thanks Pieter and Mikko for putting this together! -mato -- Martin Lucina mar...@lucina.net ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Proposal for libzmq maintenance
On Mon, 9 Jan 2012 23:20:01 -0600 Pieter Hintjens p...@imatix.com wrote: [snip] It would be good to see pull request explicity defined as an e-mail sent to the zeromq-dev mailing list by the repository owner, asking the maintainers to pull from git://.../some-branch. Automatically. There's no benefit in forcing this to be manual, and there are risks. People can also subscribe separately to pull request emails. Doing this automatically mandates that contributors must use Github for creating pull requests. That would be bad, and that is what I am arguing against. What specific risks do you see in sending an email manually? Further, pull requests created on Github but *not* also sent to the mailing list must be ignored by the maintainers, and this would be prominently documented. Having used pull requests for some time now, my preference is to have them announced and discussed unavoidably, and then deleted if they don't come up to scratch. Otherwise you get an accumulation of old pull requests. A pull request becomes stale rapidly, it has to be accepted or deleted within a few days depending on the rate of change. I think we're in agreement here, but see below. Rationale: This is needed to keep the process of accepting new changes into master transparent to all interested parties (maintainers, developers, users, lurkers, ...) and to avoid fragmenting the discussion or code review. Indeed. We'll hook up github to email the list automatically. IMO the same should eventually happen for new issues (not comments on issues). I would be wary of directing too much automated traffic to the main mailing list; JIRA is far too chatty to ask that everyone read those emails. What could be done for issues is creating a separate zeromq-bugs list that gets everything from JIRA for those who want to follow that. This doesn't stop anyone from using Github or other Web 3.0 coolness to create pull requests, but it does ensure that everyone else is kept in the loop on any discussion while not being forced to use the same Web 3.0 coolness. Personally, after trying all the options, I'm unwilling to apply patches by any other means except pull requests and quite willing to argue this one. The advantages so far outweigh the philosophical objections. You misunderstood what I wrote; I'm not arguing against using pull requests, I'm arguing against *mandating* the Github *interface* to pull requests. The Github stuff is just a wrapper that (AFAIK) creates a branch for each pull request. What I'm proposing is that a pull request is a *request sent by e-mail by the requester* to the zeromq-dev list, asking that the changes staged on branch xyz_feature of the repository found at git:// be pulled into libzmq master. That's all. From the POV of Github contributors, this means the additional step of sending an email with said text to zeromq-dev. For anyone else, that is what they would be doing anyway. From the POV of the maintainer processing the pull request, the process is identical in both cases - git pull git://host/repo/branch. If for some reason you want to be extra nice to Github users then by all means figure out a way to hook up sending notifications of Github-created pull requests to the mailing list but I think it's just extra work with no real benefit, plus you run the risk of fragmenting discussion about the content of the pull request since people *will* comment using the Github interface, and we don't want that. Am I making sense now? -mato -- Martin Lucina mar...@lucina.net ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] [PATCH] Fix pgm_receiver.cpp: zmq_assert (pending_bytes == 0) (LIBZMQ-205)
This patch fixes the problem described in LIBZMQ-205. The assertion itself
is probably caused by previously queued POLLIN events arriving after POLLIN
has been disabled on the socket.
The following additional bugs have been fixed as part of debugging this
problem:
- pgm_receiver_t does not flush messages written to the session in all
cases which can lead to a stalled reader. Add calls to session-flush ()
in the appropriate places.
- ensure to restart polling when a pending message is flushed in
activate_in ().
Signed-off-by: Martin Lucina mar...@lucina.net
---
src/pgm_receiver.cpp | 13 +++--
1 files changed, 11 insertions(+), 2 deletions(-)
diff --git a/src/pgm_receiver.cpp b/src/pgm_receiver.cpp
index 122d110..99e882b 100644
--- a/src/pgm_receiver.cpp
+++ b/src/pgm_receiver.cpp
@@ -117,8 +117,15 @@ void zmq::pgm_receiver_t::activate_in ()
// processed the whole buffer but failed to write
// the last message into the pipe.
if (pending_bytes == 0) {
-if (mru_decoder != NULL)
+if (mru_decoder != NULL) {
mru_decoder-process_buffer (NULL, 0);
+session-flush ();
+}
+
+// Resume polling.
+set_pollin (pipe_handle);
+set_pollin (socket_handle);
+
return;
}
@@ -128,6 +135,7 @@ void zmq::pgm_receiver_t::activate_in ()
// Ask the decoder to process remaining data.
size_t n = mru_decoder-process_buffer (pending_ptr, pending_bytes);
pending_bytes -= n;
+session-flush ();
if (pending_bytes 0)
return;
@@ -145,7 +153,8 @@ void zmq::pgm_receiver_t::in_event ()
unsigned char *data = NULL;
const pgm_tsi_t *tsi = NULL;
-zmq_assert (pending_bytes == 0);
+if (pending_bytes 0)
+return;
if (has_rx_timer) {
cancel_timer (rx_timer_id);
--
1.7.2.5
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Too many open files (signaler.cpp:330)
On Wed, 04 Jan 2012 22:29:46 +0100 Martin Sustrik sust...@250bpm.com wrote: On 04/01/12 14:55, Chuck Remes wrote: What elements should I printf? Is the +sv+ variable important? What values should it have (or not have)? I believe there's nothing to print there except of fact that a socketpair was created. Is there a way to print the total number of FDs in use at that point in time? That is, can I add a counter ivar to the class and increment it in that function? I am not aware of such a function. However, it's possible there's a tool on OSX that provides you with the number interactively. For the total number of fds in use in the system, you could try something like 'lsof'. If you just need the total number used in that particular process, then fds are just integers numbered from 0 up, so printing whatever is the last one you get from socket() or similar will tell you how many that process is using up till now. -mato -- Martin Lucina mar...@lucina.net ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Too many open files (signaler.cpp:330)
For the total number of fds in use in the system, you could try something like 'lsof'. If you just need the total number used in that particular process, then fds are just integers numbered from 0 up, so printing whatever is the last one you get from socket() or similar will tell you how many that process is using up till now. Actually, the latter won't work since fds are re-used within a process, i.e. 0 = open (...) 1 = open (...) 2 = open (...) close (0) 0 = open (...) So 'lsof' is your best bet. -mato -- Martin Lucina mar...@lucina.net ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] Pending patches for merging to libzmq master
Martin, I have fixes ready for two issues in PGM with libzmq master, LIBZMQ-303 and LIBZMQ-205. Both patches have been sent to the ML for inclusion, but have not yet been merged. When can I expect these to be merged into libzmq master? Without this PGM is more or less unusable in 3.1. Thanks, -mato -- Martin Lucina mar...@lucina.net ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Pending patches for merging to libzmq master
On Thu, 05 Jan 2012 13:30:51 +0900 Martin Sustrik sust...@250bpm.com wrote: There's no separate repo for 3-1 yet. Pieter is going to create it shortly as far as I am aware. As for the master, the system of merging experimental work into a single repo have proved extremely painful in the past (recall 0MQ/4.0). I would strongly recommend keeping experimental work on topic branches. This is not experimental work but fixes for the current libzmq master. If fixes are not going to be merged into libzmq master from now on, then all I can do is keep them in my own staging repository. -mato -- Martin Lucina mar...@lucina.net ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] [PATCH] Fix assertion in pgm_sender_t::plug() (LIBZMQ-303)
Opening any PGM socket gives this assertion. The problem is in pgm_sender_t::plug() which is incorrectly testing the return value from session::write(). Signed-off-by: Martin Lucina mar...@lucina.net --- src/pgm_sender.cpp |4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/src/pgm_sender.cpp b/src/pgm_sender.cpp index 759802f..1a1ae9a 100644 --- a/src/pgm_sender.cpp +++ b/src/pgm_sender.cpp @@ -98,8 +98,8 @@ void zmq::pgm_sender_t::plug (io_thread_t *io_thread_, session_base_t *session_) msg_t msg; msg.init_size (1); *(unsigned char*) msg.data () = 1; -bool ok = session_-write (msg); -zmq_assert (ok); +int rc = session_-write (msg); +errno_assert (rc == 0); session_-flush (); } -- 1.7.2.5 ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] [ANNOUNCE] ZeroMQ/3.1.0 (beta) is now available
Hi all, Martin Sustrik and myself are pleased to announce a new beta release of ZeroMQ, version 3.1.0. Packages are now available for download from http://www.zeromq.org/intro:get-the-software. This release is based off the libzmq master codebase. As agreed by the community, development of the 0MQ 3.0.x series will be discontinued. All users are encouraged to upgrade to 3.1. Please refer to the attached release notes for a list of major changes in this release. Thanks to everyone involved for reporting bugs and helping test the new release! Regards, -- Martin Lucina Lucina Associates OMQ version 3.1.0 (beta), released on 2011/12/18 General information --- Based on community consensus, the 0MQ 3.1.x release reverts a number of features introduced in version 3.0. The major reason for these changes is improving backward compatibility with 0MQ 2.1.x. Development of the 0MQ 3.0.x series will be discontinued, and users are encouraged to upgrade to 3.1. The 0MQ 3.1.x releases use ABI version 3. Reverted functionality -- The following functionality present in 0MQ 3.0 has been reverted: * Wire format changes. The 0MQ 3.1 wire format is identical to that of 0MQ 2.1. * LABELs and COMMANDs have been removed. * Explicit identies are re-introduced, however they can be used only for explicit routing, not for durable sockets. * The ZMQ_ROUTER and ZMQ_DEALER socket types are once again aliases for ZMQ_XREQ and ZMQ_XREP. New functionality - * The zmq_getmsgopt() function has been introduced. * Experimental IPv6 support has been introduced. This is disabled by default, see the zmq_setsockopt() documentation for enabling it. Other changes - * The default HWM for all socket types has been set to 1000. * Many bug fixes. Building * The dependency on libuuid has been removed. * Support for building on Android, and with MSVC 10 has been added. signature.asc Description: Digital signature ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] api.zeromq.org rebuild for v3.1
Pieter, could you please add the v3.1 manpages to api.zeromq.org? (The instructions on the release page say rebuild that site) Thanks! -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Contributing native InfiniBand/RDMA support to 0MQ
Hi Gabriele, gabriele.sve...@gmail.com said: [...] I shared your confusion when I started dealing with these technologies because they are not very well documented and some terms are often used interchangeably. By RDMA-enabled technologies I mean any kind of technology that implements the functionality provided by InfiniBand verbs. The InfiniBand verbs library supports a number of operations which can be roughly divided in four categories and this is what usually causes the confusion regarding the term: Thanks for summarizing this. I also found the following paper useful as a step-by-step walkthrough of the ibverbs APIs: http://arxiv.org/abs/1105.1827 - Send/receive operations, these work pretty much like datagrams except that you can do many of them in parallel and the hardware will deal with them without the need of intervention from the CPU. Both endpoints must be aware of the operations so the code will look very similar to what you do with sockets (one side sends, the other receives) - these are optimal for small messages and it is what I am going to use for 0MQ Makes sense. Later we could look at using RDMA read/write for large messages. - RDMA read/write operations, with these you can read/write from the memory of a remote machine without intervention from the machine itself, think of it as a memcpy() that takes a network address. When you do such an operation the other side is oblivious to what is going on - Atomic operations, this allow you to do things like atomic compare-and-swap operations inside the memory of a remote machine, these are used for implementing distributed locks, queues, barriers, etc... Again you can do them on a remote machine with the other end being oblivious to what is going on - Collective operations, these are the fanciest latest additions that allow the cards to implement 1-to-N, N-to-1 and even N-to-N MPI operations such as gather without intervention from the hosts, everything happens on the cards and switches Neat, the last one is basically PUB/SUB in hardware. I could do that however the last time I tested SDP I was not very satisfied with its performance (albeit it's been a long time ago) and it hasn't got much love from the open source community. It is available only through OFED AFAIK and many distribution do not ship it by default, for example both CentOS 6 and recent OpenSUSE releases have dropped it. Most major distributions on the other hand ship both ibverbs and the rdmacm libraries making them readily available to users. You're right about SDP; I had a look at the state of distribution and kernel.org support and it does seem that ibverbs is the way to go. Also, AFAICT SDP is a kernel-space implementation so you don't get the possible benefits of a kernel bypass. Martin Sustrik and myself have a small test lab setup, so I will try and get ibverbs working there; have a couple of machines connected back to back with Mellanox ConnectX adapters. So far I've managed to get OpenSM, ibping, IPoIB working, but ibverbs is resisting (cannot find userspace driver... blah blah). -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Contributing native InfiniBand/RDMA support to 0MQ
Gabriele, gabriele.sve...@gmail.com said: I will, looking at all the responses it seems that there's more interest for it than I had predicted :-) From my investigation of IB/RDMA it looks like the APIs are typical enterprise stuff. Big and complex. The benefit of having a single API (ZeroMQ) which you could use for all applications would be huge. I've already added a --enable-rdma option in my local repository plus the necessary checks and it's looking good. Would --with-rdma be preferred? In autoconf it's usually used for options where you need to specify something (e.g. --with-stuff=mystuff) while --enable is used for on/off flags. --enable-rdma is fine, I tend to confuse --with-stuff and --enable-stuff. Of course you might want --with-rdma=prefix for people who have things installed outside of the standard places. That sounds good, I'll create a branch on GitHub ASAP though I have to go through their docs first as I have never used the service before. You can also use your own Git hosting and/or send patches and pull requests by email if you'd prefer that, there's no requirement to use Github for contributing. -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Contributing native InfiniBand/RDMA support to 0MQ
michael.sa...@dynetics.com said: Btw, do you have any performance benchmark that compare using IB verbs vs. simply using SDP? I don't have anything offhand. I'll have to run some benchmarks. While I'm at it, I'll try to benchmark 0MQ on a QDR IB network. ... or vs. IPoIB. See the other discussion in this thread, it seems SDP is not really used much, and since it's an in-kernel implementation I don't see much advantage over just using IPoIB. -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] home page update irc
cremes.devl...@mac.com said: Could someone with the authority please update the zeromq.org web page so that on the Learn ZeroMQ page we have the FAQ listed underneath the Guide and Man pages? The FAQ, being frequently asked by definition, is buried on the Solve a Problem page, so we're getting a lot of questions in IRC about stuff covered in the FAQ now. Done. Also, it would be great if the FAQ could be linked in the IRC channel topic too. Done. I dropped the bit about the channel logs from the topic since it didn't seem that important. I'm not sure how to create a redirect on zero.mq, so I used www.zeromq.org/area:faq instead of zero.mq/faq... Pieter? Thank you. Thank you for your work on the FAQ! Cheers, -mato signature.asc Description: Digital signature ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] home page update irc
pelletier.mic...@gmail.com said: Announcing channel logging in the topic is suggested by freenode's channel guidelines: http://freenode.net/channel_guidelines.shtml If you're publishing logs on an ongoing basis, your channel topic should reflect that fact. Fair enough. Fixed. Just thinking, maybe there's a way to get ChanServ to message people with the Read the FAQ bit when they first join the channel, just to drive the point home. -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] [PATCH] Fix synchronous connect failure for ipc://, tcp:// (LIBZMQ-294)
A synchronous connect() failure in ipc_connecter can result in Assertion failed: s == retired_fd (ipc_connecter.cpp:174), as reported in LIBZMQ-294. This patch fixes the bug, and also an identical problem in tcp_connecter which has not hit people since TCP connect() usually completes via the asynchronous code path (poll, out_event). Signed-off-by: Martin Lucina mar...@lucina.net --- src/ipc_connecter.cpp |1 + src/tcp_connecter.cpp |1 + 2 files changed, 2 insertions(+), 0 deletions(-) diff --git a/src/ipc_connecter.cpp b/src/ipc_connecter.cpp index dc0ee21..58dccf4 100644 --- a/src/ipc_connecter.cpp +++ b/src/ipc_connecter.cpp @@ -135,6 +135,7 @@ void zmq::ipc_connecter_t::start_connecting () } // Handle any other error condition by eventual reconnect. +close (); wait = true; add_reconnect_timer(); } diff --git a/src/tcp_connecter.cpp b/src/tcp_connecter.cpp index 75079da..042e82a 100644 --- a/src/tcp_connecter.cpp +++ b/src/tcp_connecter.cpp @@ -146,6 +146,7 @@ void zmq::tcp_connecter_t::start_connecting () } // Handle any other error condition by eventual reconnect. +close (); wait = true; add_reconnect_timer(); } -- 1.7.2.5 signature.asc Description: Digital signature ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] Cross-compiling and embedded builds (was Re: Cross-compiling ZeroMQ to ARM for use in a MonoTouch iPhone app configure settings)
Martin, Mikko, others, it looks like these questions are coming up more and more; what I'd like to do is start a more formal Wiki page about cross-compiling and embedded systems, and also a FAQ section. The idea is that this would contain generic instructions and answers to common problems, and the instructions themselves could later go (in an abridged version) also into the package README/INSTALL files. This should answer 80% of people outright, and give the rest a place to document their efforts, although I would like to avoid too much of a proliferation of Building zeromq for $FOO_PLATFORM with $BAR_VENDOR SDK, since the embedded world is a mess and the amount of combinations of various broken SDKs causing problems is thus infinite :-) What do you think? Cheers, -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] x64 build target for Windows?
sust...@250bpm.com said: On 11/11/2011 10:43 AM, Mikko Koppanen wrote: Looking at http://code.google.com/p/encmq/source/browse/trunk/src/zeromq/CMakeLists.txt it's under GPL license. Is this problematic for us? Maybe we could start from Steven's CMakeLists and expand from there. I would say the license of the build tool doesn't affect the license of the built product. Technically true, but things rapidly get confusing for distributors (Debian, etc.) if files in a single package are covered by multiple licenses. Therefore, to avoid any such problems we should not mix licenses in libzmq, i.e. stay with the current status where the contents of the core libzmq package are licensed in their entirety[*] under the same terms (LGPL v3 + static linking exception). [*] with the exception of the included OpenPGM library, since this is included verbatim and not used by distributors anyway) The other issue with CMake is obviously the extra maintenance involved. We can see that there have been a couple of mostly one-off efforts to provide a CMake build but until someone steps up to actually maintain such a build there's no point in making it official. -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Mission accomplished
sust...@250bpm.com said: http://www.250bpm.com/blog:1 Martin, this is great! And it brings back memories. Baroque cellar indeed :-) Finally, we have a narrative of the last 5 years. Here's to the next 5 years and a big thank you to everyone involved. -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] [PATCH] Set libzmq ABI version to 3
Martin, libzmq master is still using ABI v1 which is obviously incorrect. This patch sets it to v3 which is the safest choice since functionality has been removed versus ZeroMQ 3.0 which uses ABI v2. Please apply. Thanks, -mato From 991f7e2c85919daef43c62175da046e0a085f8e3 Mon Sep 17 00:00:00 2001 From: Martin Lucina mar...@lucina.net Date: Wed, 9 Nov 2011 13:12:46 +0100 Subject: [PATCH] Set libzmq ABI version to 3 libzmq master (3.1) is not ABI compatible with libzmq 2.1.x or 3.0 (removed functionality), hence the ABI version needs to be set to 3. Signed-off-by: Martin Lucina mar...@lucina.net --- configure.in |8 +--- 1 files changed, 5 insertions(+), 3 deletions(-) diff --git a/configure.in b/configure.in index 86147ea..9006f03 100644 --- a/configure.in +++ b/configure.in @@ -25,11 +25,13 @@ AC_SUBST(PACKAGE_VERSION) # # Changes: # -# ZeroMQ versions prior to 2.1.0 use 0.0.0 (unstable) -# ZeroMQ version 2.1.0: 1:0:0 +# ZeroMQ versions prior to 2.1.0 use 0:0:0 (undefined) +# ZeroMQ versions 2.1.x: 1:0:0 (ABI version 1) +# ZeroMQ version 3.0: 2:0:0 (ABI version 2) +# ZeroMQ version 3.1: 3:0:0 (ABI version 3) # # libzmq -version-info current:revision:age -LTVER=1:0:0 +LTVER=3:0:0 AC_SUBST(LTVER) # Take a copy of original flags -- 1.7.2.5 ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] C++ high level binding
ilej...@narod.ru said: Not sure I've convinced you that zmqmessage is a right thing for C++, but I hope that I managed to give some reasons behind this design. I think it's great that we have another more/different C++-like alternative to the very lightweight zmqcpp. I also prefer libraries to do one bit and do it well, Pieter is more a one-hammer-for-everything kind of guy, sometimes. Regarding SO_LINGER/blocking zmq_term(), it is a matter of opinion. Technically, your application will only block forever on zmq_term() if you have outstanding messages which cannot be sent anywhere. As for a better name for zmqmessage, how about zmqcppplus :-) -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Copyrights in the source files
Martin, sust...@250bpm.com said: Hi all, I have some code paid for by a company that I want to add to 0MQ. However, at the moment we have no process to attribute copyrights to companies as opposed to individual contributors. Let's do it this way: If you submit a patch you (as a person) are going to be listed in AUTHORS file the same way you are today. Aside of that, feel free to add company copyrights to the files you've modified and make it a part of the patch. If you believe the historical patches you've made should be attributed to a company, let me know the patch ID and the company name and I'll add the copyrights by hand. Why the distinction between individual authors and corporate entities? That seems unfair. The way I've seen it done in other projects is: - If you create a new source file, you add the requisite license header and copyright (whether your own, or that of a company) - When you submit a patch to an existing source file, if you (the submitter) perceive that you have modified a substantial portion of that file, or if you simply care enough about having your copyright listed, you add in your copyright statement to the affected file. Again, no distinction between individual and company. This does not in any way change the policy of adding all authors to the AUTHORS file. What do you think? -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Copyrights in the source files
sust...@250bpm.com said: On 10/31/2011 12:26 PM, Martin Lucina wrote: The way I've seen it done in other projects is: - If you create a new source file, you add the requisite license header and copyright (whether your own, or that of a company) - When you submit a patch to an existing source file, if you (the submitter) perceive that you have modified a substantial portion of that file, or if you simply care enough about having your copyright listed, you add in your copyright statement to the affected file. Again, no distinction between individual and company. This does not in any way change the policy of adding all authors to the AUTHORS file. What do you think? No preference either way. However, if the copyrights are listed in the source files what is AUTHORS file good for? Well, as I wrote above, you only put the (c) in if you care enough, and if you have created the file in question/modified a substantial portion of an existing file. Fuzzy, I know. So the AUTHORS file is there to credit every contributor equally. -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] 0MQ/3.0, 0MQ/4.0 and 0MQ process
sust...@250bpm.com said: As for the rollback of 4.0, are there commits in 4.0 that are not in 3.0 that would want to be kept in the new 3.0-tracking master? I'm just confused as to why the reverting would happen if the plan is simply to make 3.0 master, and 4.0 'experimental', when you can just rename libzmq/master to libzmq/experimental, and push current zeromq3-0/master as libzmq/master. How would that solve the problem of separating the stable patchsets from experimental patchsets? Whatever way you take you have to manually inspect all the patches and move them either onto stable branch or on individual topic branches. I am not against better way of doing this -- it would save me large amount of ultimately boring unpaid work -- however, I don't see any other way out at the moment. I'm not sure that renaming the libzmq/master branch is a good idea, what would that do in practice to people tracking that branch in their clones of the repository? -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] ZMQ for transit project, and debian package
mich...@frumin.net said: All, Just an FYI, that ZeroMQ has been selected as the messaging bus for the back end of the NYC-wide expansion of http://bustime.mta.info/ (http://bustime.mta.info/wiki/Technology), which is going to be completely open source. It would have been fun to use it for the bus-to-server over-the-air comms, but alas. Cool! One question I have though -- is there any sort of debian precompiled ZMQ package? If not, curious, why not? The contractor is developing some pretty rigorously scripted deployment for the whole system (using chef, onto AWS) and at this point the slowest step is the compiling of ZMQ on every machine... I maintain the official Debian package, and 2.1.x is in Debian testing/unstable, see http://packages.debian.org/search?keywords=libzmq1 3.0.x will be packaged once it's stable/when enough people ask for it. AFAIK the Ubuntu package uses the Debian one as its upstream, but I'm not sure how they do things these days. Cheers, -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] [ANNOUNCE] ZeroMQ/2.1.8 is now available
p...@imatix.com said: Changes --- * Removed Debian packaging, which is now available at packages.debian.org or via apt-get. To avoid any confusion, I'd just like to point out that the built-in Debian packaging was never actually distributed in the release tarballs, it only ever existed in Git. So this doesn't actually change anything for users using release tarballs. -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Leaving my role as maintainer
p...@imatix.com said: On Mon, Jul 11, 2011 at 6:04 PM, Martin Lucina m...@kotelna.sk wrote: Not sure what you mean by update. As it stands that packaging is maintained externally as part of the official Debian package. What would need to be done is replace the current stuff in Git with the official packaging, but modified slightly to be usable for a build me a snapshot package of this particular Git checkout type workflow. Is that what you want? Will anyone actually use this? I guess the larger question is where will the Debian packaging be maintained. If it's not in the 0MQ distribution, that directory should be deleted. Otherwise, it should be reflect the current (real) Debian packaging, just so that future contributors know where to find it. Then there's the option of actually distributing DEBs from zeromq.org, but I don't really want to get into that ... No-one wants that IMO. People have used the debian packaging to make debs for private distribution, e.g. for networks where software may only be installed as debs. In that case let's delete it. Anyone who is going to be building custom DEBs for internal use already knows to get the source package using either apt-get source zeromq, or downloading it manually from packages.debian.org. Martin, Pieter, can you please delete the debian/ directory from the various Git repos? No point submitting a patch for this... Cheers, -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] IPv6 support (was Re: Leaving my role as maintainer)
sust...@250bpm.com said: Btw, one final question. You've explained to me how to design IPv6 support so that it plays well with dual-stack boxes. Of course, I forgot what you've said in the meantime, but it seemed to make sense back then. Can you outline how it should work in few sentences? Hmm, I thought I wrote about this some time ago, but maybe not. Here's my view on how things should work, anyone else with practical IPv6 experience please comment/correct. It's been a while since I looked at this so the actual APIs I talk about might not be entirely correct. Also, the asynchronous nature of connecting 0MQ sockets might pose problems I haven't anticipated. Listening (bind) side - 1) If the user has specified an endpoint of '*', INADDR_ANY/AF_UNSPEC(?) should be used to create a dual-stack listening socket. This is the simplest case. 2) If the user has specified an endpoint using an interface name, e.g. 'eth0', 0MQ must determine if the interface has an IPv6 or IPv4 address, and bind two separate listening sockets, one to each address family. 3) If the user specifies a numeric IPv4 or IPv6 address then only that address shall be used (i.e. existing behaviour with the addition of IPv6). 4) For PGM endpoints, I don't know. To be discussed with Steve. Connecting (connect) side - 1) If the user has specified a host name, and the local machine has IPv6 enabled at all (I forget exactly how to detect that, but it's part of the getaddrinfo() stuff), then 0MQ must first attempt to resolve an AF_INET6 address for the hostname. 2) If the resolution fails, skip this step. Else, attempt to connect to the IPv6 address. If that succeeds then we are done. 2a) Not sure how well this is possible with the current codebase, but if the connection to the IPv6 address fails using a hard error that could be attributed to faulty IPv6 connectivity, then we should fall back to IPv4. Such an error could be e.g. No route to host or even Connection refused. 3) Attempt to resolve an AF_INET address for the host. If that succeeds, connect to it (and assume IPv6 to that endpoint is not functional). If that fails, then fail. 4) For numeric addresses nothing changes, other than the addition of IPv6 addressing (i.e. no fallback). As you can see from the above, the general idea is always to bind to all available address families, and then, provided the box claims to have IPv6, attempt to resolve/connect to IPv6 first, falling back to IPv4 if that doesn't work. Step 2a) is the complex part for us, since most normal TCP applications connect synchronously, and for UDP/PGM it's pratically impossible to tell if there's anyone at the other end so I'm not sure what to do there (possibly no fallback at all, since that's really only interesting in the simple case of users using TCP.) Hope this helps. -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Leaving my role as maintainer
p...@imatix.com said: On Tue, Jul 12, 2011 at 3:03 PM, Martin Lucina m...@kotelna.sk wrote: In that case let's delete it. Anyone who is going to be building custom DEBs for internal use already knows to get the source package using either apt-get source zeromq, or downloading it manually from packages.debian.org. Happy to delete it, can you confirm that http://packages.debian.org/source/sid/zeromq is the correct resource? That's whichever version is in Debian unstable, hence the newest package. So if that's what you want then yes, it is. -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] Leaving my role as maintainer
Hi all, due to lack of funding or any significant revenue that would offset the costs I can no longer justify spending time as maintainer of the build system or native API reference manual. The build system is in the good hands of Mikko Koppanen; as for the API reference, it would be great if someone from the user community would step up to work with Martin Sustrik on reviewing and improving that documentation. As the canonical documentation of the API it's a critical part of the project and I'm sure that Martin would appreciate the help. I will continue to maintain the Debian package for 2.1.x for the time being since that does not take up too much time. Martin/Pieter, note that this is really independent of the Debian packaging that is currently in the debian/ directory in Git, this is now quite out of date and should probably be removed or at least replaced with what is being used in the official package. Cheers, -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Leaving my role as maintainer
p...@imatix.com said: On Mon, Jul 11, 2011 at 4:41 PM, Martin Lucina m...@kotelna.sk wrote: due to lack of funding or any significant revenue that would offset the costs I can no longer justify spending time as maintainer of the build system or native API reference manual. This is a loss to the 0MQ community. I'm sure everyone will agree that you've always made impeccably accurate contributions. Good luck with whatever projects you work on. Thanks. I do hope to work with 0MQ in future, but the things I'm doing right now are pretty unrelated :-( The build system is in the good hands of Mikko Koppanen; as for the API reference, it would be great if someone from the user community would step up to work with Martin Sustrik on reviewing and improving that documentation. As the canonical documentation of the API it's a critical part of the project and I'm sure that Martin would appreciate the help. I'm happy to help with editing and review. However it'd be most fun to open this to the community and suggest that it's time for any 0MQ user who feels they are good technical writers to consider sending patches for the man pages. Note that documentation for 2.1 and 3.0 overlaps in most areas but you do need to target the two versions separately. The manual has always been open to contributions and the occasional patch has turned up. A couple of people did offer to help but that never transpired into actual contributions. I think the lack of contributions is more about good technical writing being hard work than any other factors. Also, writing a reference manual is a different kind of effort/less fun than e.g. what you're doing with The Guide; at least the way I tried to do things was to refrain from prose and opinions as much as possible and stick to spec-like facts. I got this from the POSIX manuals; very dry but accurate and to the point which is IMO what you want from a reference. Just my 2c. I will continue to maintain the Debian package for 2.1.x for the time being since that does not take up too much time. Martin/Pieter, note that this is really independent of the Debian packaging that is currently in the debian/ directory in Git, this is now quite out of date and should probably be removed or at least replaced with what is being used in the official package. Would you send a patch to update that? Not sure what you mean by update. As it stands that packaging is maintained externally as part of the official Debian package. What would need to be done is replace the current stuff in Git with the official packaging, but modified slightly to be usable for a build me a snapshot package of this particular Git checkout type workflow. Is that what you want? Will anyone actually use this? Then there's the option of actually distributing DEBs from zeromq.org, but I don't really want to get into that ... -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] [PATCH] License text in autogen.sh fixed
sust...@250bpm.com said: Nope. It was a overlooked leftover from the past. I would say the build system should be either GPL or LGPL. Keeping one file GPL'd doesn't make much sense. Mixing licenses in a single package would be a bad idea unless absolutely necessary. It creates confusion for distributors, among other things. -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Slight problem with make check
sust...@250bpm.com said: Hi Mikko, Mato, It seems that if libzmq git directory is on path that contains space character, make check fails. There's probably any number of things that will fail in the build system if a path component has whitespace in it. Ideas? Don't do that. -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] [ANNOUNCE] libzmq issue tracking migrated to JIRA
steven.mc...@miru.hk said: Comments are still messed up in JIRA, and is there anyway I can actually get an account tied to my existing cases? I'm listed as steve-o but cannot create an account with that name. This is now fixed, finally. I've associated the imported steve-o with your email address, you should be able to reset your password and log in, etc. Anyone else who needs this done please e-mail me privately with your githuv login and email address. -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] [PATCH] Improved response to socket violations
sust...@250bpm.com said: On 05/23/2011 03:22 AM, Ian Barber wrote: On Fri, May 20, 2011 at 4:50 PM, Martin Lucina m...@kotelna.sk mailto:m...@kotelna.sk wrote: 3) Last but not least, IMHO libraries should *not* print helpful messages. This leads to horrible practices, for example start a random Gtk application; you will more often than not see all sorts of assertion failures and other crap printed and it's obvious that no one cares, much less does anything about it. I have to say I am a big +1 on not ever printing from libzmq - to be honest even in cases where it completely blows up, I've worked with bindings for a library that does that and it's can make it very hard to have consistent responses to failures when sometimes extra output starts occurring. Are you guys saying you want silent failures, ie. application asserts without even printing out the error? If so, I guess it can be done as a compile time option. I'm saying keep the library as silent as possible. Fatal assertions are obviously fine and should be printed, but as I wrote in another thread we should work towards making libzmq as robust as possible; i.e. in an ideal future version you'd only get a fatal assertion if you hit an actual internal bug. Everything else would be reported back to the application somehow (either API calls or sys://log or a similar tool for asynchronous events). Just my 2c. -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Should zeromq handle OOM? (Re: [PATCH] Fixed OOM handling while writing to a pipe)
p...@colomiets.name said: On Sat, May 21, 2011 at 2:31 AM, Pieter Hintjens p...@imatix.com wrote: I think what you need to do is make a test bench that proves the case. Perhaps an EC2 instance with low virtual memory, or somesuch. Something people can play with. The problem is that fix all failures will take big amount of time and it would be quite huge refactoring (minor in terms of functinality, but huge in numer of lines of code), which whould take a time and would be hard to merge afterwards. Something realistic I can try to hack is: 1. turn off disconnecting on OOM (so that new connection would not crash application) 2. fix everything about sending/receiving messages 3. setup a two zmq_streamer-like devices (old code and new code) opened to the exernal world, so that anybody can test, set small ulimit -a for both 4. setup a slow consumer which also shows statistics from both devices Will having this example working convince you? +1 Paul, I like what you're doing and I think we agree on the general approach. As for Pieter's philosophical arguments, ultimately the person you need to convince is not Pieter via philosophical arguments but Martin Sustrik by sumbitting a patch :-) Cheers, -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Can't enter new entries in Jira
Hi, jdmitch...@gmail.com said: Howdy, tl/dr: I have a new jira account and I can login and see issues but there's literally nothing in the UI allowing me to enter new bugs. Longer: I created an account inn the new jira a couple of days ago (johnm). At the time, I could login but not see anything. :-( Now, that's fixed so when I log in I can search and browse the issues in the libzmq project (which seems to be the only one of the projects there atm)... But there's nowhere that I can actually enter new bugs. My jira profile shows that I'm in the users group (so I'm guessing that that group is lacking permission to enter bugs but that's just a guess). Can someone with jira admin tweak this please? If I understand the setup correctly, anyone with a Project Role of Users should be able to create issues (but not move/assign/otherwise edit them). Can you double-check that you don't have a Create new issue link at the top right next to the search box? Cheers, -mato ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
