Re: [zeromq-dev] ZeroMQ (and clrzmq) on Windows Mobile
On Thu, Feb 16, 2012 at 11:39 AM, Boris Gulay bo...@boressoft.ru wrote: 27.01.2012 19:30, Pieter Hintjens пишет: See http://www.zeromq.org/docs:contributing for submitting the patches. You can attach comments to a github pull request, or use this list to discuss your changes. Both will work fine. Whet repo should I use to submit a patch? zeromq2-1 is OK? I have made my port to WM on ZeroMQ 2.1 source. Hi, please submit the patch againts libzmq repo. 2.1 is closed for new features and receives only bug fixes. Thanks, Mikko ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] Compile branch with MSVC
I've just downloaded latest branch (from libzmq repo on GitHub) and try to compile it for Windows with MSVC 2008. Compilation fails because if inet_ntop function which is available only on Vista and later but windows.hpp declares _WIN32_WINNT as 0x0501 which means Windows XP ( http://msdn.microsoft.com/en-us/library/aa383745(v=vs.85).aspx ). Any advice? signature.asc Description: OpenPGP digital signature ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] ZeroMQ (and clrzmq) on Windows Mobile
On Thu, Feb 16, 2012 at 5:39 AM, Boris Gulay bo...@boressoft.ru wrote: Whet repo should I use to submit a patch? zeromq2-1 is OK? I have made my port to WM on ZeroMQ 2.1 source. Like Mikko said, the right place is libzmq (3.1). You may also want to point people to your 2.1 fork, if they want this port. -Pieter ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] zeromq2-1 pull request 39
Any chance of pull request 39 for the zeromq2-1 repo getting applied? Without them, 2-1 won't build on HPUX Thank you, -- AJ Lewis Software Engineer Quantum Corporation Work:651 688-4346 -- The information contained in this transmission may be confidential. Any disclosure, copying, or further distribution of confidential information is not permitted unless such privilege is explicitly granted in writing by Quantum. Quantum reserves the right to have electronic communications, including email and attachments, sent across its networks filtered through anti virus and spam software programs and retain such messages in order to comply with applicable data security and retention requirements. Quantum is not responsible for the proper and complete transmission of the substance of this communication or for any delay in its receipt. ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] zeromq2-1 pull request 39
On Thu, Feb 16, 2012 at 11:38 AM, AJ Lewis aj.le...@quantum.com wrote: Any chance of pull request 39 for the zeromq2-1 repo getting applied? Without them, 2-1 won't build on HPUX Sorry for missing that. I've pulled it into zeromq2-1 master, and the patch will go into 2.1.12. Thanks Pieter ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] inet_ntop is not available in Windows XP
Hi guys, In short: inet_ntop is not available in Windows XP. I'm maintaining a side project, where I have luajit ffi bindings for zeromq, and support them for osx, windows, linux, etc. One of my platforms is Windows XP, and I use Windows DDK, now called just WDK, so I can target MSVCRT.DLL (mingw for example also targets it). Anyway, a week or more ago a change has been introduced that relies on inet_ntop (src/tcp_listener.tcp), and that function is not available on Windows XP. For now I can sync to earlier version that that one, and keep going (I'm still learning zeromq, so it's not critical for me). But there is alternative for inet_ntop (I quickly googled), and here it is: http://stackoverflow.com/questions/1561469/is-there-an-alternative-to-inet-ntop-inetntop-in-windows-xp I'm just not experienced enough with zeromq's code base to provide a patch, hence I don't ever expect this to get fixed, but thought that other people might be affected. (Then again who would used Windows XP still :)) Thanks, Dimiter 'malkia' Stanev. ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] inet_ntop is not available in Windows XP
On 17/02/2012, at 5:43 AM, Dimiter 'malkia' Stanev wrote: other people might be affected. (Then again who would used Windows XP still :)) I do, for my yacht's backup chart plotter (because the software doesn't run on Vista). -- john skaller skal...@users.sourceforge.net ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] Error codes in Windows (ZMQ_HAUSNUMERO)
Hi guys, I have a question about ZMQ_HAUSNUMERO in zmq.h. It looks like this is a workaround the Windows headers, so that proper Exxx error codes are declared. I'm just having hard-time translating this correctly into a binding. For example: #ifndef EPROTONOSUPPORT #define EPROTONOSUPPORT (ZMQ_HAUSNUMERO + 2) #endif I can expand at the time of compilation, and get the value, but what if the library gets recompiled and the value changes? Any thoughts? I'll look in the meantime, at other people's bindings to see how it's solved, but this has puzzled me a bit Thanks, Dimiter 'malkia' Stanev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] zeromq2-1 pull request 39
On Thu, Feb 16, 2012 at 12:11:17PM -0600, Pieter Hintjens wrote: On Thu, Feb 16, 2012 at 11:38 AM, AJ Lewis aj.le...@quantum.com wrote: Any chance of pull request 39 for the zeromq2-1 repo getting applied? Without them, 2-1 won't build on HPUX Sorry for missing that. I've pulled it into zeromq2-1 master, and the patch will go into 2.1.12. Awesome - thanks! -- AJ Lewis Software Engineer Quantum Corporation Work:651 688-4346 -- The information contained in this transmission may be confidential. Any disclosure, copying, or further distribution of confidential information is not permitted unless such privilege is explicitly granted in writing by Quantum. Quantum reserves the right to have electronic communications, including email and attachments, sent across its networks filtered through anti virus and spam software programs and retain such messages in order to comply with applicable data security and retention requirements. Quantum is not responsible for the proper and complete transmission of the substance of this communication or for any delay in its receipt. ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] inet_ntop is not available in Windows XP
16.02.2012 22:43, Dimiter 'malkia' Stanev пишет: Hi guys, In short: inet_ntop is not available in Windows XP. I write message about that five hours ago (subject Compile branch with MSVC). :) I agree, this function should not be used if we want ZeroMQ compile on XP. But your solution is not acceptable because it requires boost library. signature.asc Description: OpenPGP digital signature ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] inet_ntop is not available in Windows XP
Ah, no :) Boost is one thing that I try to avoid. I'm actually more of a C guy that C++ and zmq has been nice to have first and foremost C interface, even if written in C++. I was more thinking of the second solution: http://stackoverflow.com/a/1564907/743263 From the winsock layer: WSAAddressToString (ntop) and WSAStringToAddress (pton), although there is a comment from someone: Note that WSAAddressToString will append the port number as well if your address structure has one. So to just get the IP, you may have to create a new address with no port. But for now I'm sticking to the change before Ian Barber's one (which seems very cool indeed, but unfortunately relying on something that XP does not support). There might be somewher inet_ntop.c compat layer for XP (probably part of mingw/cygwin), but I haven't started digging yet. On 2/16/2012 11:58 AM, Boris Gulay wrote: 16.02.2012 22:43, Dimiter 'malkia' Stanev пишет: Hi guys, In short: inet_ntop is not available in Windows XP. I write message about that five hours ago (subject Compile branch with MSVC). :) I agree, this function should not be used if we want ZeroMQ compile on XP. But your solution is not acceptable because it requires boost library. ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] inet_ntop is not available in Windows XP
On 16 February 2012 13:43, Dimiter 'malkia' Stanev mal...@gmail.com wrote: Hi guys, In short: inet_ntop is not available in Windows XP. No idea why Microsoft finally added it to Windows Vista when the API is already known to be broken with IPv6, just use getnameinfo() -- Steve-o ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] differences in polling between device and non device fd's
I'm using a jruby gem that has it's own reactor, and monitors zeromq file descriptors so you can send messages to the reactor via zeromq. I'm running into a strange issue where the reactor is not getting events from the fd. It gets a couple then no more. This happens to me when using the queue device, but not when I just use req/rep or push/pull. Is there anything different about how devices monitor file descriptors that could cause issues for external code that is monitoring them? At this point I'm assuming it's a bug in the jruby gem, but it would be helpful to know if devices do something different in this area before I go hacking on the jruby code. Anyone have any insight into this? Chris ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] inet_ntop is not available in Windows XP
On Thu, Feb 16, 2012 at 8:36 PM, Steven McCoy steven.mc...@miru.hk wrote: On 16 February 2012 13:43, Dimiter 'malkia' Stanev mal...@gmail.comwrote: Hi guys, In short: inet_ntop is not available in Windows XP. No idea why Microsoft finally added it to Windows Vista when the API is already known to be broken with IPv6, just use getnameinfo() Ah, this one is my bad, I'll use getnameinfo and send a pull req. Ian ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] Thread-safe sockets (cont.)
Hi, Does anyone have a valid use case for thread-safe sockets? It seems that the semantics are fuzzy and using this would lead to poor design. What happens if two threads are polling the same sockets, but one message arrives? What if two threads are in a blocking recv on the same socket? If we don't have a clear problem that this change is fixing, I'd like to patch it out. -Pieter ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Thread-safe sockets (cont.)
On Feb 16, 2012, at 3:51 PM, Pieter Hintjens wrote: Hi, Does anyone have a valid use case for thread-safe sockets? It seems that the semantics are fuzzy and using this would lead to poor design. What happens if two threads are polling the same sockets, but one message arrives? zmq_poll() would probably need to be modified to disallow multiple threads from polling. Alternately, zmq_poll() would need to disallow a poll_item list that contained a socket that is already a member of another call to zmq_poll() from another thread. Ugly. Error prone. I don't see how this could work well without some hacks. And now thread safety would generate a shit-ton of additional questions about why zmq_poll() was so strict, broken, etc. What if two threads are in a blocking recv on the same socket? Only one thread would be allowed through the mutex to block on the receive. The second thread would block on the mutex. No problem. If we don't have a clear problem that this change is fixing, I'd like to patch it out. I don't have a use-case but I think there are at least 2 people on the list who do. I hope they speak up. cr ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Thread-safe sockets (cont.)
On Thu, Feb 16, 2012 at 4:05 PM, Chuck Remes cremes.devl...@mac.com wrote: I don't have a use-case but I think there are at least 2 people on the list who do. I hope they speak up. Well, the minimal implementation is on libzmq/master, so anyone who wants to try it can do so now. -Pieter ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] inet_ntop is not available in Windows XP
Thanks guys! Awesome community! On 2/16/2012 12:45 PM, Ian Barber wrote: On Thu, Feb 16, 2012 at 8:36 PM, Steven McCoy steven.mc...@miru.hk mailto:steven.mc...@miru.hk wrote: On 16 February 2012 13:43, Dimiter 'malkia' Stanev mal...@gmail.com mailto:mal...@gmail.com wrote: Hi guys, In short: inet_ntop is not available in Windows XP. No idea why Microsoft finally added it to Windows Vista when the API is already known to be broken with IPv6, just use getnameinfo() Ah, this one is my bad, I'll use getnameinfo and send a pull req. Ian ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] inet_ntop is not available in Windows XP
On Thu, Feb 16, 2012 at 4:10 PM, Dimiter 'malkia' Stanev mal...@gmail.com wrote: Thanks guys! Awesome community! Ian's patch is now merged into libzmq/master. -Pieter ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] inet_ntop is not available in Windows XP
Synced to latest, but ssize_t is not declared for Windows WDK 7.1, and one of the zmq.h functions uses it. There is an easy workaround -Dssize_t=ptrdiff_t for my compilation process (I compile the library using my own batch file). There is also intptr_t, but I chose ptrdiff_t for the workaround, as it's declared in more header files in the WDK :) I think VS2010 supports ssize_t, but VS2008 probably not, and WDK 7.1 is closer to VS2008 than VS2010. On 2/16/2012 2:12 PM, Pieter Hintjens wrote: On Thu, Feb 16, 2012 at 4:10 PM, Dimiter 'malkia' Stanev mal...@gmail.com wrote: Thanks guys! Awesome community! Ian's patch is now merged into libzmq/master. -Pieter ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] inet_ntop is not available in Windows XP
Actually in WDK there is SSIZE_T (not a macro), which seems more appropriate for the workaround (e.g. -Dssize_t=SSIZE_T). There is one more problem, unrelated to it: socket_base.cpp ..\..\..\src\socket_base.cpp(468) : error C2512: 'zmq::ipc_address_t' : no appropriate default constructor available ..\..\..\src\socket_base.cpp(470) : error C2027: use of undefined type 'zmq::ipc_address_t' e:\p\libzmq\src\address.hpp(29) : see declaration of 'zmq::ipc_address_t' ..\..\..\src\socket_base.cpp(470) : error C2227: left of '-resolve' must point to class/struct/union/generic type stream_engine.cpp On 2/16/2012 2:31 PM, Dimiter 'malkia' Stanev wrote: Synced to latest, but ssize_t is not declared for Windows WDK 7.1, and one of the zmq.h functions uses it. There is an easy workaround -Dssize_t=ptrdiff_t for my compilation process (I compile the library using my own batch file). There is also intptr_t, but I chose ptrdiff_t for the workaround, as it's declared in more header files in the WDK :) I think VS2010 supports ssize_t, but VS2008 probably not, and WDK 7.1 is closer to VS2008 than VS2010. On 2/16/2012 2:12 PM, Pieter Hintjens wrote: On Thu, Feb 16, 2012 at 4:10 PM, Dimiter 'malkia' Stanev mal...@gmail.com wrote: Thanks guys! Awesome community! Ian's patch is now merged into libzmq/master. -Pieter ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] inet_ntop is not available in Windows XP
On Thu, Feb 16, 2012 at 4:38 PM, Dimiter 'malkia' Stanev mal...@gmail.com wrote: Actually in WDK there is SSIZE_T (not a macro), which seems more appropriate for the workaround (e.g. -Dssize_t=SSIZE_T). You may define this conditionally in include/zmq.,h, and in src/msg.hpp. If that works, send us a pull request. Better than relying on a specific project file, IMO. -Pieter ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Thread-safe sockets (cont.)
I would like to propose another point, for your consideration. I don't know that it is a good idea to marry 0mq to a threading library (even if it is a platform foundation like pthreads), as this might make adoption hard for folks that need to in other threading environments, such as TBB or openmp, for example. After thinking about it, I believe a much more germane place for it would be in the language binding libs such as czmq. I other words I think that thread safety is an application level concern. Also, if my app is using already libzmq in a thread safe way, why should I pay for all this locking and unlocking? Luca On 17/02/12 11:09, Pieter Hintjens wrote: On Thu, Feb 16, 2012 at 4:05 PM, Chuck Remescremes.devl...@mac.com wrote: I don't have a use-case but I think there are at least 2 people on the list who do. I hope they speak up. Well, the minimal implementation is on libzmq/master, so anyone who wants to try it can do so now. -Pieter ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev -- Luca Fascione Rendering Research Lead - Weta Digital Phone: +64 4 909 6870 (x6870) Mobile: +64 21 0764 862 ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] Getting 0MQ/3.1 to stable status
Hi all, It's probably a good time to start moving 0MQ/3.1 towards stable status. Several things you can do to help: * If you have outstanding issues in 3.1 that you need fixed, please remind us of them * If you're maintaining a language binding, make sure you have 3.1 support in place * If you're maintaining documentation that refers to 2.1, time to start making a 3.1 upgrade We may be able to get all our projects ready for a 3.1 release candidate within 2-3 weeks. -Pieter ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Thread-safe sockets (cont.)
On Thu, Feb 16, 2012 at 5:12 PM, Luca Fascione lu...@wetafx.co.nz wrote: I don't know that it is a good idea to marry 0mq to a threading library (even if it is a platform foundation like pthreads), as this might make adoption hard for folks that need to in other threading environments, such as TBB or openmp, for example. After thinking about it, I believe a much more germane place for it would be in the language binding libs such as czmq. I other words I think that thread safety is an application level concern. Quite plausible. Also, if my app is using already libzmq in a thread safe way, why should I pay for all this locking and unlocking? You don't pay anything, except that the API is a little more complex. The real question is whether the functionality is accurate and profitable. -Pieter ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Getting 0MQ/3.1 to stable status
Great news, Pieter. I just did a build of our rsyslog plugins against 3.1 today to start testing. Brian On Thu, Feb 16, 2012 at 6:14 PM, Pieter Hintjens p...@imatix.com wrote: Hi all, It's probably a good time to start moving 0MQ/3.1 towards stable status. Several things you can do to help: * If you have outstanding issues in 3.1 that you need fixed, please remind us of them * If you're maintaining a language binding, make sure you have 3.1 support in place * If you're maintaining documentation that refers to 2.1, time to start making a 3.1 upgrade We may be able to get all our projects ready for a 3.1 release candidate within 2-3 weeks. -Pieter ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Getting 0MQ/3.1 to stable status
So the two main issues we need to fix are #264 and #270. They are, by all accounts, extremely hard to solve, and anyone finding a clean solution would be regarded with awe and admiration. -Pieter On Thu, Feb 16, 2012 at 5:28 PM, Brian Knox taote...@gmail.com wrote: Great news, Pieter. I just did a build of our rsyslog plugins against 3.1 today to start testing. Brian On Thu, Feb 16, 2012 at 6:14 PM, Pieter Hintjens p...@imatix.com wrote: Hi all, It's probably a good time to start moving 0MQ/3.1 towards stable status. Several things you can do to help: * If you have outstanding issues in 3.1 that you need fixed, please remind us of them * If you're maintaining a language binding, make sure you have 3.1 support in place * If you're maintaining documentation that refers to 2.1, time to start making a 3.1 upgrade We may be able to get all our projects ready for a 3.1 release candidate within 2-3 weeks. -Pieter ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Getting 0MQ/3.1 to stable status
On Feb 16, 2012, at 5:50 PM, Pieter Hintjens wrote: So the two main issues we need to fix are #264 and #270. They are, by all accounts, extremely hard to solve, and anyone finding a clean solution would be regarded with awe and admiration. -Pieter Yes, plus both issues have C test cases to reproduce the problems. #264 has the test case attached directly while the test code for 270 is here: https://github.com/zeromq/issues/tree/master/270 cr ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] differences in polling between device and non device fd's
On 17/02/2012, at 7:41 AM, snacktime wrote: I'm using a jruby gem that has it's own reactor, and monitors zeromq file descriptors so you can send messages to the reactor via zeromq. I'm running into a strange issue where the reactor is not getting events from the fd. It gets a couple then no more. This happens to me when using the queue device, but not when I just use req/rep or push/pull. Is there anything different about how devices monitor file descriptors that could cause issues for external code that is monitoring them? The queue device is using zmq_poll, which in turn is calling select or whatever. If you're also polling the same files it can't work. It should work if you're polling different ones. HTH -- john skaller skal...@users.sourceforge.net ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Thread-safe sockets (cont.)
On 17/02/2012, at 8:51 AM, Pieter Hintjens wrote: Hi, Does anyone have a valid use case for thread-safe sockets? There's an example in the test cases. Using ts sockets to serialise sends from different threads is much easier than using inproc transport. You can take any existing code that writes to a socket, and split some of the work, previously done in a single thread, out into several threads without changing anything other than the socket thread safety bit. With the current design, this would occur at context construction time, in a place likely to be different from the code handling transmission. In other words, this provides transparency in *some* cases. The alternative would be a network topology redesign: you'd have to add an inproc socket for every thread, and also you'd have to get the server to read them and write to the output. This requires yet another socket and code to do the transfer. TS sockets are hands-down-no-argument easier than all that, especially considering there are no devices in 3.1 which might automate it. It seems that the semantics are fuzzy The semantics seem well defined to me. All access to the a ts_safe socket are serialised by a mutex. Au contrare . .. it is the semantics of inproc that seem fuzzy to me. It isn't clear to me how much buffering is done by inproc. Therefore it isn't clear to me to what extent, if any, inproc allows threads to synchronise. Felix pchannels are unbuffered and provide rigid synchronisation. inproc would also do this with HWM = 0 if that option is available. HWM=1 is quite different, since a write would proceed after buffering a message. and using this would lead to poor design. What happens if two threads are polling the same sockets, but one message arrives? That would be bad design. It has nothing to do with the existence of thread-safety. What if two threads are in a blocking recv on the same socket? It cannot happen. One thread will block on recv, the other will block on the mutex set by the first thread. After the first thread receives, the second one blocks. If multiple threads are waiting one will be selected at random to proceed next. If we don't have a clear problem that this change is fixing, I'd like to patch it out. If we don't have the API available, no one will every have a clear use case. It is likely I will would use this in my Felix binding .. not that this is a clear use case. That feature would allow 0MQ transport to be used safely from a thread pool. BTW: please patch it out manually if you have to, don't revert, as my patch made some other useful changes (refactoring the API to not call itself and consistently convert from the C data type to the C++ one). The easiest way to patch it out is to just remove the API call and test case: there is no doco at this time. Or better, MOVE the API call to another header: zmq_experimental.h I hope you don't because a binding based mutex is NOT the same thing, since it will not inter-operate with C/C++ code. -- john skaller skal...@users.sourceforge.net ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Thread-safe sockets (cont.)
On 17/02/2012, at 9:05 AM, Chuck Remes wrote: On Feb 16, 2012, at 3:51 PM, Pieter Hintjens wrote: Hi, Does anyone have a valid use case for thread-safe sockets? It seems that the semantics are fuzzy and using this would lead to poor design. What happens if two threads are polling the same sockets, but one message arrives? zmq_poll() would probably need to be modified to disallow multiple threads from polling. Alternately, zmq_poll() would need to disallow a poll_item list that contained a socket that is already a member of another call to zmq_poll() from another thread. Ugly. Error prone. I don't see how this could work well without some hacks. And now thread safety would generate a shit-ton of additional questions about why zmq_poll() was so strict, broken, etc. No need to change it at all. It is the same as now: if you misuse a feature, you can't expect reliable results. The difference is now you get unexpected results instead of a segfault. -- john skaller skal...@users.sourceforge.net ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Thread-safe sockets (cont.)
On 17/02/2012, at 10:12 AM, Luca Fascione wrote: Also, if my app is using already libzmq in a thread safe way, why should I pay for all this locking and unlocking? You pay for a boolean test only: the locking is conditional on a flag. -- john skaller skal...@users.sourceforge.net ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Thread-safe sockets (cont.)
On Thu, Feb 16, 2012 at 6:32 PM, john skaller skal...@users.sourceforge.net wrote: No need to change it at all. It is the same as now: if you misuse a feature, you can't expect reliable results. The difference is now you get unexpected results instead of a segfault. There needs to be a very clear explanation of how to safely use sockets from multiple threads, then. It's one of those areas that causes endless problems. One thread closing a socket while another tries to use it, etc. After a lot of this, we finally just said do not use sockets in multiple threads, period. Now this becomes possible again, people will use it, and will get it wrong. That's one risk I see. The second risk is that the design is not complete, and people not using it, means it won't mature in time for a 3.1 stable release. I'm not arguing against the feature but we need some positive feedback from people using it, IMO. John, if you had a single clear use case yourself, code which is ugly and fragile without this, and simpler with it, that would help a lot. -Pieter ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Getting 0MQ/3.1 to stable status
I've updated the 2.1 - 3.1 conversion guide here: http://www.zeromq.org/docs:3-1-upgrade Will retest tomorrow with CZMQ. If anyone has things to correct or add on that page, go for it... -Pieter ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Getting 0MQ/3.1 to stable status
On 17/02/2012, at 10:14 AM, Pieter Hintjens wrote: Hi all, It's probably a good time to start moving 0MQ/3.1 towards stable status. Several things you can do to help: Sure: please fork or whatever is necessary, the Guide so there is an editable 3.1 version. Please check all the 3.1 Guide examples work in C. Please wait a certain time for all bindings to be checked against a release candidate, including all Guide examples working. API changes: the ZQM_AFFINITY should be fixed. I can do that. My fix will just take the first 64 bits of the array and use the older code, but the new API will be in place even thought it doesn't work. -- john skaller skal...@users.sourceforge.net ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Getting 0MQ/3.1 to stable status
nice. I'll rename the perl binding to ZMQ for 3.1 support as well --d 2012/02/17 8:14 Pieter Hintjens p...@imatix.com: Hi all, It's probably a good time to start moving 0MQ/3.1 towards stable status. Several things you can do to help: * If you have outstanding issues in 3.1 that you need fixed, please remind us of them * If you're maintaining a language binding, make sure you have 3.1 support in place * If you're maintaining documentation that refers to 2.1, time to start making a 3.1 upgrade We may be able to get all our projects ready for a 3.1 release candidate within 2-3 weeks. -Pieter ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Thread-safe sockets (cont.)
On 17/02/2012, at 11:41 AM, Pieter Hintjens wrote: John, if you had a single clear use case yourself, code which is ugly and fragile without this, and simpler with it, that would help a lot. libzmq/tests/test_ts_context.cpp uses it, you could rewrite that with inproc and compare. I may have a real world use: the process manager norK I'm writing. But it isn't there yet and I have other things to do simultaneously. At present I'm gathering published diagnostics using Posix tricks and my own thread safe queue, rather than 0MQ. This doesn't scale across networks. But it was much easier to use in a prototype and leaves the program not dependent on 0MQ, especially a variant that isn't stable and might get the feature removed! As mentioned elsewhere it just isn't clear if inproc will cut it. I'm not a 0MQ expert, so I can't tell if HWM=0 is allowed, but that would be required (I mean, blocking write where the result is sent before proceeding: the read end can be buffered). The messages are event diagnostics so temporal sequencing is desirable :) -- john skaller skal...@users.sourceforge.net ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Thread-safe sockets (cont.)
Just a comment on the *misconception* that it is OK to do thread safety in a binding or application logic. No, it isn't. The reason is illustrated by a counter example: 1. CPython-ZMQ binding. 2. CPython-fun-twister stuff client API. The problem is that (1) might add thread safety. But (2) doesn't call (1). It calls the C API, just as (1) does. So now, there's no safety. Because the lifted API is not the only one available, there's no way to enforce sharing. Another trivial example: a C program hosting embedded Lua code. Same problem. Placing the mutex lock *inside* the C APi enforces sharing over all clients including all bindings. It doesn't enforce sharing on those daring to use the raw C++ code -- but it is optionally available because the mutex is in the raw C++ class object. There is an analogue in Posix: optional locking. It's fairly useless. Because anyone can write into a locked region of a file by ignoring the locks. It's not entirely useless but in todays world software is so complex people want rigid guarantees of correctness for some things, because there are always a heap of other things where there are no assurances: [casts in C .. dynamic typing in web applications OMG what a regressive step in CS] -- john skaller skal...@users.sourceforge.net ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Router/Dealer lost messages
For 3.1.x release: More tests and ifo about ZMQ_ROUTER to ZMQ_ROUTER bug LIBZMQ-304. https://zeromq.jira.com/browse/LIBZMQ-304 ZMQ_ROUTER to ZMQ_ROUTER does not work for first sent messages. Message parts are ok. In this test s_dump() is a non-blocking read. Blocking read will block in first s_dump() call forever. CODE: long err; void *context = zmq_init (1); void *server = zmq_socket (context, ZMQ_ROUTER); err = zmq_setsockopt (server, ZMQ_IDENTITY, SERVER, 6); err = zmq_bind (server, tcp://127.0.0.1:); void *worker = zmq_socket (context, ZMQ_ROUTER); err = zmq_setsockopt (worker, ZMQ_IDENTITY, WORKER, 6); err = zmq_connect (worker, tcp://127.0.0.1:); sleep_seconds (0.5); // sleep in seconds s_sendmore (worker, (char *)SERVER); s_sendmore (worker, (char *)); s_send (worker, (char *)send to server1); sleep_seconds (0.001); // sleep in seconds s_dump (server); // print to console s_sendmore (worker, (char *)SERVER); s_sendmore (worker, (char *)); s_send (worker, (char *)send to server2); sleep_seconds (0.001); // sleep in seconds s_dump (server); // print to console s_sendmore (server, (char *)WORKER); s_sendmore (server, (char *)); s_send (server, (char *)send to worker1); sleep_seconds (0.001); // sleep in seconds s_dump (worker); // print to console s_sendmore (worker, (char *)SERVER); s_sendmore (worker, (char *)); s_send (worker, (char *)send to server3); sleep_seconds (0.001); // sleep in seconds s_dump (server); // print to console s_sendmore (server, (char *)WORKER); s_sendmore (server, (char *)); s_send (server, (char *)send to worker2); sleep_seconds (0.001); // sleep in seconds s_dump (worker); // print to console zmq_close(worker); zmq_close(server); zmq_term (context); OUTPUT: [006] SERVER [000] [015] send to worker1 [006] WORKER [000] [015] send to server3 [006] SERVER [000] [015] send to worker2 ROUTER to DEALER works, but I'm not sure sould we get those kind of message parts? CODE: ... void *worker = zmq_socket (context, ZMQ_DEALER); err = zmq_setsockopt (worker, ZMQ_IDENTITY, WORKER, 6); err = zmq_connect (worker, tcp://127.0.0.1:); sleep_seconds (0.5); // sleep in seconds s_send (worker, (char *)send to server1); sleep_seconds (0.001); // sleep in seconds s_dump (server); // print to console s_sendmore (server, (char *)WORKER); s_sendmore (server, (char *)); s_send (server, (char *)send to worker1); sleep_seconds (0.001); // sleep in seconds s_dump (worker); // print to console ... OUTPUT: [006] WORKER [015] send to server1 [000] [015] send to worker1 -- Regards, Pasi Mankinen Finland ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Big Latencies
Aja Walker a...@ciscor.com wrote: Hi all! I posted this question about a week ago, but it was fairly long-winded, and there was some other deep, heated discussion going on so I thought I'd give it one more shot. I hope I'm not offending the etiquette of the mailing list by re-posting, but if so, my apologies, and I won't do it again. :) The previous (overly?) detailed post is here: http://lists.zeromq.org/pipermail/zeromq-dev/2012-February/015640.html Briefly, I'm occasionally seeing big, as in 10 milliseconds, end-to-end latencies on my Windows 7 machine, but not on my Linux machine. In certain circumstances these big latencies are frequent rather than occasional, as in 80% of messages sent. Has anyone else seen anything like this? Sincerely, Aja Walker Hello. I had big performance problems with Apache APR (http://apr.apache.org/) based program. Setting registry values fixed performance problems. I don't think this is problem with zmq, but maybe you should test it? My tests with zqm were very small. http://support.microsoft.com/kb/823764 Slow performance occurs when you copy data to a TCP server by using a Windows Sockets API program. -- Regards, Pasi Mankinen Finland ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Thread-safe sockets (cont.)
On Feb 16, 2012, at 8:56 PM, john skaller wrote: There is an analogue in Posix: optional locking. It's fairly useless. Because anyone can write into a locked region of a file by ignoring the locks. That is a pretty broad generalization. Advisory locks are quite useful when you are dealing with co-operative access, which happens automatically if you've got multiple clients utilizing the same library code. I think it would be unusual to have arbitrary clients accessing something like a shared file (or shared socket...). Gary Wright ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Router/Dealer lost messages
On Thu, Feb 16, 2012 at 8:50 PM, Pasi Mankinen mailing.l...@manageapp.com wrote: For 3.1.x release: More tests and ifo about ZMQ_ROUTER to ZMQ_ROUTER bug LIBZMQ-304. https://zeromq.jira.com/browse/LIBZMQ-304 Pasi, can you add this to the Jira issue? It's possible that the removal of durable sockets (identities) is causing the ROUTER-ROUTER issue. -Pieter ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Thread-safe sockets (cont.)
On 17.02.2012, at 3:32, john skaller skal...@users.sourceforge.net wrote: On 17/02/2012, at 9:05 AM, Chuck Remes wrote: On Feb 16, 2012, at 3:51 PM, Pieter Hintjens wrote: zmq_poll() would probably need to be modified to disallow multiple threads from polling. Alternately, zmq_poll() would need to disallow a poll_item list that contained a socket that is already a member of another call to zmq_poll() from another thread. Ugly. Error prone. I don't see how this could work well without some hacks. And now thread safety would generate a shit-ton of additional questions about why zmq_poll() was so strict, broken, etc. No need to change it at all. It is the same as now: if you misuse a feature, you can't expect reliable results. The difference is now you get unexpected results instead of a segfault. I believe, such a feature (unexpected behavior instead of segfault) is enough reason to revert the patch. ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Thread-safe sockets (cont.)
I am by far no expert in 0MQ, but actually the question of thread-safe socket is no question any more for me. It has been written in the documentation in a clear way, that one should not use sockets from different threads. Over the last weeks - and due to a question from me - it was in a more detailed way explained, what is allowed to do and what not. These informations should be added to the documentation. I've no problem to follow this guideline in my Smalltalk language binding. Its ok and I can work with it and I offer a thread safe interface to the 0MQ library. When people starts working around my official language binding - ok, then its their fault. I can not and would not like to forbid this - I assume, that others also might know how to use the 0MQ library. Therefore: leave the decision to the end user and give them enough information so that it can make their own decisions based o this free available information. I like 0MQ because it has such a small C-level API and it is easy to use. The examples guides are well done and show how to use it. Perhaps thread-safe sockets are a nice marketing idea. Some people might think, that this is an absolute must to have - and without that a library is not a reliable piece of software. It is perhaps the same as thinking, that dynamic typed languages are a regressive step in CS. At least this last idea is simply wrong and has been shown over the last 50 years in CS. There is an analogue in Posix: optional locking. It's fairly useless. Because anyone can write into a locked region of a file by ignoring the locks. This optional locking is quite a good design pattern: Perhaps the base system contains an error and the strict locking prevents you from doing the last emergency execution step. Its then always good to have the choice (thats the keyword in building libraries) of turning of this locking ... Marten Am 17.02.2012 02:56, schrieb john skaller: It's not entirely useless but in todays world software is so complex people want rigid guarantees of correctness for some things, because there are always a heap of other things where there are no assurances: [casts in C .. dynamic typing in web applications OMG what a regressive step in CS] ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Thread-safe sockets (cont.)
I apologize for jumping in mid-conversation, but I see two thread-safety goals: 1) 0MQ has no built-in locks on sockets. Applications may use a socket in different threads provided the application itself uses its own locks around the socket accesses. 2) 0MQ has its own locks built-in, so you can use a socket in different threads and the application does not need its own locking. As I understand it, the current stable 0MQ does not even allow #1. At least, I have read documentation stating that a socket should not be used except in the thread it was created in, meaning sockets have thread-affinity. Is this true? I personally would like to see #1 solved if it has not been already. I think it should just work, but offering a function to change thread affinity of a socket would be acceptable. Otherwise, 0MQ is thread impossible, which to me is much worse than not thread safe. :) I have no need for #2 and I think it's overkill. The only justification I can see for it is if 0MQ is trying to emulate posix/kernel behavior and this is a necessity in its long term goals. On Thursday, February 16, 2012 10:53:19 PM Marten Feldtmann wrote: I am by far no expert in 0MQ, but actually the question of thread-safe socket is no question any more for me. It has been written in the documentation in a clear way, that one should not use sockets from different threads. Over the last weeks - and due to a question from me - it was in a more detailed way explained, what is allowed to do and what not. These informations should be added to the documentation. I've no problem to follow this guideline in my Smalltalk language binding. Its ok and I can work with it and I offer a thread safe interface to the 0MQ library. When people starts working around my official language binding - ok, then its their fault. I can not and would not like to forbid this - I assume, that others also might know how to use the 0MQ library. Therefore: leave the decision to the end user and give them enough information so that it can make their own decisions based o this free available information. I like 0MQ because it has such a small C-level API and it is easy to use. The examples guides are well done and show how to use it. Perhaps thread-safe sockets are a nice marketing idea. Some people might think, that this is an absolute must to have - and without that a library is not a reliable piece of software. It is perhaps the same as thinking, that dynamic typed languages are a regressive step in CS. At least this last idea is simply wrong and has been shown over the last 50 years in CS. There is an analogue in Posix: optional locking. It's fairly useless. Because anyone can write into a locked region of a file by ignoring the locks. This optional locking is quite a good design pattern: Perhaps the base system contains an error and the strict locking prevents you from doing the last emergency execution step. Its then always good to have the choice (thats the keyword in building libraries) of turning of this locking ... Marten Am 17.02.2012 02:56, schrieb john skaller: It's not entirely useless but in todays world software is so complex people want rigid guarantees of correctness for some things, because there are always a heap of other things where there are no assurances: [casts in C .. dynamic typing in web applications OMG what a regressive step in CS] ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] ECANTROUTE not found in OSX
On Fri, Feb 17, 2012 at 3:25 AM, Pasi Mankinen mailing.l...@manageapp.comwrote: Hi. I am not a C/C++ developer. I use higher level languages but sometimes I do some simple C plugins. Zmq is so easy to use that even I can use it :). I don't have a glue how to fix this correctly in zmq source, sorry. Maybe some conditional define in zmq.h? I created bug LIBZMQ-324: https://zeromq.jira.com/browse/LIBZMQ-324 I just try to help by testing latest git version with OSX and occasionally with VisualStudio 2010/2008 in Win7/XP. Later I will start testing performance of different RPC architectures and language bindings (libzmq, czmq, node.js and maybe Lua). I think this is just a doc error, don't believe anything actually sends ECANTROUTE. Will check and update docs. Ian ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
