[zeromq-dev] ZeroMQ : Issue with PUB-SUB
Hi All, Currently we are using PUB-SUB and PUSH-PULL . Our server is on Linux ( C/C++ ) and clients are on JAVA Android. Server Version : ZeroMq 3.3.0 Client Version : JeroMq 3.2.2 *Design we are using : * Client queries server using PUSH. Server receives this query on PULL. Now server responds to the client via PUB. On the other hand clients are waiting with SUB ids. *Issues : * 1. After sometime some of the subscribers are NOT receiving the message, when client is re-started , they start getting the message . 2. After some time , we have observed , subscribers start receiving the old message multiple times , where as server has published different message . Both the issues are frequent , please let us know How to fix the issue . Thanks And With Regards Ranjeet Kumar ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Problem installing python language binding on openwrt
Greg, thank you for answering. Sure, it is definitely known that it is impossible to compile C language programs on OpenWRT box itself. In case you need compilation, cross-compiler is used on another platform. In my case I already have zeromq library compiled and installed on device. Do I really need to compile Python bindings in case of preinstalled library? On Mon, Dec 9, 2013 at 10:59 PM, Greg Ward g...@gerg.ca wrote: On 08 December 2013, Roman said: I tried to install python zeromq binding on openwrt issuing command python setup.py install. Script ended with the following error: unable to execute ccache_cc: No such file or directory error: command 'ccache_cc' failed with exit status 1 It sounds like Python was built with a compiler (ccache_cc) that is not installed on your machine. You should probably ask on an OpenWRT forum where to get ccache_cc. Things to investigate: * Are you able to compile other C programs/libraries? * Are you able to compile other Python extensions? Also, try DISTUTILS_DEBUG=1 python setup.py install That might shed some light. Greg ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] czmq_selftest fails after fresh clone
I've sent a patch to remove the assertion. On Mon, Dec 9, 2013 at 2:23 PM, KIU Shueng Chuan nixch...@gmail.com wrote: You should be right that the 64-bit alignment assertion is bogus. Even the openssl manpage of sha1_update doesn't mention any alignment requirements. The czmq included version doesn't seem to have any requirement either. ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Can I change a socket mechanism after a connect ?
The security mechanism, like some other socket options, applies to the next connection (in or outgoing). A single TCP connection cannot switch between mechanisms. There's no protocol for that, nor is there any sense in it. Changing security level on a socket after having connections is also dangerous and we've discussed banning it. On Mon, Dec 9, 2013 at 3:24 PM, Laurent Alebarde l.aleba...@free.fr wrote: Hi Devs, Is it possible to start a socket as NULL, send some messages, then change the socket to CURVE ? Does the identity of the socket change in the process ? Cheers, Laurent ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Question about identities
Laurent, please read the ZMTP v3 protocol spec with regard to identities. A tells B, this is my identity. B then uses that value to refer to A, IF AND ONLY IF B IS A ROUTER. More usually, A says I have no identity and B assigns an internal numeric identity. On Mon, Dec 9, 2013 at 3:59 PM, Laurent Alebarde l.aleba...@free.fr wrote: Hi Devs, Are the identities of two different sockets on the same peer the same, viewed by one single other peer ? Say: Node 1 Node 2 Socket A1 (NULL) -- Socket A2 (NULL) : Node 1 has identity I1a Socket B1 (CURVE) -Socket B2 (CURVE) : Node 1 has identity I1b Is I1a == I1b ? Cheers, Laurent ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] zmq_msg_move
On Mon, Dec 9, 2013 at 3:23 PM, KIU Shueng Chuan nixch...@gmail.com wrote: 1) yes, needed. 2) yes, it should. After the move, the origin message returns to an init-ed state. You could get away with not closing, but in my opinion that relies on knowing the underlying implementation. 3) yes. However in this case, closing the origin message is mandatory. (Unless you also know that very short messages are copied and not shared) Would you like to make a patch to the man page? I implemented a simple multi part message c++ wrapper using the move and copy functions. https://github.com/pijyoi/msg_parts/blob/master/msg_parts.hpp Consider adding this to the C++ high level binding... (we may also want to rename that to cppzmq, and move the current cppzmq .h file back into libzmq). -Pieter ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] zauth vs linger time
Hello out there! I have been confounded by something I ran into while learning about the security additions to 0MQ. I was going through the security tutorial at http://hintjens.com/blog:49 and everything was cashmere until the final code snippet. It happened in the refactored ironhouse2, where the client and server are running on different threads. I noticed that (after double-checking zauth_set_verbose was indeed being set) I was not seeing the I: ALLOWED (CURVE) ... output I had just become accustomed to -- but the test still claimed to have passed. Curious, I started messing around with the zauth_allow statements and found that my client would receive no matter what I did to the authentication or curve settings. After some prodding it became apparent that letting the server thread sleep a little longer (another 100ms) the authentication mechanism would kick in and deny the client connection due to the now bogus whitelist of 227.0.0.2. Interesting. I'm still quite a bit confused by it, potentially due to my perilously weak grasp of C. Did the fact that the linger time was set (100ms) longer than the zauth mechanism allow the client to read the lingering message? Running with libzmq 4.1.0 on OS X 10.7.5 Best, Sean ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Question about identities
I did read ZMTP 3 RFC23 several times, but it is not so obvious, even with the Worked Example. In particular if two clients connect in the same time to a ROUTER server. As the identity is communicated in the metadata at the end of the handcheck, how the server distinguishes the two clients ? Is there some temporary identities ? If yes, on what is it built ? Besides, setting an identity is an option. So a DEALER talking to a ROUTER SHALL set an arbitrary one. But it knows it talks to a ROUTER after having sent its identity. So it doesn't know it talks to a ROUTER when it sends its identity, and therefore, it can send an empty one. I remain a bit confused. Le 10/12/2013 10:17, Pieter Hintjens a écrit : Laurent, please read the ZMTP v3 protocol spec with regard to identities. A tells B, this is my identity. B then uses that value to refer to A, IF AND ONLY IF B IS A ROUTER. More usually, A says I have no identity and B assigns an internal numeric identity. On Mon, Dec 9, 2013 at 3:59 PM, Laurent Alebarde l.aleba...@free.fr wrote: Hi Devs, Are the identities of two different sockets on the same peer the same, viewed by one single other peer ? Say: Node 1 Node 2 Socket A1 (NULL) -- Socket A2 (NULL) : Node 1 has identity I1a Socket B1 (CURVE) -Socket B2 (CURVE) : Node 1 has identity I1b Is I1a == I1b ? Cheers, Laurent ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] Pub-Sub written in same file ..!!
All,
Below code is working fine when SSOMSSub () and SSOMSPub () API are written
as separate two files (sub.cpp and pub.cpp) but it doesn't execute properly
when both API SSOMSSub () SSOMSPub () are written in same single
test1.hpp file as below and this file being called by driver file test.cpp
as extreme below -
---test1.hpp
#include iostream
#include sstream
#include cstdio
#include cstdlib
#include string
#include zmq.hpp
#include zmq_utils.h
using namespace std;
struct MessageStruct {
int UniqueID;
short ProClientIndicator;
charAccountNumber;
short ORDERTYPE;
charSymbol;
int Volume;
int Price;
short Buy_Sell;
} ss_oms_call;
class Test {
public:
int SSOMSSub ();
int SSOMSPub ();
};
int Test::SSOMSSub () {
zmq::context_t context (1);
std::cout Collecting updates from Strategy Server.\n
std::endl;
zmq::socket_t subscriber (context, ZMQ_SUB);
subscriber.connect(tcp://localhost:5556);
const char *filter = ;
subscriber.setsockopt(ZMQ_SUBSCRIBE, filter, strlen (filter));
int update_nbr;
for (update_nbr = 0; update_nbr 100; update_nbr++) {
zmq::message_t update;
subscriber.recv(update);
std::istringstream iss(static_castchar*(update.data()));
iss ss_oms_call.UniqueID
ss_oms_call.ProClientIndicator ss_oms_call.AccountNumber
ss_oms_call.ORDERTYPE ss_oms_call.Symbol
ss_oms_call.Volume ss_oms_call.Price
ss_oms_call.Buy_Sell;
}
std::cout The Unique ID is .. filter
ss_oms_call.UniqueID std::endl;
std::cout The Type of Account is .. filter
ss_oms_call.ProClientIndicator std::endl;
std::cout The AccountNumber is .. filter
ss_oms_call.AccountNumber std::endl;
std::cout The ORDERTYPE is .. filter
ss_oms_call.ORDERTYPE std::endl;
std::cout The Symbol is .. filter ss_oms_call.Symbol
std::endl;
std::cout The Volume is .. filter ss_oms_call.Volume
std::endl;
std::cout The Price is .. filter ss_oms_call.Price
std::endl;
std::cout The Buy or Sell is .. filter
ss_oms_call.Buy_Sell std::endl;
return 0;
}
int Test::SSOMSPub () {
zmq::context_t context (1);
zmq::socket_t pubs (context, ZMQ_PUB);
pubs.bind(tcp://*:5556);
pubs.bind(ipc://ssoms.ipc);
while (1) {
ss_oms_call.UniqueID = 10001;
ss_oms_call.ProClientIndicator = 1;
ss_oms_call.AccountNumber = '4';
ss_oms_call.ORDERTYPE = 0x;
ss_oms_call.Symbol = 0x4e;
ss_oms_call.Volume = 50;
ss_oms_call.Price = 625000;
ss_oms_call.Buy_Sell = 2;
zmq::message_t message(1000);
snprintf ((char *) message.data(), 1000 , %d %hu %x %hu %x
%d %d %hu , ss_oms_call.UniqueID,
ss_oms_call.ProClientIndicator, ss_oms_call.AccountNumber,
ss_oms_call.ORDERTYPE, ss_oms_call.Symbol,
ss_oms_call.Volume, ss_oms_call.Price,
ss_oms_call.Buy_Sell);
pubs.send(message);
}
return 0;
}
and driver file (test2.cpp) -
---
#include iostream
#include test1.hpp
int main () {
Test tst;
tst.SSOMSSub ();
tst.SSOMSPub ();
return 0;
}
--
It's execution hangs as -
$ ./test2
Collecting updates from Strategy Server.
--
Any clue.
Thanks!!
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Can I change a socket mechanism after a connect ?
Thanks Pieter for your reply. nor is there any sense in it : The usual opinion is that when one invent the wheel, he doesn't know 90% of the usefull applications others will do with it. In my use case, it concerns curve proxying. Registering workers under NULL before making the proxy active with CURVE is simplier. So, may I assume it is alright if I I zmq_disconnect, change the security options of the socket, and re-zmq_connect ? Other solution from our other discussion, may I assume I can use a specific NULL socket between the worker and the proxy, just for registration, and setting the same identities on both sockets, the NULL one for registration, and the CURVE one that shall be proxified ? Le 10/12/2013 10:15, Pieter Hintjens a écrit : The security mechanism, like some other socket options, applies to the next connection (in or outgoing). A single TCP connection cannot switch between mechanisms. There's no protocol for that, nor is there any sense in it. Changing security level on a socket after having connections is also dangerous and we've discussed banning it. On Mon, Dec 9, 2013 at 3:24 PM, Laurent Alebardel.aleba...@free.fr wrote: Hi Devs, Is it possible to start a socket as NULL, send some messages, then change the socket to CURVE ? Does the identity of the socket change in the process ? Cheers, Laurent ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Can I change a socket mechanism after a connect ?
Thanks Pieter for your reply. nor is there any sense in it : The usual opinion is that when one invent the wheel, he doesn't know 90% of the usefull applications others will do with it. In my use case, it concerns curve proxying. Registering workers under NULL before making the proxy active with CURVE is simplier. I could use a specific socket, but I assume the identity is different then ? So, may I assume it is alright if I I zmq_disconnect, change the security options of the socket, and re-zmq_connect ? Le 10/12/2013 10:15, Pieter Hintjens a écrit : The security mechanism, like some other socket options, applies to the next connection (in or outgoing). A single TCP connection cannot switch between mechanisms. There's no protocol for that, nor is there any sense in it. Changing security level on a socket after having connections is also dangerous and we've discussed banning it. On Mon, Dec 9, 2013 at 3:24 PM, Laurent Alebarde l.aleba...@free.fr wrote: Hi Devs, Is it possible to start a socket as NULL, send some messages, then change the socket to CURVE ? Does the identity of the socket change in the process ? Cheers, Laurent ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] Support for ZeroMQ/C# on Windows
Hi all, This may be off-topic for development, however we have a request from a ZeroMQ user for commercial support of a ZeroMQ C# application on Windows. If there's a freelancer or small firm here with experience of the C# binding who would like to engage with this user, please contact me directly (or reply to this list). Thanks -- Pieter Hintjens iMatix ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] Differences between ZMTP 3 RFC23 and libzmq master implementation
Hi Pieter, It looks like there are differences between the ZMTP 3 RFC23 ws libzmq master implementation. My test is the same case than the /worked example/ of the RFC: /A DEALER client connects to a ROUTER server. Both client and server are running ZMTP and the implementation has backwards compatibility detection. The peers will use the NULL security mechanism to talk to each other/. 1. /The client sends a partial greeting (11 octets) / : I receive only the 10 first bytes,which are the signature, but not the major version as expected. 2. I receive next a 97 bytes message (The first byte is the major version). In the RFC, this supposed to be two different messages or frames (first one should end with the filler): 3 0 NULL 00 4415 READY 11 Socket-Type 0 0 0 6 DEALER 8 Identity 0 0 0 0 M m mecha as-server filler flags size Cmd name Prop Name Prop Val Prop Name Prop Val 0 1 2-21 22 23-5354 5556-61 62-7374-83 84-92 93-96 10 11 12-31 32 33-6364 This raises a few questions : 1. I have not and won't test the version negotiation. does it work ? 2. test_stream.cpp works in conformance with what I have monitored. What is the baseline, the RFC or the implementation ? Cheers, Laurent ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Question about identities
Only ROUTERs care about identities. My understanding is that normally, a NULL ID is sent to the ROUTER at connection time, which then assigns an identity to that connection (I assume this is stored in a lookup table, where one column is the connection, and the other is the ID). If the DEALER has explicitly set an ID, then that is sent at connection time and the ROUTER stores it instead of a random one. The DEALER doesn't do anything else with the ID. The ROUTER then uses the lookup table when it gets messages with a routing envelope containing an ID to figure out which connection to route the message to. Please correct me if I'm wrong about this. :) AJ On Tue, Dec 10, 2013 at 11:38:33AM +0100, Laurent Alebarde wrote: I did read ZMTP 3 RFC23 several times, but it is not so obvious, even with the Worked Example. In particular if two clients connect in the same time to a ROUTER server. As the identity is communicated in the metadata at the end of the handcheck, how the server distinguishes the two clients ? Is there some temporary identities ? If yes, on what is it built ? Besides, setting an identity is an option. So a DEALER talking to a ROUTER SHALL set an arbitrary one. But it knows it talks to a ROUTER after having sent its identity. So it doesn't know it talks to a ROUTER when it sends its identity, and therefore, it can send an empty one. I remain a bit confused. Le 10/12/2013 10:17, Pieter Hintjens a écrit : Laurent, please read the ZMTP v3 protocol spec with regard to identities. A tells B, this is my identity. B then uses that value to refer to A, IF AND ONLY IF B IS A ROUTER. More usually, A says I have no identity and B assigns an internal numeric identity. On Mon, Dec 9, 2013 at 3:59 PM, Laurent Alebarde l.aleba...@free.fr wrote: Hi Devs, Are the identities of two different sockets on the same peer the same, viewed by one single other peer ? Say: Node 1 Node 2 Socket A1 (NULL) -- Socket A2 (NULL) : Node 1 has identity I1a Socket B1 (CURVE) -Socket B2 (CURVE) : Node 1 has identity I1b Is I1a == I1b ? Cheers, Laurent ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev -- AJ Lewis Software Engineer Quantum Corporation Work:651 688-4346 email: aj.le...@quantum.com -- The information contained in this transmission may be confidential. Any disclosure, copying, or further distribution of confidential information is not permitted unless such privilege is explicitly granted in writing by Quantum. Quantum reserves the right to have electronic communications, including email and attachments, sent across its networks filtered through anti virus and spam software programs and retain such messages in order to comply with applicable data security and retention requirements. Quantum is not responsible for the proper and complete transmission of the substance of this communication or for any delay in its receipt. ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] What exact networking problems ZMQ does solve?
Hi community, Can you please itemize what exact networking issues ZMQ does solve? I have count 1 -- reconnection. Ok. But can you please provide more? BR -artemv ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] What exact networking problems ZMQ does solve?
The guide covers many, many use cases: http://zguide.zeromq.org/ -Michel On Tue, Dec 10, 2013 at 7:59 AM, artemv zmq artemv@gmail.com wrote: Hi community, Can you please itemize what exact networking issues ZMQ does solve? I have count 1 -- reconnection. Ok. But can you please provide more? BR -artemv ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] What exact networking problems ZMQ does solve?
az Can you please itemize what exact networking issues ZMQ does solve? az I have count 1 -- reconnection. Ok. But can you please provide more? Have you read the bullet points at http://zeromq.org/, or skimmed http://zeromq.org/intro:read-the-manual or the preface in the guide (http://zguide.zeromq.org/page:all)? -- Gregg ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Blocking issues with signaler_t::make_fdpair
Sorry for my late reply, been sick for a few days. I've done some tests using the make_fdpair from the master, and it seems like using the ephemeral port support and avoiding the locking solved it. Thanks! But I do believe that if supporting a fixed signaler port is still desired, we should better protect against the scenarios I've described in my first mail. What do you think? Koby On Tue, Dec 10, 2013 at 12:37 AM, KIU Shueng Chuan nixch...@gmail.comwrote: I believe no permission is needed to do a pull request. :) Upon rereading Koby's mail more closely, his problem can be reproduced by having one background program use version 3.2.2. The leaked event handle ensures that the global event stays alive and doesn't get recreated each time by Windows. On Dec 10, 2013 2:44 AM, Felipe Farinon felipe.fari...@powersyslab.com wrote: As Koby didn't answered, and I am not able to reproduce the problem anymore, could I make the modification even being unable to reproduce the problem (indirectly it will be tested, since I am going to run the modification in the same environment where the problem was happening)? Em 01/12/2013 21:27, KIU Shueng Chuan escreveu: In master, you can switch to using ephemeral ports by modifying signaler_port to 0 in config.hpp. A new ephemeral port is used per make_fdpair call and no critical section is used. Could you try that and see if it solves your problems? On Dec 1, 2013 9:39 PM, Koby Boyango kob...@mce-sys.com wrote: Hi I'm fairly new to ZeroMQ, and have been working on integrating it using czmq in several projects, Windows only. I've opened an issue on GitHub*, *#767, and to Pieter's request I'm moving the discussion here. So here is what I've written there: While trying to integrate ZeroMQ in different modules\processes (Windows only), I've encountered a problem where in some situations a ZeroMQ call blocks - forever. After debugging the issue, I've found out that zmq_init wasn't returning, and after further debugging and digging through the code I've found out that the problem was in signaler_t::make_fdpair, where the WaitForSingleObject on the zmq-signaler-port-sync didn't return. Initially i wasn't sure in which situations it occurs. So I did some further investigation and found out that in my case: - For some reason, when I close a test program with Ctrl+C, the event stays un-signaled. Not sure why yet, will need further debugging. - I had a node.js script, which uses ZeroMQ, running in the background. Because it uses version 3.2.2 of libzmq, which leaks the event handle, the existing event wasn't deleted, and stayed in an un-signaled state. - Basically, from that point no one on the system can use ZeroMQ. I find make_fdpair to be very problematic on Windows: - If one call exits without signaling the event, while someone else is holding a handle to the event - All further calls on the system will block. It can happen, for example, if an assertion fails, and the process crashes because of the exception raised. - It can also happen if an assertion has failed, an exception was raised, but caught by the caller using a __try __except block (SEH). We can't simply rely on the exception to crash the process (for example, a program might wrap calls to its plugins with __try __except, so a faulty plugin won't crash the while program). - So it basically means that one faulty program can cause other, unrelated programs, to block. I suggest: - No matter which synchronization mechanism is used, wrap the code with __try __finally, and release the lock in the finally block. This will make sure that we'll release in case of an exception (In my case, though, I tried it and it didn't help. the thread might be terminated during the call). - If possible, don't use a global, system wide, lock. From my understanding, it is used in order to reuse the signaler port. So either use a random, available, port, or make the port libzmq instance specific (the first calls binds on a random port, further calls will reuse the port) and protect it with critical section. This will at least limit the problems to the same process. - If the system wide lock is really needed, I suggest using a mutex instead of the event. When using a mutex, if the owning thread dies without releasing it, Windows automatically releases it and the next call to WaitForSingleObject will return WAIT_ABANDONED, and do not block. We can than check if the port was left in a listening state, close it if necessary, and re-listen with a new socket. I'm using libzmq 4.0.1 with czmq 2.0.2. I saw that the make_fdpair was improved in the master, but I believe it still doesn't entirely solve it. What do you say? Koby ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org
Re: [zeromq-dev] What exact networking problems ZMQ does solve?
Yes I read a guide. But I want to ask question and get answer from real ppl. based on real use cases. Ok. Let me re-phrase a bit. - how one can benefit from automatic reconnection? - how one can benefit from HWM and in-memory message queueing if he/she develops peer-to-peer transactional appl.? - if peer is not connected and we send message there, why it's good idea to keep message in memory? where it helps? - can I use ZMQ if I want develop something akin to http server? if this server is down, then can I get exception that I can't send him a msg? BR -artemv 2013/12/10 Gregg Irwin gr...@pointillistic.com az Can you please itemize what exact networking issues ZMQ does solve? az I have count 1 -- reconnection. Ok. But can you please provide more? Have you read the bullet points at http://zeromq.org/, or skimmed http://zeromq.org/intro:read-the-manual or the preface in the guide (http://zguide.zeromq.org/page:all)? -- Gregg ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Can I change a socket mechanism after a connect ?
The identity is set in stream.cpp / zmq::stream_t::identify_peer from a counter. So, there is no possibility to hack it since other sockets may be interleaved. It is not built with masks, or possibly the base number, I have still some hopes here. Otherwise, I assume that's the connect which is at the origin of that process. Then I may find a simplier hack in avoiding to disconnect/reconnect the socket and find how to force a new handcheck, providing that the new settings (CURVE) will be taken into account. Advices are welcome. Le 10/12/2013 18:38, Laurent Alebarde a écrit : When I disconnect and reconnect after having changed the mechanism to CURVE, the identity is changed. As I need the same identity as of the beginning of the handcheck, I cannot wait its setting from the metadata information, set by the socket identity option. So both solutions cannot work. The three last solutions for me are: 1. Move to ZMTP 3.1 with big changes to enable CURVE proxying - I won't go that way myself. 2. Use libcurve instead of ZMTP-CURVE - Complex with all the updates I would have to perform in my application. 3. Ack the default identity to make it independant of the mechanism - I will try this one. Le 10/12/2013 11:45, Laurent Alebarde a écrit : Thanks Pieter for your reply. nor is there any sense in it : The usual opinion is that when one invent the wheel, he doesn't know 90% of the usefull applications others will do with it. In my use case, it concerns curve proxying. Registering workers under NULL before making the proxy active with CURVE is simplier. So, may I assume it is alright if I I zmq_disconnect, change the security options of the socket, and re-zmq_connect ? Other solution from our other discussion, may I assume I can use a specific NULL socket between the worker and the proxy, just for registration, and setting the same identities on both sockets, the NULL one for registration, and the CURVE one that shall be proxified ? Le 10/12/2013 10:15, Pieter Hintjens a écrit : The security mechanism, like some other socket options, applies to the next connection (in or outgoing). A single TCP connection cannot switch between mechanisms. There's no protocol for that, nor is there any sense in it. Changing security level on a socket after having connections is also dangerous and we've discussed banning it. On Mon, Dec 9, 2013 at 3:24 PM, Laurent Alebardel.aleba...@free.fr wrote: Hi Devs, Is it possible to start a socket as NULL, send some messages, then change the socket to CURVE ? Does the identity of the socket change in the process ? Cheers, Laurent ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Problem installing python language binding on openwrt
Unfortunately, no compiler is available on OpenWRT platform. All the C
packages can be cross compiled with OpenWRT toolchain on another platform.
I have zeromq library crosscompiled and installed on OpenWRT. It was tested
on my hardware with test programs and it works.
Now I only need to install Python binding on OpenWRT in order to use zeromq
library from Python programs. As far as I understand, Python binding is not
pure Python and needs other C/C++ code to be compiled for successful
installation. Can somebody specify which parts I should crosscompile for
Python binding? Which changes needed in setup.py script to have everything
installed correctly?
On Tue, Dec 10, 2013 at 2:23 AM, MinRK benjami...@gmail.com wrote:
what C compiler do you have available? You may need to:
export CC=actual_c_compiler
because Python tries to compile extensions with the compiler used for
Python itself by default (often not available if cross compiled)
On Mon, Dec 9, 2013 at 11:15 AM, Roman consulttele...@gmail.com wrote:
Here is the output of running DISTUTILS_DEBUG=1 python setup.py
install on my device
root@OpenWrt:/home# cd pyzmq-14.0.1/
root@OpenWrt:/home/pyzmq-14.0.1# ls
AUTHORS.md PKG-INFOdocssetup.py
CONTRIBUTING.md README.md examplessetupegg.py
COPYING.BSD build perftox.ini
COPYING.LESSER buildutils setup.cfg.android zmq
MANIFEST.in bundled setup.cfg.template zmqversion.py
root@OpenWrt:/home/pyzmq-14.0.1# DISTUTILS_DEBUG=1 python setup.py
install
options (after parsing config files):
options (after parsing command line):
option dict for 'install' command:
{}
running install
Distribution.get_command_obj(): creating 'install' command object
pre-finalize_{unix,other}:
prefix: None
exec_prefix: None
home: None
user: 0
install_base: None
install_platbase: None
root: None
install_purelib: None
install_platlib: None
install_lib: None
install_headers: None
install_scripts: None
install_data: None
compile: None
compile: True
optimize: None
force: None
skip_build: 0
record: None
post-finalize_{unix,other}():
prefix: /usr
exec_prefix: /usr
home: None
user: 0
install_base: /usr
install_platbase: /usr
root: None
install_purelib: $base/lib/python$py_version_short/site-packages
install_platlib: $platbase/lib/python$py_version_short/site-packages
install_lib: None
install_headers: $base/include/python$py_version_short/$dist_name
install_scripts: $base/bin
install_data: $base
compile: None
compile: True
optimize: None
force: None
skip_build: 0
record: None
post-expand_basedirs():
prefix: /usr
exec_prefix: /usr
home: None
user: 0
install_base: /usr
install_platbase: /usr
root: None
install_purelib: $base/lib/python$py_version_short/site-packages
install_platlib: $platbase/lib/python$py_version_short/site-packages
install_lib: None
install_headers: $base/include/python$py_version_short/$dist_name
install_scripts: $base/bin
install_data: $base
compile: None
compile: True
optimize: None
force: None
skip_build: 0
record: None
config vars:
{'base': '/usr',
'dist_fullname': 'pyzmq-14.0.1',
'dist_name': 'pyzmq',
'dist_version': '14.0.1',
'exec_prefix': '/usr',
'platbase': '/usr',
'prefix': '/usr',
'py_version': '2.7.3',
'py_version_nodot': '27',
'py_version_short': '2.7',
'sys_exec_prefix': '/usr',
'sys_prefix': '/usr',
'userbase': '/root/.local',
'usersite': '/root/.local/lib/python2.7/site-packages'}
post-expand_dirs():
prefix: /usr
exec_prefix: /usr
home: None
user: 0
install_base: /usr
install_platbase: /usr
root: None
install_purelib: /usr/lib/python2.7/site-packages
install_platlib: /usr/lib/python2.7/site-packages
install_lib: None
install_headers: /usr/include/python2.7/pyzmq
install_scripts: /usr/bin
install_data: /usr
compile: None
compile: True
optimize: None
force: None
skip_build: 0
record: None
after prepending root:
prefix: /usr
exec_prefix: /usr
home: None
user: 0
install_base: /usr
install_platbase: /usr
root: None
install_purelib: /usr/lib/python2.7/site-packages
install_platlib: /usr/lib/python2.7/site-packages
install_lib: /usr/lib/python2.7/site-packages/
install_headers: /usr/include/python2.7/pyzmq
install_scripts: /usr/bin
install_data: /usr
compile: None
compile: True
optimize: None
force: None
skip_build: 0
record: None
Distribution.get_command_obj(): creating 'build' command object
running build
running build_py
Distribution.get_command_obj(): creating 'build_py' command object
running build_ext
Distribution.get_command_obj(): creating 'build_ext' command object
running configure
Distribution.get_command_obj(): creating
Re: [zeromq-dev] Blocking issues with signaler_t::make_fdpair
Maybe it's time to switch to ephemeral ports again. Em 10/12/2013 14:42, Koby Boyango escreveu: Sorry for my late reply, been sick for a few days. I've done some tests using the make_fdpair from the master, and it seems like using the ephemeral port support and avoiding the locking solved it. Thanks! But I do believe that if supporting a fixed signaler port is still desired, we should better protect against the scenarios I've described in my first mail. What do you think? Koby On Tue, Dec 10, 2013 at 12:37 AM, KIU Shueng Chuan nixch...@gmail.com mailto:nixch...@gmail.com wrote: I believe no permission is needed to do a pull request. :) Upon rereading Koby's mail more closely, his problem can be reproduced by having one background program use version 3.2.2. The leaked event handle ensures that the global event stays alive and doesn't get recreated each time by Windows. On Dec 10, 2013 2:44 AM, Felipe Farinon felipe.fari...@powersyslab.com mailto:felipe.fari...@powersyslab.com wrote: As Koby didn't answered, and I am not able to reproduce the problem anymore, could I make the modification even being unable to reproduce the problem (indirectly it will be tested, since I am going to run the modification in the same environment where the problem was happening)? Em 01/12/2013 21:27, KIU Shueng Chuan escreveu: In master, you can switch to using ephemeral ports by modifying signaler_port to 0 in config.hpp. A new ephemeral port is used per make_fdpair call and no critical section is used. Could you try that and see if it solves your problems? On Dec 1, 2013 9:39 PM, Koby Boyango kob...@mce-sys.com mailto:kob...@mce-sys.com wrote: Hi I'm fairly new to ZeroMQ, and have been working on integrating it using czmq in several projects, Windows only. I've opened an issue on GitHub*, *#767**, and to Pieter's request I'm moving the discussion here. So here is what I've written there: While trying to integrate ZeroMQ in different modules\processes (Windows only), I've encountered a problem where in some situations a ZeroMQ call blocks - forever. After debugging the issue, I've found out that zmq_init wasn't returning, and after further debugging and digging through the code I've found out that the problem was in signaler_t::make_fdpair, where the WaitForSingleObject on the zmq-signaler-port-sync didn't return. Initially i wasn't sure in which situations it occurs. So I did some further investigation and found out that in my case: * For some reason, when I close a test program with Ctrl+C, the event stays un-signaled. Not sure why yet, will need further debugging. * I had a node.js script, which uses ZeroMQ, running in the background. Because it uses version 3.2.2 of libzmq, which leaks the event handle, the existing event wasn't deleted, and stayed in an un-signaled state. * Basically, from that point no one on the system can use ZeroMQ. I find make_fdpair to be very problematic on Windows: * If one call exits without signaling the event, while someone else is holding a handle to the event - All further calls on the system will block. It can happen, for example, if an assertion fails, and the process crashes because of the exception raised. * It can also happen if an assertion has failed, an exception was raised, but caught by the caller using a __try __except block (SEH). We can't simply rely on the exception to crash the process (for example, a program might wrap calls to its plugins with __try __except, so a faulty plugin won't crash the while program). * So it basically means that one faulty program can cause other, unrelated programs, to block. I suggest: * No matter which synchronization mechanism is used, wrap the code with __try __finally, and release the lock in the finally block. This will make sure that we'll release in case of an exception (In my case, though, I tried it and it didn't help. the thread might be terminated during the call). * If possible, don't use a global, system wide, lock. From my understanding, it is used in order to reuse the signaler port. So either use a random, available, port, or make
[zeromq-dev] core dump when running zeromq example
I am trying to run zeromq java example like hwserver. I have followed all the instructions. # # A fatal error has been detected by the Java Runtime Environment: # # SIGSEGV (0xb) at pc=0x7f4abfbced00, pid=24332, tid=139959612520192 # # JRE version: 7.0_06-b24 # Java VM: Java HotSpot(TM) 64-Bit Server VM (23.2-b09 mixed mode linux-amd64 compressed oops) # Problematic frame: # C [libzmq.so.3+0x2cd00] zmq::socket_base_t::process_term(int)+0x70 # I installed zmq, libmq and jzmq as stated. While running the example getting core dump. I am actually stuck now. Any help is really appreciated. -- ~suchi -- ~suchi To the optimist, the glass is half full.To the pessimist, the glass is half empty.To the engineer, the glass is twice as big as it needs to be. ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Blocking issues with signaler_t::make_fdpair
I think the code could be modified such that if the signaler port is 5905, then the old event mechanism is used, else if it is not 0, then a new mutex mechanism would be used. Signaler port equal to 0 would be the new default. People who still prefer fixed port would choose their own port of choice other than 5905. It's still a compile time choice though. If the default use of ephemeral ports works, don't think it's worth the effort to make it runtime configurable. On Dec 11, 2013 12:43 AM, Koby Boyango kob...@mce-sys.com wrote: Sorry for my late reply, been sick for a few days. I've done some tests using the make_fdpair from the master, and it seems like using the ephemeral port support and avoiding the locking solved it. Thanks! But I do believe that if supporting a fixed signaler port is still desired, we should better protect against the scenarios I've described in my first mail. What do you think? Koby On Tue, Dec 10, 2013 at 12:37 AM, KIU Shueng Chuan nixch...@gmail.comwrote: I believe no permission is needed to do a pull request. :) Upon rereading Koby's mail more closely, his problem can be reproduced by having one background program use version 3.2.2. The leaked event handle ensures that the global event stays alive and doesn't get recreated each time by Windows. On Dec 10, 2013 2:44 AM, Felipe Farinon felipe.fari...@powersyslab.com wrote: As Koby didn't answered, and I am not able to reproduce the problem anymore, could I make the modification even being unable to reproduce the problem (indirectly it will be tested, since I am going to run the modification in the same environment where the problem was happening)? Em 01/12/2013 21:27, KIU Shueng Chuan escreveu: In master, you can switch to using ephemeral ports by modifying signaler_port to 0 in config.hpp. A new ephemeral port is used per make_fdpair call and no critical section is used. Could you try that and see if it solves your problems? On Dec 1, 2013 9:39 PM, Koby Boyango kob...@mce-sys.com wrote: Hi I'm fairly new to ZeroMQ, and have been working on integrating it using czmq in several projects, Windows only. I've opened an issue on GitHub*, *#767, and to Pieter's request I'm moving the discussion here. So here is what I've written there: While trying to integrate ZeroMQ in different modules\processes (Windows only), I've encountered a problem where in some situations a ZeroMQ call blocks - forever. After debugging the issue, I've found out that zmq_init wasn't returning, and after further debugging and digging through the code I've found out that the problem was in signaler_t::make_fdpair, where the WaitForSingleObject on the zmq-signaler-port-sync didn't return. Initially i wasn't sure in which situations it occurs. So I did some further investigation and found out that in my case: - For some reason, when I close a test program with Ctrl+C, the event stays un-signaled. Not sure why yet, will need further debugging. - I had a node.js script, which uses ZeroMQ, running in the background. Because it uses version 3.2.2 of libzmq, which leaks the event handle, the existing event wasn't deleted, and stayed in an un-signaled state. - Basically, from that point no one on the system can use ZeroMQ. I find make_fdpair to be very problematic on Windows: - If one call exits without signaling the event, while someone else is holding a handle to the event - All further calls on the system will block. It can happen, for example, if an assertion fails, and the process crashes because of the exception raised. - It can also happen if an assertion has failed, an exception was raised, but caught by the caller using a __try __except block (SEH). We can't simply rely on the exception to crash the process (for example, a program might wrap calls to its plugins with __try __except, so a faulty plugin won't crash the while program). - So it basically means that one faulty program can cause other, unrelated programs, to block. I suggest: - No matter which synchronization mechanism is used, wrap the code with __try __finally, and release the lock in the finally block. This will make sure that we'll release in case of an exception (In my case, though, I tried it and it didn't help. the thread might be terminated during the call). - If possible, don't use a global, system wide, lock. From my understanding, it is used in order to reuse the signaler port. So either use a random, available, port, or make the port libzmq instance specific (the first calls binds on a random port, further calls will reuse the port) and protect it with critical section. This will at least limit the problems to the same process. - If the system wide lock is really needed, I suggest using a mutex instead of the event. When using a mutex, if the owning thread dies without releasing
Re: [zeromq-dev] What exact networking problems ZMQ does solve?
Hi Bruno, On Wed, Dec 11, 2013 at 12:38 AM, Bruno D. Rodrigues bruno.rodrig...@litux.org wrote: Xsub xpub for distribution load balancing What is Xsub xpub? Could you please explain these terms or give me a link to an online resource where these terms are explained in detail? Thanks in advance, -Asif ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
