[zeromq-dev] Message queuing broker for mobile device
Dear all, I am new to zeromq. I have some questions about message queuing broker (msgqueue.c) which use zmq_proxy() forward msgs between frontend and backend. In my testing code, the mobile client is using ZMQ_DEALER socket to connect to this broker. But after some time, I found there are more and more connected TCP at broker server side, all in ESTABLISHED state. I am sure there are no more than ten clients and even all apps closed. Because there are no way to get low level TCP sockets of broker, so is there any way to clean those DEAD TCP connections at WI-FI or 3G env? regards, Gang ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Understanding Broker
Can the mechanism of load balancing broker be changed so that exactly the same send/recv pattern is followed regardless of which side connects? On Thu, Jan 8, 2015 at 5:57 PM, Pieter Hintjens p...@imatix.com wrote: In the lbbroker example all traffic flows through the broker. On Thu, Jan 8, 2015 at 8:39 PM, Kenneth Adam Miller kennethadammil...@gmail.com wrote: Does the broker demonstrated in the manual under: http://zguide.zeromq.org/page:all#A-Load-Balancing-Message-Broker demonstrate that A) when each end makes a connection request, after they link up from the broker, their messages route directly between one another In this scenario, the messages sent by each side on request will hit the broker, but the replies they send will go directly to one another. or B) that at all times, information is routing via the broker? In this scenario, the requests have to first go to the broker, and then the replies also hit the broker, which it ferries across to each side. Which is the case? ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] lost message due to EINTR
Hi * !
I recently switched from ZMQ2 (pretty old) to ZMQ 4 and
since then have some problems in debugging with EINTR.
Following code:
do {
rc = zmsg_send (zrep, clsocket_);
if (rc0) {
if (errno ==
EINTR || errno == EAGAIN) {
logWarn(temporary failure in zmq send()
... will be tried again.);
} else {
logFatal(hard error in sending
zmq ... manually destroying message ... it will be lost);
zmsg_destroy(zrep);
}
if (zrep) {
logWarn(sending of reply msg
returned rc(rc), zmq_errno(zmq_errno())
zmq_strerror(zmq_errno()));
logWarn(but message is still existent
... retrying);
} else {
logError(sending of reply msg returned
rc(rc), zmq_errno(zmq_errno())
zmq_strerror(zmq_errno()));
logFatal(message nulled anyway by zmq
... seems lost ...);
}
}
} while (zrep); // repeat until message is
gone
This snippet usualy works, but sometimes I get the warning of
EINTR. No problem, I thought, but despite returning an error (rc==-1,
errno==EINTR) the message pointer is NULLed, so I cannot resend the
message.
The Logs prove, that indeed the message is NOT sent, and for
resending I'd need a copy ... what am I doing wrong?
^5
sven
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] How to use zmq_getsockopt with option ZMQ_IDENTITY_FD
Suppose the identity string is only two bytes long, I pad with zeros to get a string of eight bytes. What should the value of option_len be, 2 or 8? -- Peter Kleiweg http://pkleiweg.home.xs4all.nl/ ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] How to use zmq_getsockopt with option ZMQ_IDENTITY_FD
This is an oddball API choice (and there is a bug in the implementation), in that option_value* is being used as both and input and an output parameter. The size you pass in must be be *at least* sizeof(fd_t) bytes, because it will overwrite the supplied identity string with the resulting file descriptor. The bug is, it does not check to see that the size of supplied output is sufficient to hold sizeof(fd_t), so bad things (stack/heap corruption) would happen if you actually passed option_len = 2. On Thu, Jan 8, 2015 at 7:26 AM, Peter Kleiweg pklei...@xs4all.nl wrote: Suppose the identity string is only two bytes long, I pad with zeros to get a string of eight bytes. What should the value of option_len be, 2 or 8? -- Peter Kleiweg http://pkleiweg.home.xs4all.nl/ ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] How to use zmq_getsockopt with option ZMQ_IDENTITY_FD
yes.
using blob_t = std::basic_stringunsigned char;
char buf[8] = { 'f', 'o', 'o', 0, 0, 0, 0, 0 };
blob_t a(foo);
blob_t b(buf, 8);
assert(a == b); // fails
I think there is another issue with all of this. By default (IIRC) the
identity is 5 bytes, so on a 64 bit platform, the passed in identity would
never be sufficient to receive sizeof(fd_t) if fd_t was in fact defined as
a UINT_PTR. I don't have a Windows machine handy so I can't verify what
sizeof(SOCKET) returns, but as many of Window's 'handle' types are in fact
pointers, I suspect this might also be an issue when fd_t is typedef'd to
SOCKET. On Posix systems, it's fine.
On Thu, Jan 8, 2015 at 8:28 AM, Peter Kleiweg pklei...@xs4all.nl wrote:
Thomas Rodgers schreef op de 8e dag van de louwmaand van het jaar 2015:
This is an oddball API choice (and there is a bug in the implementation),
in that option_value* is being used as both and input and an output
parameter.
The size you pass in must be be *at least* sizeof(fd_t) bytes, because it
will overwrite the supplied identity string with the resulting file
descriptor. The bug is, it does not check to see that the size of
supplied
output is sufficient to hold sizeof(fd_t), so bad things (stack/heap
corruption) would happen if you actually passed option_len = 2.
But option_len is used for retrieving the identity string:
blob_t identity= blob_t((unsigned char*)optval_,*optvallen_);
Won't I get a wrong 'blob' if I use option_len = 8?
On Thu, Jan 8, 2015 at 7:26 AM, Peter Kleiweg pklei...@xs4all.nl
wrote:
Suppose the identity string is only two bytes long, I pad with
zeros to get a string of eight bytes. What should the value of
option_len be, 2 or 8?
--
Peter Kleiweg
http://pkleiweg.home.xs4all.nl/
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] new deprecated options
Good question. They are made redundant by authentication via the ZAP protocol. However they're not removed as apps may be using them. Thus, they are deprecated. They should not be mentioned in the NEWS. On Thu, Jan 8, 2015 at 3:40 PM, Peter Kleiweg pklei...@xs4all.nl wrote: ZeroMQ introduces three new options for setsockopt: ZMQ_IPC_FILTER_PID ZMQ_IPC_FILTER_UID ZMQ_IPC_FILTER_GID In the docs, it says these options are deprecated. Why introduce options that are deprecated at the same time? -- Peter Kleiweg http://pkleiweg.home.xs4all.nl/ ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] new deprecated options
ZeroMQ introduces three new options for setsockopt: ZMQ_IPC_FILTER_PID ZMQ_IPC_FILTER_UID ZMQ_IPC_FILTER_GID In the docs, it says these options are deprecated. Why introduce options that are deprecated at the same time? -- Peter Kleiweg http://pkleiweg.home.xs4all.nl/ ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] How to use zmq_getsockopt with option ZMQ_IDENTITY_FD
Thomas Rodgers schreef op de 8e dag van de louwmaand van het jaar 2015: This is an oddball API choice (and there is a bug in the implementation), in that option_value* is being used as both and input and an output parameter. The size you pass in must be be *at least* sizeof(fd_t) bytes, because it will overwrite the supplied identity string with the resulting file descriptor. The bug is, it does not check to see that the size of supplied output is sufficient to hold sizeof(fd_t), so bad things (stack/heap corruption) would happen if you actually passed option_len = 2. But option_len is used for retrieving the identity string: blob_t identity= blob_t((unsigned char*)optval_,*optvallen_); Won't I get a wrong 'blob' if I use option_len = 8? On Thu, Jan 8, 2015 at 7:26 AM, Peter Kleiweg pklei...@xs4all.nl wrote: Suppose the identity string is only two bytes long, I pad with zeros to get a string of eight bytes. What should the value of option_len be, 2 or 8? -- Peter Kleiweg http://pkleiweg.home.xs4all.nl/ ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] How to use zmq_getsockopt with option ZMQ_IDENTITY_FD
Sorry to keep flogging this particular horse, but in re-reading the docs
for this option as I'm trying to clean it up...
NB: _option_value_ must be always big enough to hold sizeof(fd_t) bytes
no matter how small the identity length is.
There are few issues here -
Prior to commit 45c681
https://github.com/zeromq/libzmq/commit/45c68154460b5cc828cb7ac027e5407776bff2ca
there
was no validation that option_len was sufficient to hold sizeof(fd_t), so
in the 'no matter how small' case where the supplied identity length was
less than sizeof(fd_t), which is not something that is part of the public
API, so the user has to 'just know', you may get silent stack/heap
corruption.
With the the check, zmq_getsockopt(ZMQ_IDENTITY_FD, ...) will now return
EINVAL and set option_len to the minimum required size (I think this is
better than the risk of silent corruption). Now you have the following -
In the case where ZeroMQ assigns a default identity, it will have a length
of 5, and on a Posix system, the FD will be an int, sizeof(int) 5, things
are fine. On 64bit Windows however, sizeof(fd_t) = 8. If I set option_len
to 8 on call, but only have 5 bytes, the blob will be constructed with
whatever 8 bytes of data happen to be in option_value, which will then fail
to match a valid identity (std::string can contain any char, is not null
terminated). How would this option ever work in this case?
Now, say a *nix user calls zmq_setsockopt(ZMQ_IDENTITY, ...) and uses two
byte socket identities. You get the same problem trying to pass sizeof(int)
as option_len in that you create a bogus identity to check. How would this
option ever work in this case?
The only 'safe' option seems to be blindly assume that the caller passed
something at least 4 or 8 bytes long, as the code did prior to 45c681
https://github.com/zeromq/libzmq/commit/45c68154460b5cc828cb7ac027e5407776bff2ca,
but then there is NO check against buffer overrun. Default alignment would
likely save us most of the time here, but it makes my spider sense tingle
and if the caller for some reason passed a pointer into a packed struct,
they would not be happy with the result.
On Thu, Jan 8, 2015 at 9:16 AM, Thomas Rodgers rodg...@twrodgers.com
wrote:
yes.
using blob_t = std::basic_stringunsigned char;
char buf[8] = { 'f', 'o', 'o', 0, 0, 0, 0, 0 };
blob_t a(foo);
blob_t b(buf, 8);
assert(a == b); // fails
I think there is another issue with all of this. By default (IIRC) the
identity is 5 bytes, so on a 64 bit platform, the passed in identity would
never be sufficient to receive sizeof(fd_t) if fd_t was in fact defined as
a UINT_PTR. I don't have a Windows machine handy so I can't verify what
sizeof(SOCKET) returns, but as many of Window's 'handle' types are in fact
pointers, I suspect this might also be an issue when fd_t is typedef'd to
SOCKET. On Posix systems, it's fine.
On Thu, Jan 8, 2015 at 8:28 AM, Peter Kleiweg pklei...@xs4all.nl wrote:
Thomas Rodgers schreef op de 8e dag van de louwmaand van het jaar 2015:
This is an oddball API choice (and there is a bug in the
implementation),
in that option_value* is being used as both and input and an output
parameter.
The size you pass in must be be *at least* sizeof(fd_t) bytes, because
it
will overwrite the supplied identity string with the resulting file
descriptor. The bug is, it does not check to see that the size of
supplied
output is sufficient to hold sizeof(fd_t), so bad things (stack/heap
corruption) would happen if you actually passed option_len = 2.
But option_len is used for retrieving the identity string:
blob_t identity= blob_t((unsigned char*)optval_,*optvallen_);
Won't I get a wrong 'blob' if I use option_len = 8?
On Thu, Jan 8, 2015 at 7:26 AM, Peter Kleiweg pklei...@xs4all.nl
wrote:
Suppose the identity string is only two bytes long, I pad with
zeros to get a string of eight bytes. What should the value of
option_len be, 2 or 8?
--
Peter Kleiweg
http://pkleiweg.home.xs4all.nl/
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] Should I use socket option ZMQ_TCP_ACCEPT_FILTER in ZeroMQ 4.0?
The API doc version 4.1 for setsockopt option ZMQ_TCP_ACCEPT_FILTER says this option is deprecated, and you should use the ZAP API instead. The API doc version 4.0 doesn't say this, but the ZAP API is already available. Writing bindings for ZeroMQ 4, should I write in the docs that this option is deprecated since 4.1, or should I just write that it is deprecated? -- Peter Kleiweg http://pkleiweg.home.xs4all.nl/ ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] Understanding Broker
Does the broker demonstrated in the manual under: http://zguide.zeromq.org/page:all#A-Load-Balancing-Message-Broker demonstrate that A) when each end makes a connection request, after they link up from the broker, their messages route directly between one another In this scenario, the messages sent by each side on request will hit the broker, but the replies they send will go directly to one another. or B) that at all times, information is routing via the broker? In this scenario, the requests have to first go to the broker, and then the replies also hit the broker, which it ferries across to each side. Which is the case? ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
