Hi again !
I worked on some demontrating testcase and build up the
router code below (see end of this mail).
Please excuse the ugly look
of the code, it was typed without planing and worsened during test ;o)
It is doing the following:
* create a router (binding 2 ZMQ_ROUTER
Oops ... I overlooked, that there's no time between connect and send
in client ;o)
please excuse this silly thing.
Result: I still don't
have an easy test case for my vanished messages ;o((
Am 2015-03-07
13:41, schrieb Sven Koebnick:
Thanks for the fast response ;-)
Did you
with tdmalloc, valgrind, efence and duma, but they all say that
I'm fine with memory access. (or does zmq somehow work arround those memory
libs?)
mit freundlichen Grüßen
Sven Koebnick
E = mc² ± 2dBA - everything is relative
Hi all!
after migration from ZMQ2 to ZMQ4 I had problems with
vanished messages (on REP/REQ as well as on PUB/SUB).
I now MANAGED TO
BUILD SOME DIRTY MINIMAL TESTCASE (see below) displaying at least
one of my problems.
The testcase does following:
- create a router
binding to one SUB
Hi *!
I just read through this thread and would like to share some
thoughts about several things in discussion:
I picked ZMQ to have a
straight forward lib that is easy to handle and makes it easy to both
stick to high level things (application concerned) as well as going into
details where
Hi *!
I found the bug that I thought to be caused by
EINTR:
The EINTR-thingy was gone, when I fixed some illegal refference
usages, that the gcc compiler didn't complain about (but clang of LLVM
did ;o)
... the lost messages problem was still there.
BUT: when I
switched from ZMQ2 to
another related thing buthers me in this context:
When zmsg_send()
indeed returns with rc==-1 and a NULLed message (data is definitely
lost) I have a chance to check for this lost message (simply asserting
on rc==-1 message==NULL).
BUT (!!) what is with zmsg_recv() ?
Situation: I
the referenced fix should work for zmsg_send() but the same bug is
in zmsg_recv() and there the application does not (yet) have any data.
zmsg_recv() destroyes properly received frames 1 to n-1 if receiving
frame n failes. Since the data is ONLY inside ZMQ at this time, the
application cannot
I get that error only during debuging inside Eclipse C++ (gdb).
It
seems irrelevant, if there are breakpoints set or not (but if there are
any breakpoints, that stop the program, the number of EINTR is
significantly higher).
I also wonder about getting EINTR because of a
debugger. Thet is
I now use some code doublicate of your cmzq code, that does frame
send()ing with REUSE and retries in case of EINTR:
I copied
zmsg_recv() and wrapped the frame receiving in a loop checking EINTR.
zmsg_t *zmsg_recv (void *source)
{
assert (source);
zmsg_t *self =
zmsg_new ();
if (!self)
Hi * !
I recently switched from ZMQ2 (pretty old) to ZMQ 4 and
since then have some problems in debugging with EINTR.
Following code:
do {
rc = zmsg_send (zrep, clsocket_);
if (rc0) {
if (errno ==
EINTR || errno == EAGAIN) {
logWarn(temporary failure in zmq send()
... will be tried
Hi all and a merry christmas!
I recently switched from ZMQ V2
(self compiled C++) to V4 installed as part of Linux-Distribution.
After several days of fixing the modified interfaces (classes formerly
delivered as helper.cpp and now are separately packaged etc.) the
system compiled fine and
there is a queue in zmq that you can use as template for own mods.
I took it years ago for following functionality:
- creating a queue
device, supplying two endpoints that others can connect to (one bcast,
one rep/req)
- creating two internal endpoints, one for broadcasting,
one for req/rep
Hi @ all!
I'm developing against ZMQ for quite some time now and the distributed
service architechture is ready and working stable (also under heavy
load).
(No, I din't update to the latest version because the behaviour doesn't
look like an accidential bug)
While hardening the system, a
you have to:
- create a zmq msg with sufficient size
- copy the struct into the msg buffer
This means, that you need a func, that knows the struct and copies the data of
the inner pointers etc.
In short: you need to write a serialization function for your struct.
^5
Sven
===
sent
Hi Martin,
the main issue is, that poll(... ,-1) does not at all return despite
the fact that there is a message available!
The usage of timeout value is a current workaround, because the second
call to poll (the first returned zero) successfully returns a number
greater one.
Sven
Hi
all!
I have a queer behaviour in my services build on zmq. I tried to
write a minimal testcase, but failed to rebuild this bug in a smaller
program ;o(Hopefully my description is enough to find the
cause:I have a PUB/SUB socket, where several messages
already have passed successfully. As the
17:50:22 +0100
Subject: [zeromq-dev] poll does not return on a SUB socket
From: Sven Koebnick sven.koebn...@t-online.de
To: zeromq-dev@lists.zeromq.org
Hi all! I have a queer
behaviour in my services build on zmq. I tried to write a minimal
testcase, but failed to rebuild this bug in a smaller
hi
toni,i had a comparable scenario and got this link:
http://zguide.zeromq.org/chapter:all#toc45 (LRU-Queue)
it descrives a technique to know about available workers. It looks a bit
complicated, but only on the first view. the described concept works
fine for me.
^5sven
such
extreme performance drops should not be too difficult to find.My
first guess would be a wrong NIC-config. There is (since years) a kind
of quirk, that (seemly intensionally) guesses the doublex setting wrong
in some constellations. this would perfectly fit to your problem with
fast onw way
did you
set socket ID on both ends? req AND rep?
^5
Sven
-
E = mc² ± 2dBA - everything is relative
-
-Original Message-
Date: Sat, 04 Dec 2010 13:59:45 +0100
Subject:
a few
years ago, I read some artical about cheating IPC into Windows. it
worked with using parts of the paging file as a shared datafile and
thereby simulating shared memory.Maybe this can be wrapped to just
plug it into where IPC sits on real operating systems.
^5
Sven
maybe
this is helpful:in my system, I have dedicated dispatcher
services, that just know, where to forward the different messages.for shutdown, I send a shut up message to all
services, which afterwards refuse do do anything. refused messages are
written to DB by the dispatchers, as well as fresh
talking
about Java-bindings, I wonder (expecting you to think about roasting me
slowly on a small fire ;-) how much work it would be to write a PURE
java implementation ... without JNI or other ways to break an applet.my current (already accepted) design has a small translater that
works a gateway
what's
wrong in just pluging in some additional functions on lowest protocol
level implementing ipsec stuff ?surely, this does not deliver
application level security, but this should anyway (IMHO) be implemented
clearly on top of zmq (well, there could be some (also ontop) libs
helping zmq-apps to
wouldn't
it be easiest to just subscribe to regex's instead of fixed prefix
strings?that shouldn't be too much effort to build into the
subscription filter and would also allow a big deal of other fine
stuff.
^5sven
-
E = mc² ± 2dBA -
Jaime,I stumbled over the same thing, but alas, at
a state, where I already had several thousand lines of code arround my
service frame ;o(The sockets do not really make balancing, they
round robin through all connected servers. When I asked
about this (here), I got this REALLY helpful link:
Hey,
great! just a few hours ago, I tagged a make a shell tool to
my TODOs. I'm gonna post my extension to your code since I'm using
google protocol buffers as data format.Sven
-
E = mc² ± 2dBA - everything is relative
have a
look at the overloaded socket_t::operator void* (). I also stumbled over
this tiny difference when modifying a device.
sven
-
E = mc² ± 2dBA - everything is relative
-
just a
thought: isn't subscription forwarding more or less, what multicast hubs
do? sounds to me like reinventing the wheel under a new name.
rgdssven
-
E = mc² ± 2dBA - everything is relative
-
-Original Message-
Date: Wed, 20 Oct 2010 09:32:12 +0200
Subject: Re: [zeromq-dev] loadbalancing problem?
From: Martin Sustrik sust...@250bpm.com
To: Sven Koebnick sven.koebn...@t-online.de, ZeroMQ development
list zeromq-dev@lists.zeromq.org
Hi Sven
Hi
Nir!
I had a look at the LRU method and found that it propably destroys the
routing information.Data runs like this:- someone send new
msg to dispatcher queue device- queue device has several workers
and send msg to one- the dispatcher worker forwards the message to
the proper service (again
-
-Original Message-
Date: Wed, 20 Oct 2010 15:47:15 +0200
Subject: Re: [zeromq-dev] loadbalancing problem?
From: Martin Sustrik sust...@250bpm.com
To: Sven Koebnick sven.koebn...@t-online.de, ZeroMQ development
list zeromq-dev@lists.zeromq.org
Sven,
I saw, there already is some technique
Hi
all!I have some tricky problem, which I guess is caused by
the balancing algorhythm of the XREQ sockets in a queue device.following situation:The system has several service types, each
having several worker threads on one queue device per service type.One of these service types is a dispatcher,
Imho you shouldnt invest too much time in getting dists sound and safe
for everybody. S'one that needs zmq surely has no prob in upgrading
one or two pkgs. Think about the typical help text in kernal config:
if you dont know what this is, you propably dont need it
^5
Sven
===
that
reminds me of my recently reported problem, that poll returns wrong
codes.I thought, that it's been completely my fault, because I
accidentially called poll() twice on a socket before recv()ing, BUT:the second call to poll() should still tell me, that data is
availably, which does not happen
when I
stripped the system down for a short demo, I found a bug (of mine) that
caused a second poll when it's been clear (by the first
poll), that data had arrived.
== Sorry, my fault!
^5
Sven
-
E = mc² ± 2dBA - everything is relative
IMHO it
should be decided by the developer of the application, if different
message type/streams can sensefully share one stream. this should not be
decided by the developer of a used lib (e.g. zmq).if the feature
is simply not implemented (yet) it's OK, but I also think it's a fine
feature.But it
38 matches
Mail list logo
