Hi Pieter, Thanks a lot for your help. I have done some further tests and it seems that setting the HWM on the server to some higher value generally helps, but setting it to 20 is not always enough. Setting it to 30 seems to work, but I am not sure if that is deterministic.
What would you say a value that always guarantees proper functioning would be, until the issue is fixed? Would this value only depend on the size of the PIPELINE, rather than on other random factors as well? Thanks, Stathis -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Pieter Hintjens Sent: Monday, November 19, 2012 9:26 AM To: ZeroMQ development list Subject: Re: [zeromq-dev] Problem with fileserver / fileclient It's definitely timing dependent but the limit isn't two times, it still failed with a HWM 3x higher than the calculated amount. Sending more or fewer frames had no impact, as you say. So let's treat this as a documentation issue for now, and we can think about how to process LWM commands faster in the future. -Pieter On Mon, Nov 19, 2012 at 2:36 PM, Min <[email protected]> wrote: > At the pipe.cpp > > the variables, msgs_written and peers_msgs_read, which are used at the > HWM full calculation are updated only at the last frame. > > My guess is that the peers_msg_reads can be late updated because it is > updated asynchronously. > > For example, the dealer got chunks and sent subsequent requests, but > the router get the new requests before it get 'activate_write'. > The outgoing pipe can grow to twice of PIPELINE at maximum in an > extreme case. > > > Thanks > Min > > > On Mon, Nov 19, 2012 at 10:23 AM, Pieter Hintjens <[email protected]> wrote: >> >> I've updated the doc to make this clear, and will change the examples >> in the Guide accordingly. >> >> Thanks for spotting this problem! >> >> >> https://github.com/hintjens/libzmq/commit/7533ebb33d1cc6f23061e0228bf >> 7eb22713aaa17 >> >> -Pieter >> >> On Mon, Nov 19, 2012 at 10:01 AM, Pieter Hintjens <[email protected]> wrote: >> > Stathis, >> > >> > I've tested this in detail. >> > >> > Tthere seems to be a problem with the HWM in the ROUTER, which is >> > dropping outgoing messages unexpectedly. That is, the HWM setting >> > of >> > 10 isn't actually enough for 10 outgoing messages. >> > >> > What it looks like is that the HWM is in fact per message frame, >> > not per message. So in this case you'd have to use: >> > >> > zsocket_set_hwm (router, PIPELINE * 2); >> > >> > In the router since the identity frame is counted in the HWM >> > calculation. >> > >> > This is a bug IMO, or at least it's rather surprising. >> > >> > -Pieter >> > >> > >> > >> > >> > On Mon, Nov 19, 2012 at 3:13 AM, Stathis Gkotsis >> > <[email protected]> wrote: >> >> Indeed, sorry about this, here are the two links to the code: >> >> >> >> http://pastebin.com/1DLa4e2J >> >> http://pastebin.com/cbJUxJB4 >> >> >> >> Thanks, >> >> Stathis >> >> ________________________________________ >> >> From: [email protected] >> >> [[email protected]] On Behalf Of Min >> >> [[email protected]] >> >> Sent: Sunday, November 18, 2012 7:36 PM >> >> To: ZeroMQ development list >> >> Subject: Re: [zeromq-dev] Problem with fileserver / fileclient >> >> >> >> The two links seem to be identical. >> >> >> >> Could you upload the client code also ? >> >> >> >> Thanks >> >> Min >> >> >> >> >> >> On Mon, Nov 19, 2012 at 1:22 AM, Stathis Gkotsis >> >> <[email protected]<mailto:stathis.gkotsis@commsquare. >> >> com>> >> >> wrote: >> >> Hello, >> >> >> >> I am using zeromq v3.2.1-RC2 on Ubuntu 10.04 / Ubuntu 12.04 >> >> 64bit. I created two simple apps: one fileserver and one >> >> fileclient, based on the "File Transfer" example code in the >> >> documentation >> >> (http://zguide.zeromq.org/page:all#header-151 , model 3). The code >> >> for these two example apps can be found here: >> >> http://pastebin.com/AuGU9EDC and here: >> >> http://pastebin.com/LusbjGCa . As you can see, I use a file of 100KB. >> >> >> >> The problem is that when I start the server and then run the >> >> client multiple times, sometimes it receives less than 102400 >> >> bytes, like: 98304 or >> >> 94208 or 90112 bytes. For example, when I run the client 1000 >> >> times with an interval of 100ms between the runs, for 970 runs it >> >> receives 12400 bytes, while for the rest (30 runs) it receives >> >> less bytes. I am bit puzzled by this, could you help? >> >> >> >> Thanks, >> >> Stathis >> >> _______________________________________________ >> >> zeromq-dev mailing list >> >> [email protected]<mailto:[email protected]> >> >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >> >> >> >> _______________________________________________ >> >> zeromq-dev mailing list >> >> [email protected] >> >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >> _______________________________________________ >> zeromq-dev mailing list >> [email protected] >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev > > > > _______________________________________________ > zeromq-dev mailing list > [email protected] > http://lists.zeromq.org/mailman/listinfo/zeromq-dev > _______________________________________________ zeromq-dev mailing list [email protected] http://lists.zeromq.org/mailman/listinfo/zeromq-dev _______________________________________________ zeromq-dev mailing list [email protected] http://lists.zeromq.org/mailman/listinfo/zeromq-dev
