Indeed, some environments can't use heap memory, so custom allocators into pre-allocated memory could be helpful. One thing at a time though.
For FAST I would assume that after say 1 Mb there is no measurable benefit in increasing block size, no matter the rates and message sizes. It's probably lower than that. On Tue, Jul 14, 2015 at 9:11 AM, Auer, Jens <[email protected]> wrote: > Hi Pieter, > > How would we come up with a value for FAST? As I see it, the value depends on > - the data rate (msg/s) > - the message size > - the amount of memory the developer wants to spend > > We can define a small struct as parameter to the new socket option with data > rate, message size and the maximum memory available to zeroMQ, and then > decide internally how much space we allocate. But I still it is hard to come > up with a general rule. > > I also thought about new socket options to set allocation/deallocation > functions. This would allow users to use a memory pool for the buffers, but > it is a very risky option with a lot of potential for problems. On the other > hand, many people working on systems with resource constraints would love it. > What is your opinion on this? > > Best wishes, > Jens > > -- > Dr. Jens Auer | CGI | Software Engineer > CGI Deutschland Ltd. & Co. KG > Rheinstraße 95 | 64295 Darmstadt | Germany > T: +49 6151 36860 154 > [email protected] > Unsere Pflichtangaben gemäß § 35a GmbHG / §§ 161, 125a HGB finden Sie unter > de.cgi.com/pflichtangaben. > > CONFIDENTIALITY NOTICE: Proprietary/Confidential information belonging to CGI > Group Inc. and its affiliates may be contained in this message. If you are > not a recipient indicated or intended in this message (or responsible for > delivery of this message to such person), or you think for any reason that > this message may have been addressed to you in error, you may not use or copy > or deliver this message to anyone else. In such case, you should destroy this > message and are asked to notify the sender by reply e-mail. > > > -----Original Message----- > From: [email protected] > [mailto:[email protected]] On Behalf Of Pieter Hintjens > Sent: 13 July 2015 23:44 > To: ZeroMQ development list > Subject: Re: [zeromq-dev] New option to set receive buffer for tcp > > A socket option is a good idea. Is there any way we can abstract this so > users don't need to calculate buffer sizes (they will get it wrong often > enough)? E.g. _SMALL vs. _FAST? > > On Mon, Jul 13, 2015 at 5:29 PM, Auer, Jens <[email protected]> wrote: >> Hi, >> >> >> >> I ran some benchmarks with the current master to see if the zero-copy >> changes have an effect on the performance in my system. As it turns >> out, in my setup, the effect is not so large because of the number of >> malloc calls still being done. In my setup, I send and receive data >> streams with up to >> 200000 messages per second with a message size of 1125 bytes. The >> receive buffer is fixed to 8kb, so I need one allocation for every 7 >> messages, and one deallocation. I experimentally increased the size of >> the receive buffer to 65kb, and this resulted in a good performance >> gain, basically trading memory (which I have lots of) for speed. What >> is the opinion on a new socket option to let users set the size of the >> receive buffer? I think the size depends on the expected data rate and >> message size, so it seems fair for users to be able to influence it. >> >> >> >> Best wishes, >> >> Jens >> >> >> >> -- >> >> Dr. Jens Auer | CGI | Software Engineer >> >> CGI Deutschland Ltd. & Co. KG >> Rheinstraße 95 | 64295 Darmstadt | Germany >> >> T: +49 6151 36860 154 >> >> [email protected] >> >> Unsere Pflichtangaben gemäß § 35a GmbHG / §§ 161, 125a HGB finden Sie >> unter de.cgi.com/pflichtangaben. >> >> >> >> CONFIDENTIALITY NOTICE: Proprietary/Confidential information belonging >> to CGI Group Inc. and its affiliates may be contained in this message. >> If you are not a recipient indicated or intended in this message (or >> responsible for delivery of this message to such person), or you think >> for any reason that this message may have been addressed to you in >> error, you may not use or copy or deliver this message to anyone else. >> In such case, you should destroy this message and are asked to notify the >> sender by reply e-mail. >> >> >> >> >> _______________________________________________ >> zeromq-dev mailing list >> [email protected] >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >> > _______________________________________________ > zeromq-dev mailing list > [email protected] > http://lists.zeromq.org/mailman/listinfo/zeromq-dev > _______________________________________________ > zeromq-dev mailing list > [email protected] > http://lists.zeromq.org/mailman/listinfo/zeromq-dev _______________________________________________ zeromq-dev mailing list [email protected] http://lists.zeromq.org/mailman/listinfo/zeromq-dev
