> Am 23.02.2017 um 16:38 schrieb Yann Ylavic <ylavic....@gmail.com>: > > On Wed, Feb 22, 2017 at 6:36 PM, Jacob Champion <champio...@gmail.com> wrote: >> On 02/22/2017 12:00 AM, Stefan Eissing wrote: >>> >>> Just so I do not misunderstand: >>> >>> you increased BUCKET_BUFF_SIZE in APR from 8000 to 64K? That is what you >>> are testing? >> >> >> Essentially, yes, *and* turn off mmap and sendfile. My hope is to disable >> the mmap-optimization by default while still improving overall performance >> for most users. >> >> Technically, Yann's patch doesn't redefine APR_BUCKET_BUFF_SIZE, it just >> defines a new buffer size for use with the file bucket. It's a little less >> than 64K, I assume to make room for an allocation header: >> >> #define FILE_BUCKET_BUFF_SIZE (64 * 1024 - 64) > > Actually I'm not very pleased with this solution (or the final one > that would make this size open / configurable). > The issue is potentially the huge (order big-n) allocations which > finally may hurt the system (fragmentation, OOM...). > > So I'm thinking of another way to achieve the same with the current > APR_BUCKET_BUFF_SIZE (2 pages) per alloc. > > The idea is to have a new apr_allocator_allocv() function which would > fill an iovec with what's available in the allocator's freelist (i.e. > spare apr_memnodes) of at least the given min_size bytes (possibly a > max too but I don't see the need for now) and up to the size of the > given iovec.
Interesting. Not only for pure files maybe. It would be great if there'd be a SSL_writev()...but until there is, the TLS case will either turn every iovec into its own TLS record or one needs another copy before that. This last strategy is used by mod_http2. Since there are 9 header bytes per frame, copying data into a right sized buffer gives better performance. So, it would be nice to read n bytes from a bucket brigade and get iovecs back. Which, as I understand it, you propose? > This function could be the base of a new apr_bucket_allocv() (and > possibly apr_p[c]allocv(), though out of scope here) which in turn > could be used by file_bucket_read() to get an iovec of available > buffers. > This iovec could then be passed to (new still) apr_file_readv() based > on the readv() syscall, which would allow to read much more data in > one go. > > With this the scheme we'd have iovec from end to end, well, sort of > since mod_ssl would be break the chain but still produce transient > buckets on output which anyway will end up in the core_output_filter's > brigade of aside heap buckets, for apr_socket_sendv() to finally > writev() them. > > We'd also have more recycled heap buckets (hence memnodes in the > allocator) as the core_output_filter retains buckets, all with > APR_BUCKET_BUFF_SIZE, up to THRESHOLD_MAX_BUFFER which, if > configurable and along with MaxMemFree, would be the real limiter of > recycling. > So it's also adaptative. > > Actually it looks like what we need, but I'd like to have feedbacks > before I go further the prototype I have so far (quite straightforward > apr_allocator changes...). > > Thoughts? > > > Regards, > Yann. Stefan Eissing <green/>bytes GmbH Hafenstrasse 16 48155 Münster www.greenbytes.de
