On Tue, 21 Feb 2017, Jacob Champion wrote:
Is there interest in more real-life numbers with increasing
FILE_BUCKET_BUFF_SIZE or are you already on it?
Yes please! My laptop probably isn't representative of most servers; it can
do nearly 3 GB/s AES-128-GCM. The more machines we test, the better.
To make results less confusing, any specific patches/branch I should
test? My baseline is httpd-2.4.25 + httpd-2.4.25-deps
--with-included-apr FWIW.
I have an older server
that can do 600 MB/s aes-128-gcm per core, but is only able to deliver
300 MB/s https single-stream via its 10 GBps interface. My guess is too
small blocks causing CPU cycles being spent not delivering data...
Right. To give you an idea of where I am in testing at the moment: I have a
basic test server written with OpenSSL. It sends a 10 MiB response body from
memory (*not* from disk) for every GET it receives. I also have a copy of
httpd trunk that's serving an actual 10 MiB file from disk.
My test call is just `h2load --h1 -n 100 https://localhost/`, which should
send 100 requests over a single TLS connection. The ciphersuite selected for
all test cases is ECDHE-RSA-AES256-GCM-SHA384. For reference, I can do
in-memory AES-256-GCM at 2.1 GiB/s.
- The OpenSSL test server, writing from memory: 1.2 GiB/s
- httpd trunk with `EnableMMAP on` and serving from disk: 850 MiB/s
- httpd trunk with 'EnableMMAP off': 580 MiB/s
- httpd trunk with my no-mmap-64K-block file bucket: 810 MiB/s
At those speeds your results might be skewed by the latency of
processing 10 MiB GET:s.
I'd go for multiple GiB files (whatever you can cache in RAM) and
deliver files from disk. Discard the results from the first warm-up
access and your results delivering from memory or disk (cache)
shouldn't differ.
So just bumping the block size gets me almost to the speed of mmap, without
the downside of a potential SIGBUS. Meanwhile, the OpenSSL test server seems
to suggest a performance ceiling about 50% above where we are now.
I'm guessing that if you redo the tests with a bigger file you should
see even more potential.
As I said, our live server does 600 MB/s aes-128-gcm and can deliver
300 MB/s https without mmap. That's only a factor 2 difference
between aes-128-gcm speed and delivered speed.
Your results above are almost a factor 4 off, so something's fishy :-)
Even with the test server serving responses from memory, that seems like
plenty of room to grow. I'm working on a version of the test server that
serves files from disk so that I'm not comparing apples to oranges, but my
prior testing leads me to believe that disk access is not the limiting factor
on my machine.
Hmm. Perhaps I should just do a quick test with openssl s_server, just
to see what numbers I get...
/Nikke
--
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Niklas Edmundsson, Admin @ {acc,hpc2n}.umu.se | ni...@acc.umu.se
---------------------------------------------------------------------------
BETA testing is hazardous to your health.
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
