On Mon, 6 Feb 2017, Jacob Champion wrote:
<snip>
Considering the massive amount of caching that's built into the entire HTTP
ecosystem already, O_DIRECT *might* be an effective way to do that (in which
we give up filesystem optimizations and caching in return for a DMA into
userspace). I have a PoC about halfway done, but I need to split my time this
week between this and the FCGI stuff I've been neglecting.
As O_DIRECT bypasses the cache, all your IO will hit your storage
device. I think this makes it useful only in exotic usecases.
For our workload that's using mod_cache with a local disk cache we
still want to use the remaining RAM as a disk read cache, it doesn't
make sense to force disk I/O for files that simply could have been
served from RAM. We typically see 90-95% of the requests served by
mod_cache actually being served from the disk cache in RAM rather than
hitting the local disk cache devices. I'm suspecting the picture is
similar for most occurrances of static file serving, regardless of
using mod_cache etc or not.
O_DIRECT also bypasses any read-ahead logic, so you'll have to do nice
and big IO etc to get good performance.
We've played around with O_DIRECT to optimize the caching process in
our large-file caching module (our backing store is nfs). However,
since all our hosts are running Linux we had much better results with
doing plain reads and utilizing posix_fadvise with POSIX_FADV_WILLNEED
to trigger read-ahead and POSIX_FADV_DONTNEED to drop the original
file from cache when read (as future requests will be served from
local disk cache). We're doing 8MB fadvise chunks to get full
streaming performance when caching large files.
But that's way out of scope for this discussion, I think ;-)
In conclusion, I wouldn't expect any benefits of using O_DIRECT in the
common httpd usecases. That said, I would gladly be proven wrong :)
/Nikke
--
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Niklas Edmundsson, Admin @ {acc,hpc2n}.umu.se | [email protected]
---------------------------------------------------------------------------
"The point is I am now a perfectly safe penguin!" -- Ford Prefect
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
