Il 03/10/2012 20:17, Anand Avati ha scritto: > On 10/03/2012 11:17 AM, Paolo Bonzini wrote: >> Il 03/10/2012 19:58, Anand Avati ha scritto: >>>> >>>> I think these 3 lines should be removed. We're bypassing the host >>>> buffer cache just by virtue of using a userspace driver, and that's >>>> what >>>> cache=none cares about. >>> >>> O_DIRECT also has an effect on the behavior of the "client side" (the >>> part within the qemu) of Gluster stack as well. I presume the intention >>> of O_DIRECT is to minimize use of memory (whether as host' page cache or >>> buffered data in user space). To that end it is a good idea to leave >>> O_DIRECT flag set. >>> >>> The behavior of whether gluster bricks need to get the O_DIRECT >>> propagated or not is a different issue. We are exploring the possibility >>> of not sending O_DIRECT flag over the wire to mimic NFS behavior. That >>> would be independent of the qemu block driver setting the open flag. >> >> What is the effect of O_DIRECT on the client exactly? > > To avoid caching in the io-cache module, disable read-ahead etc (if > those translators are loaded). The behavior in write-behind is tunable. > You could either disable write-behind entirely (which will happen once > libgfapi supports 0-copy/RDMA) or perform a sliding-window like > size-limited write-behind (defaults to 1MB).
Thanks for the information. I think we need to benchmark it and see what is the actual difference in memory usage vs. performance. The usual reasons for cache=none (enable Linux native AIO + allow migration with shared non-coherent storage) do not apply to gluster. Paolo
