On Tue, Jun 23, 2009 at 9:09 PM, Greg Stark<[email protected]> wrote: > On Tue, Jun 23, 2009 at 7:54 PM, Kok, Auke<[email protected]> wrote: > >>> * use posix_fadvise(fd,off,len,POSIX_FADV_WILLNEED) instead of >>> readahead(fs,off,len): posix_fadvise is non-blocking until there is >>> room in the queue, so you can get rid of multithreading, and it will >>> better interact with the block layer, allowing it to reorder more >>> requests according to disk placement. >> >> I'm currently contemplating removing all the threading support, because I'm >> not >> seeing an improvement - even on SSD's. > > > We have some experience with this with Postgres. It uses > posix_fadvise() to prefetch buffers and it works quite well. The > effects from re-ordering and parallelism are both very visible on > tests on raid arrays. > > On Linux at least it's entirely non-blocking. If you prefetch too much > the OS will just discard the requests. >
It will shrink too big prefetch requests, but if you prefetch a lot of small chunks, it may block (see mm/fadvise.c and mm/readahead.c). The block layer, with default configuration, accepts at most 256 in-flight requests for a given disk, and the I/O scheduler makes sure that this precious resource is not completely consumed by a single process, so per-process limit is much lower, especially for ioniced ones. > > -- > greg > http://mit.edu/~gsstark/resume.pdf > -- __________________________________________________________________________ dott. Corrado Zoccolo mailto:[email protected] PhD - Department of Computer Science - University of Pisa, Italy -------------------------------------------------------------------------- _______________________________________________ Power mailing list [email protected] http://www.bughost.org/mailman/listinfo/power
