Great blog and lucid explanation I think things have changed in recent kernel versions. I am no expert, but could see some code related to this here http://lxr.free-electrons.com/source/mm/fadvise.c?v=3.14
O_DIRECT will be terrible drag no? Will a battery-backed disk cache help here? We are using a SortingMergePolicy which most-often hits data randomly. Will SEQUENTIAL help here? Any reasons why you think DONTNEED will be less-useful? -- Ravi On Wed, May 21, 2014 at 2:49 PM, Michael McCandless < luc...@mikemccandless.com> wrote: > You're right, segment merges can be catastrophic to ongoing searches. > I explored the problem here: > http://blog.mikemccandless.com/2010/06/lucene-and-fadvisemadvise.html > but a lot has changed since then... > > SEQUENTIAL is probably best (if the OS implements it; I think the > Linux kernel has improved here), because then the OS knows to read > ahead and aggressively free the page once we are done using it. > > There is also O_DIRECT (e.g., using NativeUnixDirectory) for direct IO > to bypass the buffer cache entirely. > > > Mike McCandless > > http://blog.mikemccandless.com > > > On Wed, May 21, 2014 at 4:09 AM, Ravikumar Govindarajan > <ravikumar.govindara...@gmail.com> wrote: > > Is it a good idea to use FADVISE_DONTNEED/MADVISE_DONTNEED flags during > > segment merge reads? > > > > Buffer-Cache contains critical data belonging to searches. A > segment-merge > > has the potential to disturb the cache no? > > > > -- > > Ravi > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-user-h...@lucene.apache.org > >
