Perhaps we'd also like to request readahead be included in JDK7? http://linux.die.net/man/2/readahead
On Tue, Jun 16, 2009 at 9:03 AM, Michael McCandless < luc...@mikemccandless.com> wrote: > Hmm... posix_fadvise lets you do this with a file descriptor; this > would be better for Lucene (per descriptor not per mapped region of > RAM) since we could "advise" independent of which FSDir impl is in > use... > > Mike > > On Tue, Jun 16, 2009 at 10:32 AM, Uwe Schindler<u...@thetaphi.de> wrote: > > But to use it, we should change MMapDirectory to also use the mapping > when > > writing to files. I thought about it, it is very simple to implement > (just > > copy the IndexInput and change all gets() to sets()) > > > > ----- > > Uwe Schindler > > H.-H.-Meier-Allee 63, D-28213 Bremen > > http://www.thetaphi.de > > eMail: u...@thetaphi.de > > > >> -----Original Message----- > >> From: Michael McCandless [mailto:luc...@mikemccandless.com] > >> Sent: Tuesday, June 16, 2009 4:22 PM > >> To: java-dev@lucene.apache.org > >> Cc: Alan Bateman; nio-disc...@openjdk.java.net > >> Subject: Re: madvise(ptr, len, MADV_SEQUENTIAL) > >> > >> Lucene could really make use of this method. When a segment merge > >> takes place, we can read & write many GB of data, which without > >> madvise on many OSs would effectively flush the IO cache (thus hurting > >> our search performance). > >> > >> Mike > >> > >> On Mon, Jun 15, 2009 at 6:01 PM, Jason > >> Rutherglen<jason.rutherg...@gmail.com> wrote: > >> > Thanks Alan. > >> > > >> > I cross posted this to the Lucene dev list where we are discussing > using > >> > madvise for minimizing unnecessary IO cache usage when merging > segments > >> > (where we really want the newly merged segments in the IO cache rather > >> than > >> > the old segment files). > >> > > >> > How would the advise method work? Would there need to be a hint in > the > >> > FileChannel.map method? > >> > > >> > -J > >> > > >> > On Mon, Jun 15, 2009 at 12:36 AM, Alan Bateman <alan.bate...@sun.com> > >> wrote: > >> >> > >> >> Jason Rutherglen wrote: > >> >>> > >> >>> Is there going to be a way to do this in the new Java IO APIs? > >> >> > >> >> Good question, as it has come up a few times and is needed for some > >> >> important use-cases. A while back I looked into adding a > >> >> MappedByteBuffer#advise method to allow the application provide hints > >> on the > >> >> expected usage but didn't complete it. We should probably look at > this > >> again > >> >> for jdk7. > >> >> > >> >> -Alan. > >> >> > >> > > >> > > >> > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org > >> For additional commands, e-mail: java-dev-h...@lucene.apache.org > > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org > > For additional commands, e-mail: java-dev-h...@lucene.apache.org > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-dev-h...@lucene.apache.org > >
