But then we also need to map, when writing to files, which is hard to do, because you do not know how large the mapping area will be (unknown filesize). As Earwin suggested, we could change MMapDirectory to also mmap on writing, but maps the buffers in something we call "pages". Filesizes of written files are then always multiples of this "page size", if this is not a problem for lucene (e.g. making the random access file larger than actually needed, having 0-bytes at the unused end). Maybe we need than an api to tell the directory implementation, how large a segment may become before writing (when merging, this should be possible, as the filesize is about the sum of all merged segments).
On windows, we then will also rely on the cleaner()-hack to correctly close files (see current MMap in lucene-trunk, where the cleaner-hack can optionally be enabled, without guaranties). ----- Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de _____ From: Jason Rutherglen [mailto:jason.rutherg...@gmail.com] Sent: Friday, June 19, 2009 12:33 AM To: Alan Bateman Cc: java-dev@lucene.apache.org; nio-disc...@openjdk.java.net Subject: Re: madvise(ptr, len, MADV_SEQUENTIAL) Hmm... So the list at the bottom of this page looks accurate? http://www.gnu.org/software/hello/manual/gnulib/posix_005ffadvise.html If it is, then posix_fadvise works on Linux only? Perhaps madvise will be better then (judging by the much smaller unsupported list)? It seems to run on most platforms: http://www.gnu.org/software/hello/manual/gnulib/madvise.html On Wed, Jun 17, 2009 at 2:19 AM, Alan Bateman <alan.bate...@sun.com> wrote: Jason Rutherglen wrote: Alan, Do you think something like FileDescriptor.setAdvise (mirroring posix_fadvise) makes sense? -J Something like a posix_fadvise would be more appropriate for FileChannel or maybe as a usage hint when opening the file (the new APIs for opening files are extensible to allow for additional options in the future or even implementation specific options). I don't think we've had much interest in doing this, maybe because it would be a no-op on many operating systems. -Alan.
