On Fri, Jun 19, 2009 at 4:15 AM, Uwe Schindler<u...@thetaphi.de> wrote:
> But then we also need to map, when writing to files, which is hard to do, > because you do not know how large the mapping area will be (unknown filesize). It may not that important on writing to avoid the IO cache, in that at least we are writing a segment that will likely be consumed be an IndexReader (if the reader reopens quickly). Still, we would not want (typically) to cache every single byte for the new segment... so it would be still good to have a solution during writing. And, for apps where the reader is not reopened often, we shouldn't cache the bytes written. It's more important to consume IO cache with the bytes we are *reading* while merging, because those bytes are presumably not needed for much longer (unless old IndexReaders do not reopen). Ideally, we need the freedom to fully decouple the bytes read & written during merging from storage in the IO cache (which is what madvise/posix_fadvise would give us). > On windows, we then will also rely on the cleaner()-hack to correctly close > files (see current MMap in lucene-trunk, where the cleaner-hack can > optionally be enabled, without guaranties). This issue also affects unix, just in a different way. Ie on unix, even though the file name appears gone from the directory listing, because a file descriptor is still open, the file is still consuming disk space (until GC finally unmaps) -- "delete on last close". Mike --------------------------------------------------------------------- To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org For additional commands, e-mail: java-dev-h...@lucene.apache.org
