Hi Michael (directly CC'd this time...),
Maybe you're not subscribed to the list? Your first email got some
responses, eg:
http://lucene.markmail.org/thread/lrv7miivzmjm3ml5
Net/net, these new directory implementations sound exciting!
Mike McCandless
http://blog.mikemccandless.com
On Sun, Mar 17, 2013 at 4:03 PM, Michael Poindexter
<[email protected]> wrote:
> As part of a project using Lucene I have implemented a trio of Directories
> roughly corresponding to the FSDirectory implementations in core. These
> directory implementations use the NIO2 API's in JDK7 when opening files.
> This ensures that on Windows the files are opened in a mode that allows
> deletion even if the file is open elsewhere.
>
> 1.) JDK7MMapDirectory - Roughly the same as MMapDirectory. Uses
> FileChannel.open (instead of RandomAccessFile) to create a FileChannel that
> then has map() called on it to create the mapped buffers.
> 2.) JDK7NIOFSDirectory - Roughly the same as NIOFSDirectory, but uses
> FileChannel.open to create the file channel instead of RandomAccessFile.
> 3.) JDK7AsyncFSDirectory - This one is new and different. I needed a
> replacement for SimpleFSDirectory (that was not susceptible to problems if
> interrupt()'ed) and did not have the synchronization problems on Windows of
> NIOFSDirectory. This one is used where SimpleFSDirectory could have been
> used, but uses an AsynchronousFileChannel to do it's work. The actual
> operation is still synchronous, but on Windows AsynchronousFileChannel uses
> overlapped IO, and hence does not require synchronization on the position
> and should be safe for interrupts.
>
> A couple of questions:
> 1.) Is there any interest in me contributing these to Lucene? They
> require JDK7+, but perhaps they could go in a contrib module?
>
> 2.) While implementing these I noticed the implementation of
> FSDirectory.sync seems a little strange: It just opens a new
> RandomAccessFile and forces a sync using it. The JavaDocs seem to imply
> that this would force a sync on the file handle associated with the
> RandomAccessFile, but that's not the file handle that was written to as
> part of an IndexOutput. On Windows at least this won't matter, but it
> seems theoretically wrong...i.e. according to the JavaDoc on a given
> platform this style of operation could have no impact if I am understanding
> it correctly. It seems like maybe it would be better to have a sync() call
> on an IndexOutput that can be called before closing it...am I missing
> something here?
>
> 3.) What is the best way to go about benchmarking/testing these new
> implementations to compare against the core FSDirectory implementations?
> I've seen some references to randomized tests and benchmarks on the
> developer pages on the Lucene website, but I didn't see anything that was
> along the lines of "Here's how to run the benchmarks"...any pointers would
> be much appreciated.
>
> Thanks,
> Mike Poindexter
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
