DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG· RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT <http://issues.apache.org/bugzilla/show_bug.cgi?id=35838>. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND· INSERTED IN THE BUG DATABASE.
http://issues.apache.org/bugzilla/show_bug.cgi?id=35838 Summary: Java NIO patch against Lucene 1.9 Product: Lucene Version: unspecified Platform: All OS/Version: All Status: NEW Severity: normal Priority: P2 Component: Store AssignedTo: [email protected] ReportedBy: [EMAIL PROTECTED] Robert Engels previously submitted a patch against Lucene 1.4 for a Java NIO- based Directory implementation. It also included some changes to FSDirectory to allow better concurrency when searching from multiple threads. The complete thread is at: http://mail-archives.apache.org/mod_mbox/lucene-java-dev/200505.mbox/% [EMAIL PROTECTED] This thread ended with Doug Cutting suggesting that someone port Robert's changes to the SVN trunk. This is what I've done in this patch. There are two parts to the patch. The first part modifies FieldsReader, CompoundFileReader, and SegmentReader, to allow better concurrency when reading an index. The second part includes the new NioFSDirectory implementation, and makes small changes to FSDirectory and IndexInput to accomodate this change. I'll put a more detailed outline of the changes to each file in a separate message. To use the new NioFSDirectory, set the system property org.apache.lucene.FSDirectory.class to org.apache.lucene.store.NioFSDirectory. This will cause FSDirectory.getDirectory() to return an NioFSDirectory instance. By default, NioFile limits the number of concurrent channels to 4, but you can override this by setting the system property org.apache.lucene.nio.channels. I did some performance tests with these patches. The biggest improvement came from the concurrency improvements. NioFSDirectory performed about the same as FSDirectory (with the concurrency improvements). I ran my tests under Fedora Core 1; uname -a reports: Linux myhost 2.4.22-1.2199.nptlsmp #1 SMP Wed Aug 4 11:48:29 EDT 2004 i686 i686 i386 GNU/Linux The machine is a dual xeon 2.8GHz with 4GB RAM, and the tests were run against a 9GB compound index file. The tests were run "hot" -- with everything already cached by linux's filesystem cache. The numbers are: FSDirectory without patch: 13.3 searches per second FSDirectory WITH concurrency patch: 14.3 searches per second Both tests were run with 6 concurrent threads, which gave the highest numbers in each case. I suspect that the concurrency improvements would make a bigger difference on a more realistic test where the index isn't all cached in RAM already, since the I/O happens whild holding the sychronized lock. Patches to follow... Thoughts? -- Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are the assignee for the bug, or are watching the assignee. --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
