I have seen this discussed before but with no conclusion. It is safe to say that SegmentReader.isDeleted is a synchronization bottleneck. When using a single IndexReader per query for highly concurrent application such as a web application, with the index entirely in the system cache, the maximum the CPU goes to is 75%-80% on a dual 2 core server. I have not tested on a dual quad core but would expect more significant performance degradation. What is the use case for SegmentReader.isDeleted being synchronized? Most Lucene applications are batch oriented and so performing in memory updates that need to be reflected immediately in a search is incongruent. I am sure this is supposed to be taken care of with the deprecation of IndexReader.deleteDocument. I do not agree with deprecating IndexReader.deleteDocument as IndexWriter does not offer a way to work directly at the doc level. If the system has a mix of DirectoryIndexReaders and non-DirectoryIndexReaders (such as InstantiatedIndexReader) then IndexWriter will not work.
Are there any other unnecessary synchronization points related to IndexReader?
