[
https://issues.apache.org/jira/browse/LUCENE-1516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12675863#action_12675863
]
Michael McCandless commented on LUCENE-1516:
--------------------------------------------
I think we can simplify the approach here:
* I don't think IW should track an "internal reader" at all.
Instead, it tracks a pool of held-open and re-used SegmentReader
instances. Maybe make a new SegmentReaderPool private class. It
exposes get() to load a SegmentReader, returning the pooled one if
present, else falling back to the normal SegmentReader.get.
* Then, the many places where we currently pull a segment reader (to
apply deletes, to do a merge, to carry over new deletes after
merge finishes) as well as the new getReader() method, should use
this pool to get the reader.
* We would never mergeSegmentInfos, never ask the "internal reader"
to commit, etc.
Other things:
* Since IndexReader will always be readonly, you can simplify the
new DirectoryIndexReader.open method, eg we don't need to copy
over the writeLock nor the DeletionPolicy, and we don't need the
readOnly param, and closeDirectory is always false. In fact we
could simply create a ReadOnly{Multi,}SegmentReader directly.
* I think IndexReader.clone() should not grab the writer's
segmentInfos? Ie it should truly be a clone(), not a reopen.
Actually, when the reader is created we should make a private full
copy of the SegmentInfos.
* We lost some "private" modifiers, unecessarily; eg
DirectoryIndexReader.writeLock.
* I don't understand why we need merge.segmentReadersClone? If we
only use to detect new deletions for carrying over deletes after
merge finishes, we should instead just grab the delCount when the
merge kicked off?
* I think the only reason for reader to hold a reference to writer
is so that on reopen, the reader realizes it was created from
IW.getReader and simply forwards the request to IW. Otherwise
writer should not be used in reader.
* Once we no longer store/maintain/merge an "internal" reader,
IW.getReader simply becomes a call to IndexReader.open, with two
differences: 1) we pass in a SegmentInfos instance (instead of
looking for last segments_N in dir), and 2) we pass in our own
SegmentReaderPool that should be used to open the SR's.
* You need to add mainDir.close() in
testAddIndexesAndDoDeletesThreads.
> Integrate IndexReader with IndexWriter
> ---------------------------------------
>
> Key: LUCENE-1516
> URL: https://issues.apache.org/jira/browse/LUCENE-1516
> Project: Lucene - Java
> Issue Type: Improvement
> Affects Versions: 2.4
> Reporter: Jason Rutherglen
> Assignee: Michael McCandless
> Priority: Minor
> Fix For: 2.9
>
> Attachments: LUCENE-1516.patch, LUCENE-1516.patch, LUCENE-1516.patch,
> LUCENE-1516.patch, LUCENE-1516.patch, LUCENE-1516.patch, LUCENE-1516.patch,
> LUCENE-1516.patch, LUCENE-1516.patch
>
> Original Estimate: 672h
> Remaining Estimate: 672h
>
> The current problem is an IndexReader and IndexWriter cannot be open
> at the same time and perform updates as they both require a write
> lock to the index. While methods such as IW.deleteDocuments enables
> deleting from IW, methods such as IR.deleteDocument(int doc) and
> norms updating are not available from IW. This limits the
> capabilities of performing updates to the index dynamically or in
> realtime without closing the IW and opening an IR, deleting or
> updating norms, flushing, then opening the IW again, a process which
> can be detrimental to realtime updates.
> This patch will expose an IndexWriter.getReader method that returns
> the currently flushed state of the index as a class that implements
> IndexReader. The new IR implementation will differ from existing IR
> implementations such as MultiSegmentReader in that flushing will
> synchronize updates with IW in part by sharing the write lock. All
> methods of IR will be usable including reopen and clone.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-dev-h...@lucene.apache.org
