[
https://issues.apache.org/jira/browse/LUCENE-1703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12722003#action_12722003
]
Tim Smith commented on LUCENE-1703:
-----------------------------------
MergeScheduler does not provide a sync() method in its abstract class.
Therefore, an implementation could be used that would not derive from or
provide a sync() method
If it did, this would be a more than acceptable solution instead of putting the
method on the IndexWriter. (i potentially want to allow configuration to
specify what MergeScheduler to use, and if sync() isn't specified on the base
class, i can't reliably use it (will have to do instanceof checks, and a new
concurrent based scheduler not derived from ConcurrentMergeScheduler breaks
this)
Also, the SerialMergeScheduler may be in use and be running due to another
thread calling addDocument triggering a merge, but i want my second thread to
wait for that merge to complete (this would require implementing sync() on
SerialMergeScheduler to allow other threads to sync on the merge as well)
optimize() is not a viable option to use to wait for merges as this could take
a long time (hours)
close() requires reopening the IndexWriter (which i am trying to avoid in the
first place)
expungeDeletes(true) is doing more work that is actually required (this could
be a viable solution, however its a bit odd to "waitForMerges()" by calling a
method called expungeDeletes())
I will try getting access to the article you suggest, however i still expect i
will want this method at the end of the day (and will end up using
expungeDeletes(true) if it never gets added (with copious comments indicating
that expungeDeletes() is actually being used to wait for background merges)
Another use for waitForMerges() is as a more lightweight version of optimize().
this will allow blocking adding more documents until background merges are
complete periodically, preventing from throttling the IO on the indexer from
being hit too hard. Could call IndexWriter.maybeMerge() followed by
IndexWriter.waitForMerges()
> Add a waitForMerges() method to IndexWriter
> -------------------------------------------
>
> Key: LUCENE-1703
> URL: https://issues.apache.org/jira/browse/LUCENE-1703
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Affects Versions: 2.4
> Reporter: Tim Smith
> Attachments: IndexWriter.java.diff, IndexWriter.java.diff
>
>
> It would be very useful to have a waitForMerges() method on the IndexWriter.
> Right now, the only way i can see to achieve this is to call
> IndexWriter.close()
> ideally, there would be a method on the IndexWriter to wait for merges
> without actually closing the index.
> This would make it so that background merges (or optimize) can be waited for
> without closing the IndexWriter, and then reopening a new IndexWriter
> the close() reopen IndexWriter method can be problematic if the close() fails
> as the write lock won't be released
> this could then result in the following sequence:
> * close() - fails
> * force unlock the write lock (per close() documentation)
> * new IndexWriter() (acquires write lock)
> * finalize() on old IndexWriter releases the write lock
> * Index is now not locked, and another IndexWriter pointing to the same
> directory could be opened
> If you don't force unlock the write lock, opening a new IndexWriter will fail
> until garbage collection calls finalize() the old IndexWriter
> If the waitForMerges() method is available, i would likely never need to
> close() the IndexWriter until right before the process being shutdown, so
> this issue would not occur (worst case scenario, the waitForMerges() fails)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-dev-h...@lucene.apache.org
