[
https://issues.apache.org/jira/browse/LUCENE-1335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12625079#action_12625079
]
Michael McCandless commented on LUCENE-1335:
--------------------------------------------
{quote}
> It's because commit() calls prepareCommit(), which throws a
> "prepareCommit was already called" exception if the commit was already
> prepared. Whereas commit(long) doesn't call prepareCommit (eg, it
> doesn't need to flush). Without this, I was hitting exceptions in one
> of the tests that calls commit() from multiple threads at the same
> time.
Is it better to simplify things by serializing all commit()/commit(long) calls?
{quote}
I don't think so: with autoCommit=true, the merges calls commit(long)
after finishing, and I think we want those commit calls to run
concurrently?
{quote}
> This is to make sure any just-started addIndexes cleanly finish or
> abort before we enter the wait loop. I was seeing cases where the
> wait loop would think no more merges were pending, but in fact an
> addIndexes was just getting underway and was about to start merging.
> It's OK if a new addIndexes call starts up, because it'll be forced to
> check the stop conditions (closing=true or stopMerges=true) and then
> abort the merges. I'll add comments to this effect.
I wonder if we can simplify the logic... Currently in setMergeScheduler,
merges can start between finishMerges and set the merge scheduler.
This one can be fixed by making setMergeScheduler synchronized.
{quote}
Good catch -- I'll make setMergeScheduler synchronized.
{quote}
> This method has always carried out merges in the FG, but it's in fact
> possible that a BG merge thread on finishing a previous merge may pull
> a merge involving external segments. So I changed this method to wait
> for all such BG merges to complete, because it's not allowed to return
> until there are no more external segments in the index.
Hmm... so merges involving external segments may be in FG or BG?
So copyExternalSegments not only copies external segments, but also
waits for BG merges involving external segments to finish. We need
a better name?
{quote}
Sure we can change the name -- do you have one in mind? Maybe
"resolveExternalSegments" or "waitForExternalSegments"?
{quote}
> It is tempting to fully schedule these external merges (ie allow them
> to run in BG), but there is a problem: if there is some error on doing
> the merge, we need that error to be thrown in the FG thread calling
> copyExternalSegments (so the transcaction above unwinds). (Ie we
> can't just stuff these external merges into the merge queue then wait
> for their completely).
Then what about those BG merges involving external segments?
{quote}
What'll happen is the BG merge will hit an exception, roll itself
back, and then the FG thread will pick up the merge and try again.
Likely it'll hit the same exception, which is then thrown back to the
caller. It may not hit an exception, eg say it was disk full: the BG
merge was probably trying to merge 10 segments, whereas the FG merge
is just copying over the 1 segment. So it may complete successfully
too.
> Correctly handle concurrent calls to addIndexes, optimize, commit
> -----------------------------------------------------------------
>
> Key: LUCENE-1335
> URL: https://issues.apache.org/jira/browse/LUCENE-1335
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Affects Versions: 2.3, 2.3.1
> Reporter: Michael McCandless
> Assignee: Michael McCandless
> Priority: Minor
> Fix For: 2.4
>
> Attachments: LUCENE-1335.patch, LUCENE-1335.patch, LUCENE-1335.patch,
> LUCENE-1335.patch
>
>
> Spinoff from here:
>
> http://mail-archives.apache.org/mod_mbox/lucene-java-dev/200807.mbox/[EMAIL
> PROTECTED]
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
