[
https://issues.apache.org/jira/browse/LUCENE-5885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shai Erera updated LUCENE-5885:
-------------------------------
Attachment: LUCENE-5885.patch
Patch fixes the bug - finishMerges(false) is more robust now, and waits for all
runningMerges to abort, ignoring ThreadInterruptedExceptions (but restoring in
the end).
But, our assumption that this is enough to release a MergeScheduler's resources
was wrong. What happens is that IW waits until all *runningMerges* are done,
but that doesn't mean all *MergeThreads* have died. So under some cases, this
is what can happen (CTX = context switch):
{noformat}
IW.finishMerges(false) calls merge.abort()
CTX
MergeThread.merge() hits a MergeAbortedException (expected)
MergeThread.merge() enters finally {} and calls IW.mergeFinish
IW.mergeFinish removes that merge from runningMerges
CTX
runningMerges.size() == 0, so thread exits finishMerges()
rollbackInternal continues, finishes
test continues, finishes (NOTE: MergeThread is still alive, as it didn't exit
yet!!)
test-framework complains about thread leak
{noformat}
So I now wonder if we should add a ref-counting mechanism to MS. It's not
simple, we cannot just add a decRef(), we need a decRef(IndexWriter) and then
in CMS we need to handle only the MergeThreads that are associated with this
IndexWriter instance. This may be easy for CMS, but is it a good API for our
users? Is it clear that they need to handle only the resources that are
associated with that IW instance?
Thoughts?
> MergeScheduler should not implement Closeable
> ---------------------------------------------
>
> Key: LUCENE-5885
> URL: https://issues.apache.org/jira/browse/LUCENE-5885
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/index
> Reporter: Shai Erera
> Attachments: LUCENE-5885.patch, LUCENE-5885.patch
>
>
> MergeScheduler implements Closeable and IndexWriter calls ms.close() when
> it's closed. But MergeScheduler can be shared between several writers, which
> means closing it by any particular writer is wrong. We should rather
> implement some ref-counting logic such that each IW will call incRef() in the
> ctor, and decRef() on close(), and MergeScheduler will truly close when the
> ref-count hits 0.
> As it is now, if you share a MergeScheduler between writers and close() does
> something terminating, I doubt if it really works.
> Also, when I look at ConcurrentMergeScheduler.close(), it calls sync() which
> joins all MergeThreads. But if that CMS instance is shared between few IWs,
> doesn't it mean that a single IW calling close() waits on MergeThreads that
> execute merges of other IWs!?!? This seems ... wrong?
--
This message was sent by Atlassian JIRA
(v6.2#6252)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
