Thanks Mike for the clarification.
Almost all merge methods are synced on IndexWriter, except for mergeMiddle.
So it is possible that you call abort, after the merge thread has already
checked if the merge was aborted ...
Maybe if we do:
sync (this) {
merge.checkAborted(directory);
new SegmentMerger(...);
}
We'll protect against this particular exception? Because if merge.abort()
wasn't called yet, the sync clause makes sure it won't be called until we
get the SegmentMerger, otherwise we'll know the merge was aborted and so
won't create SM at all.
I looked at the rest of the method and this looks to be the only unprotected
place. Although it is a loooong method, so I may have missed a spot.
Shai
On Sun, Aug 2, 2009 at 8:45 PM, Michael McCandless <
luc...@mikemccandless.com> wrote:
> The exceptions aren't expected.
>
> When you call IndexWriter.close(false), it's supposed to notify all
> running merges to abort, wait until they in fact do stop, and then
> return.
>
> But it looks as if there's a silly thread hazard, whereby IndexWriter
> believes all merges have aborted, but in fact those 4 were just about
> to get started and "missed" the abort request, and then tried to start
> and hit the ACE.
>
> Mike
>
> On Sun, Aug 2, 2009 at 6:49 AM, Shai Erera<ser...@gmail.com> wrote:
> > Isn't this expected? I mean, if you close IndexWriter, but there are
> merges
> > run in the background, aren't they expected to hit an
> > AlreadyClosedException?
> >
> > Perhaps we should fix the test to check whether CMS hit this exception
> and
> > if so clear it (as AFAIU, it's expected), and leave any other exceptions?
> > This will require some change in CMS (to expose that information, through
> > e.g. getUnhandledExceptions).
> >
> > Or add to CMS.Thread.run() a catch for AlreadyClosedException and ignore
> it,
> > as we do for MergeAbortedException?
> >
> > Or, stop doing any merges if CMS.close() has been called?
> >
> > On Sun, Aug 2, 2009 at 12:49 AM, Michael Busch <busch...@gmail.com>
> wrote:
> >>
> >> While running 'ant test' I just got this failure (on Mac OS 10.5.7).
> Even
> >> worse, the next time I ran it the test passed, so this might be hard to
> >> reproduce.
> >> I haven't tried yet to find out what caused this.
> >>
> >> [junit] Exception in thread "Lucene Merge Thread #1"
> >> org.apache.lucene.index.MergePolicy$MergeException:
> >> org.apache.lucene.store.AlreadyClosedException: this IndexWriter is
> closed
> >> [junit] at
> >>
> org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:351)
> >> [junit] at
> >>
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:315)
> >> [junit] Caused by: org.apache.lucene.store.AlreadyClosedException:
> this
> >> IndexWriter is closed
> >> [junit] at
> >> org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:791)
> >> [junit] at
> >> org.apache.lucene.index.IndexWriter.getDirectory(IndexWriter.java:2303)
> >> [junit] at
> >> org.apache.lucene.index.SegmentMerger.<init>(SegmentMerger.java:88)
> >> [junit] at
> >> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4942)
> >> [junit] at
> >> org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4598)
> >> [junit] at
> >>
> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:235)
> >> [junit] at
> >>
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:291)
> >> [junit] Exception in thread "Lucene Merge Thread #2"
> >> org.apache.lucene.index.MergePolicy$MergeException:
> >> org.apache.lucene.store.AlreadyClosedException: this IndexWriter is
> closed
> >> [junit] at
> >>
> org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:351)
> >> [junit] at
> >>
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:315)
> >> [junit] Caused by: org.apache.lucene.store.AlreadyClosedException:
> this
> >> IndexWriter is closed
> >> [junit] at
> >> org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:791)
> >> [junit] at
> >> org.apache.lucene.index.IndexWriter.getDirectory(IndexWriter.java:2303)
> >> [junit] at
> >> org.apache.lucene.index.SegmentMerger.<init>(SegmentMerger.java:88)
> >> [junit] at
> >> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4942)
> >> [junit] at
> >> org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4598)
> >> [junit] at
> >>
> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:235)
> >> [junit] at
> >>
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:291)
> >> [junit] Exception in thread "Lucene Merge Thread #3"
> >> org.apache.lucene.index.MergePolicy$MergeException:
> >> org.apache.lucene.store.AlreadyClosedException: this IndexWriter is
> closed
> >> [junit] at
> >>
> org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:351)
> >> [junit] at
> >>
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:315)
> >> [junit] Caused by: org.apache.lucene.store.AlreadyClosedException:
> this
> >> IndexWriter is closed
> >> [junit] at
> >> org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:791)
> >> [junit] at
> >> org.apache.lucene.index.IndexWriter.getDirectory(IndexWriter.java:2303)
> >> [junit] at
> >> org.apache.lucene.index.SegmentMerger.<init>(SegmentMerger.java:88)
> >> [junit] at
> >> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4942)
> >> [junit] at
> >> org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4598)
> >> [junit] at
> >>
> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:235)
> >> [junit] at
> >>
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:291)
> >> [junit] Exception in thread "Lucene Merge Thread #4"
> >> org.apache.lucene.index.MergePolicy$MergeException:
> >> org.apache.lucene.store.AlreadyClosedException: this IndexWriter is
> closed
> >> [junit] at
> >>
> org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:351)
> >> [junit] at
> >>
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:315)
> >> [junit] Caused by: org.apache.lucene.store.AlreadyClosedException:
> this
> >> IndexWriter is closed
> >> [junit] at
> >> org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:791)
> >> [junit] at
> >> org.apache.lucene.index.IndexWriter.getDirectory(IndexWriter.java:2303)
> >> [junit] at
> >> org.apache.lucene.index.SegmentMerger.<init>(SegmentMerger.java:88)
> >> [junit] at
> >> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4942)
> >> [junit] at
> >> org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4598)
> >> [junit] at
> >>
> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:235)
> >> [junit] at
> >>
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:291)
> >> [junit] ------------- ---------------- ---------------
> >> [junit] Testcase:
> >> testNoWaitClose(org.apache.lucene.index.TestIndexWriter): FAILED
> >> [junit] ConcurrentMergeScheduler hit unhandled exceptions
> >> [junit] junit.framework.AssertionFailedError:
> ConcurrentMergeScheduler
> >> hit unhandled exceptions
> >> [junit] at
> >> org.apache.lucene.util.LuceneTestCase.tearDown(LuceneTestCase.java:54)
> >> [junit] Test org.apache.lucene.index.TestIndexWriter FAILED
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org
> >> For additional commands, e-mail: java-dev-h...@lucene.apache.org
> >>
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: java-dev-h...@lucene.apache.org
>
>
